Maaz

如果将代码包含在范围循环中,则可以执行此操作:

from bs4 import BeautifulSoup
import json
import urllib2

URL = "https://www.indeed.co.in/jobs?q=software+developer&l=Bengaluru%2C+Karnataka&start="

for i in range(0 , 100 , 10):
    soup = BeautifulSoup(urllib2.urlopen(URL+str(i)).read(), 'html.parser')
    results = soup.find_all('div', attrs={'class': 'jobsearch-SerpJobCard'})
    for x in results:
        company = x.find('span', attrs={"class":"company"})
        print 'company:', company.text.strip()

        job = x.find('a', attrs={'data-tn-element': "jobTitle"})
        print 'job:', job.text.strip()  

我想是因为你打电话给 find 方法的字符串。 为了获得正确的行为,您应该将不同的参数保存在mysql表的单独列中,或者您可以这样更改值:

rs = row[3].split(',')
rtag = rs[0].split('\'')[1]

rclass = dict()
rclass[rs[1].split('\'')[1]] = rs[1].split('\'')[3]

最后:

content = soup.find(rtag, rclass).get_text().strip()