我是python web清理的新手,我想从中筛选出排名前100的工作结果,而I m只能从第一页中筛选出排名前10的结果。我用的是美组框架。这是我的代码,有人能帮我解决这个问题吗?
import urllib2
from bs4 import BeautifulSoup
import json
URL = "https://www.indeed.co.in/jobs?q=software+developer&l=Bengaluru%2C+Karnataka"
soup = BeautifulSoup(urllib2.urlopen(URL).read(), 'html.parser')
results = soup.find_all('div', attrs={'class': 'jobsearch-SerpJobCard'})
for x in results:
company = x.find('span', attrs={"class":"company"})
print 'company:', company.text.strip()
job = x.find('a', attrs={'data-tn-element': "jobTitle"})
print 'job:', job.text.strip()
