def search_list(self,response):
content_url = self.getContentURL(response)
article_title = response.xpath('//*[@id="RECORD_1"]/div[3]/div[1]/div/a/value//text()').extract()
new_content_url = urlparse.urljoin(response.url,content_url)
print article_title
print new_content_url
if new_content_url:
yield scrapy.Request(url=new_content_url,dont_filter=True,callback=self.parse)
def parse(self,response):
pass
调试时,发现上一步中拿到的new_content_url没有错误,可是怎么都跳转不进去parse函数里,allowed_domains设置正确,参数dont_filter也在,实在不知道问题出现在哪里