用scrapy爬豆瓣的某些页面返回403错误:
解决方法:
在setting.py文件中增加USER_AGENT配置:
USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
用scrapy爬豆瓣的某些页面返回403错误:
解决方法:
在setting.py文件中增加USER_AGENT配置:
USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'