I want to use nodeJS as tool for website scrapping. I have already implemented a script which logs me in on the system and parse some data from the page.
我想使用nodeJS作为网站报废的工具。我已经实现了一个脚本，它将我登录到系统并从页面解析一些数据。
The steps are defined like:
步骤定义如下：
-
Open login page
打开登录页面
-
Enter login data
输入登录数据
-
Submit login form
提交登录表单
-
Go to desired page
转到所需的页面
-
Grab and parse values from the page
从页面中抓取并解析值
-
Save data to file
将数据保存到文件
-
Exit
出口
Obviously, the problem is that every time my script has to login, and I want to eliminate that. I want to implement some kind of cookie management system, where I can save cookies to .txt file, and then during next request I can load cookies from file and send it in request headers.
显然，问题是每次我的脚本必须登录，我想消除它。我想实现某种cookie管理系统，我可以将cookie保存到.txt文件，然后在下一个请求中我可以从文件中加载cookie并将其发送到请求标头中。
This kind of cookie management system is not hard to implement, but the problem is how to access cookies in nodejs? The only way I found it is using request response object, where you can use something like this:
这种cookie管理系统并不难实现，但问题是如何访问nodejs中的cookie？我发现它的唯一方法是使用请求响应对象，您可以使用以下内容：
request.get({headers:requestHeaders,uri: user.getLoginUrl(),followRedirect: true,jar:jar,maxRedirects: 10,},function(err, res, body) {
if(err) {
console.log('GET request failed here is error');
console.log(res);
}
//Get cookies from response
var responseCookies = res.headers['set-cookie'];
var requestCookies='';
for(var i=0; i<responseCookies.length; i++){
var oneCookie = responseCookies[i];
oneCookie = oneCookie.split(';');
requestCookies= requestCookies + oneCookie[0]+';';
}
}
);
Now content of variable requestCookies can be saved to the .txt file and can loaded next time when script is executed, and this way you can avoid process of logging in user every time when script is executed.
现在，变量requestCookies的内容可以保存到.txt文件中，并且可以在下次执行脚本时加载，这样就可以避免每次执行脚本时登录用户的过程。
Is this the right way, or there is a method which returns cookies?
这是正确的方法，还是有一种方法可以返回cookie？
NOTE: If you want to setup your request object to automatically resend received cookies on every subsequent request, use the following line during object creation:
注意：如果要将请求对象设置为在每个后续请求中自动重新发送收到的cookie，请在对象创建期间使用以下行：
var request = require("request");
request = request.defaults({jar: true});//Send cookies on every subsequent requests
1 个解决方案
#1
4
In my case, i've used 'http'library like the following:
在我的情况下，我使用了'http'library，如下所示：
http.get(url, function(response) {
variable = response.headers['set-cookie'];
})
#1
4
In my case, i've used 'http'library like the following:
在我的情况下，我使用了'http'library，如下所示：
http.get(url, function(response) {
variable = response.headers['set-cookie'];
})