How to maintain a request session in NodeJS

javascript node.js session request
Ryan picture Ryan · Nov 12, 2013 · Viewed 40.4k times · Source

I'm trying to use NodeJS to scrape a website that requires a login by POST. Then once I'm logged in I can access a separate webpage by GET.

The first problem right now is logging in. I've tried to use request to POST the login information, but the response I get does not appear to be logged in.

exports.getstats = function (req, res) {
    request.post({url : requesturl, form: lform}, function(err, response, body) {
        res.writeHeader(200, {"Content-Type": "text/html"});
        res.write(body);
        res.end();
    });
};

Here I'm just forwarding the page I get back, but the page I get back still shows the login form, and if I try to access another page it says I'm not logged in.

I think I need to maintain the client side session and cookie data, but I can find no resources to help me understand how to do that.

As a followup I ended up using zombiejs to get the functionality I needed

Answer

Peter Lyons picture Peter Lyons · Nov 12, 2013

You need to make a cookie jar and use the same jar for all related requests.

 var cookieJar = request.jar();
 request.post({url : requesturl, jar: cookieJar, form: lform}, ...

That should in theory allow you to scrape pages with GET as a logged-in user, but only once you get the actual login code working. Based on your description of the response to your login POST, that may not be actually working correctly yet, so the cookie jar won't help until you fix the problems in your login code first.

Related questions

Invalidating JSON Web Tokens

For a new node.js project I'm working on, I'm thinking about switching over from a cookie based session approach (by this, I mean, storing an id to a key-value store containing user sessions in a user's browser) to a …

Read More
Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED

I working with node.js by expressjs I try to store an account to session. So, i try to test to use session with code in expressjs var RedisStore = require('connect-redis')(express); app.use(express.bodyParser()); app.use(express.cookieParser()); …

Read More
How do sessions work in Express.js with Node.js?

Using Express.js, sessions are dead simple. I'm curious how they actually work though. Does it store some cookie on the client? If so, where can I find that cookie? If required, how do I decode it? I basically want …

Read More