fetch url with cookies

Question

so im scraping a web and i need to use a specific cookies but i dont know how to exactly use "fetch"

    const url="https://www.example.com";
    let response = await fetch(url),
        html = await response.text();

    let $ = cheerio.load(html)

    var example= $('.exampleclass').text();

Now i can scrape the web but in case i would have to use a specific cookies i dont know how to put in on the fetch.

In python was something like that

response = requests.get(url, headers=headers, cookies=cookies)

Thank you!

Which module are you using? [node-fetch](https://www.npmjs.com/package/node-fetch)? — ChrisG, Mar 16 '21 at 08:37
You can refer to this [this previous post](https://stackoverflow.com/questions/34558264/fetch-api-with-cookie) — fatiu, Mar 16 '21 at 08:38
@Muhammad.fatiu That question is about in-browser fetch; OP's question is about node. — ChrisG, Mar 16 '21 at 08:40
Anyway, to set a cookie you need to add a [Cookie header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cookie): something like `await fetch(url, { headers: { "Cookie": "a=b" }})` should do it. — ChrisG, Mar 16 '21 at 08:50
FYI it's __scrape__ and __scraping__ not scrap and scrapping — DisappointedByUnaccountableMod, Mar 16 '21 at 11:03

score 0 · Answer 1 · answered Mar 16 '21 at 09:09

You can add the cookies on the headers on node-fetch, I've made a helper function that you can use for your purposes:

const cookieMaker = object => {
    const cookie = [];
    for (const [key, value] of Object.entries(object)) {
        cookie.push(`${key}=${value}`);
    }
    return cookie.join('; ');
};

const fetchText = async (url, cookie) => {
    const r = await fetch(url, {
        headers: {
            Cookie: cookieMaker(cookie),
        },
    });
    return await r.text();
};

fetchText('http://someurl.com', { token: 'abc', myValue: 'def' });

fetch url with cookies

1 Answers1