How to get all links from the DOM?

Vega picture Vega · Mar 26, 2018 · Viewed 10.5k times · Source

According to https://github.com/GoogleChrome/puppeteer/issues/628, I should be able to get all links from < a href="xyz" > with this single line:

const hrefs = await page.$$eval('a', a => a.href);

But when I try a simple:

console.log(hrefs)

I only get:

http://example.de/index.html

... as output which means that it could only find 1 link? But the page definitely has 12 links in the source code / DOM. Why does it fail to find them all?

Minimal example:

Answer

Miguel Calder&#243;n picture Miguel Calderón · Mar 26, 2018

In your example code you're using page.$eval, not page.$$eval. Since the former uses document.querySelector instead of document.querySelectorAll, the behaviour you describe is the expected one.

Also, you should change your pageFunctionin the $$eval arguments:

const hrefs = await page.$$eval('a', as => as.map(a => a.href));