I would like to reload the page whenever the page doesn't load properly or encounters a problem. I tried page.reload()
but it doesn't work.
for(const sect of sections ){
// Now collect all the URLs
const appUrls = await page.$$eval('div.main > ul.app-list > li > div.app-info a.app-info-icon', links => links.map(link => link.href));
// Visit each URL one by one and collect the data
for (let appUrl of appUrls) {
var count = i++;
try{
await page.goto(appUrl);
const appName = await page.$eval('div.det-name-int', div => div.innerText.trim());
console.log('\n' + count);
console.log(appName);
} catch(e){
console.log('\n' + count);
console.log('ERROR', e);
await page.reload();
}
}
}
It gives me this error:
ERROR Error: Error: failed to find element matching selector "div.det-name-int"
at ElementHandle.$eval (C:\Users\Administrator\node_modules\puppeteer\lib\JS
Handle.js:418:13)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at ElementHandle.<anonymous> (C:\Users\Administrator\node_modules\puppeteer\
lib\helper.js:108:27)
at DOMWorld.$eval (C:\Users\Administrator\node_modules\puppeteer\lib\DOMWorl
d.js:149:21)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at Frame.<anonymous> (C:\Users\Administrator\node_modules\puppeteer\lib\help
er.js:108:27)
at Page.$eval (C:\Users\Administrator\node_modules\puppeteer\lib\Page.js:329
:29)
at Page.<anonymous> (C:\Users\Administrator\node_modules\puppeteer\lib\helpe
r.js:109:23)
at main (C:\Users\Administrator\Desktop\webscrape\text.js:35:43)
at process._tickCallback (internal/process/next_tick.js:68:7)
Some links are unable to load successfully. When I refresh those pages manually, it works. So I hope there is a function or a method that can help me reload my page automatically when there is an error.
This works for me:
await page.reload({ waitUntil: ["networkidle0", "domcontentloaded"] });
See Puppeteer docs for details: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagereloadoptions