Issue with scraping a list to get href using Goutte and PHP

php web-scraping goutte
Oliver Bayes-Shelton picture Oliver Bayes-Shelton · Mar 16, 2015 · Viewed 7.5k times · Source

I am trying to scrape the following, I basically want the text and the link, I am using Goutte with PHP. I can get the text fine using the following code but I cannot get the href value. Any help would be amazing.

$crawler->filter('#most-popular > div > ol > li > a')->each(function ($node) {
    var_dump($node->getAttribute('href'));
});


<li class="first-child ol1">
  <a href="http://www.bbc.co.uk/news/uk-england-south-yorkshire-31895703" class="story">
    <span class="livestats-icon livestats-1">1: </span>MP claims £17 poppy wreath expenses</a>
</li>

Answer

Burak picture Burak · Feb 23, 2016

getAttribute() is implemented as attr() within the Crawler class.

$crawler->filter('#most-popular > div.panel.open > ol > li.first-child.ol1 > a')->each(function ($node) {
    var_dump($node->attr('href'));
});

Related questions

How to scrape in laravel 5.2 using Goutte?

I am new to Laravel 5.2, I want to scrape a web page. I came to know that it can be done by using Goutte. And in don't know how to use it. I have installed Laravel and Goutte, But how …

Read More
Can Goutte/Guzzle be forced into UTF-8 mode?

I'm scraping from a UTF-8 site, using Goutte, which internally uses Guzzle. The site declares a meta tag of UTF-8, thus: <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> However, the content type header is thus: Content-Type: text/…

Read More
Web scraping in PHP

I'm looking for a way to make a small preview of another page from a URL given by the user in PHP. I'd like to retrieve only the title of the page, an image (like the logo of the website) …

Read More