simple html dom failed to open stream for a site

parsecer picture parsecer · May 25, 2016 · Viewed 8.4k times · Source

I'm trying to parse throught http://whatismyip.com page and get my location (state and country). The data seems to be inside <table class="table"> tags, so i'm looking for "table". But I get a mistake Warning: file_get_contents(https://whatismyip.com): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden in C:\xampp4\htdocs\scraping\libs\simple_html_dom.php on line 1081

Can't figure out what's wrong.

 <?php
        require_once('libs/simple_html_dom.php');
        $html=new simple_html_dom();

        $html->load_file('https://whatismyip.com');

        $element=$html->find("table");


    ?>

Answer

ShiraNai7 picture ShiraNai7 · May 25, 2016

That website is checking the User-Agent header of the request but PHP doesn't send any (by default). You'll have to "impersonate" a browser:

$context = stream_context_create(array(
    'http' => array(
        'header' => array('User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201'),
    ),
));

$html = file_get_contents('http://whatismyip.com/', false, $context);

// do what you want with the $html

A better (and faster) option would be to use some library for this. I've used GeoIP2-php before but I'm sure there are more.