Screen-scraping, also known as web-scraping or data-scraping, is a software technique used to collect and parse information from user interfaces.
I want to screen-scrape a web-site that uses JavaScript. There is mechanize, the programmatic web browser for Python. However, it (…
javascript python browser screen-scraping mechanizeI noticed that iTunes preview allows you to crawl and scrape pages via the http:// protocol. However, many of the …
language-agnostic itunes screen-scraping web-crawlerI need to set the referer url, before scraping a site, the site uses refering url based Authentication, so it …
screen-scraping scrapyI know this has been asked before but I can't find a good answer for node.js I need server-side …
javascript node.js screen-scrapingI've got a python web crawler and I want to distribute the download requests among many different proxy servers, probably …
python proxy screen-scraping web-crawler squidI need to scrape a site with python. I obtain the source html code with the urlib module, but I …
javascript python browser screen-scrapingHi I want to create a desktop app (c# prob) that scrapes or manipulates a form on a 3rd party …
programming-languages screen-scraping web-scrapingif a page has <div class="class1"> and <p class="class1">, then soup.findAll(True, 'class1…
python screen-scraping beautifulsoupProblem When screen-scraping a webpage using python one has to know the character encoding of the page. If you get …
python character-encoding screen-scraping urllib2 urllibI'm looking for a library that has functionality similar to Perl's WWW::Mechanize, but for PHP. Basically, it should allow …
php automation screen-scraping mechanize www-mechanize