Top "Screen-scraping" questions

Screen-scraping, also known as web-scraping or data-scraping, is a software technique used to collect and parse information from user interfaces.

Screen Scraping HTML with C#

I have been given the task at work of screen scraping one of our legacy web apps to extract certain …

c# screen-scraping
Scraping in Python - Preventing IP ban

I am using Python to scrape pages. Until now I didn't have any complicated issues. The site that I'm trying …

python selenium web-scraping screen-scraping
unable to call firefox from selenium in python on AWS machine

I am trying to use selenium from python to scrape some dynamics pages with javascript. However, I cannot call firefox …

python selenium amazon-web-services screen-scraping web-scraping
Protection from screen scraping

Following on from my question on the Legalities of screen scraping, even if it's illegal people will still try, so: …

screen-scraping
How do screen scrapers work?

I hear people writing these programs all the time and I know what they do, but how do they actually …

screen-scraping web-scraping html-content-extraction pdf-scraping console-scraping
Headless, scriptable Firefox/Webkit on linux?

I'm looking to automate some web interactions, namely periodic download of files from a secure website. This basically involves entering …

firefox webkit screen-scraping headless-browser
Nokogiri, open-uri, and Unicode Characters

I'm using Nokogiri and open-uri to grab the contents of the title tag on a webpage, but am having trouble …

ruby unicode screen-scraping nokogiri open-uri
How to use Goutte

Issue: Cannot fully understand the Goutte web scraper. Request: Can someone please help me understand or provide code to help …

web-crawler screen-scraping goutte
PHP CSS Selector Library?

Is there a PHP class/library that would allow me to query an XHTML document with CSS selectors? I need …

php screen-scraping css-selectors
Rightmove API and scraping technical and legal

I'm looking to build an app using property data. Nestoria has a free API and rules of use and Zoopla …

api screen-scraping