Top "Screen-scraping" questions

Screen-scraping, also known as web-scraping or data-scraping, is a software technique used to collect and parse information from user interfaces.

How does a site like kayak.com aggregate content?

Greetings, I've been toying with an idea for a new project and was wondering if anyone has any idea on …

api architecture screen-scraping aggregate
PhantomJS failing to open HTTPS site

I'm using the following code based on loadspeed.js example to open up a https:// site which requires http server …

https screen-scraping phantomjs
Scrape web page contents

I am developing a project, for which I want to scrape the contents of a website in the background and …

php curl web-scraping httprequest screen-scraping
How to fetch HTML in Java

Without the use of any external library, what is the simplest way to fetch a website's HTML content into a …

java html screen-scraping
jsoup posting and cookie

I'm trying to use jsoup to login to a site and then scrape information, I am running into in a …

java screen-scraping jsoup
Scraping and parsing Google search results using Python

I asked a question on realizing a general idea to crawl and save webpages. Part of the original question is: …

python screen-scraping web-scraping google-search-api
XPath to Parse "SRC" from IMG tag?

Right now I successfully grabbed the full element from an HTML page with this: //img[@class='photo-large'] for example it …

html parsing xpath screen-scraping
How to use the WebClient.DownloadDataAsync() method in this context?

My plan is to have a user write down a movie title in my program and my program will pull …

c# asynchronous download screen-scraping
Text Extraction from HTML Java

I'm working on a program that downloads HTML pages and then selects some of the information and write it to …

java html screen-scraping html-content-extraction text-extraction