Top "Screen-scraping" questions

Screen-scraping, also known as web-scraping or data-scraping, is a software technique used to collect and parse information from user interfaces.

Scraping data from all asp.net pages with AJAX pagination implemented

I want to scrap a webpage containing a list of user with addresses, email etc. webpage contain list of user …

php asp.net curl web-scraping screen-scraping
Maintaining cookies between Mechanize requests

I'm trying to use the Ruby version of Mechanize to extract my employer's tickets from a ticket management system that …

ruby screen-scraping mechanize
Trying to get authentication cookie(s) using HttpWebRequest

I have to scrape a table from a secure site and I'm having trouble logging in to the page and …

c# httpwebrequest screen-scraping webclient httpwebresponse
Click on a javascript link within python?

I am navigating a site using python's mechanize module and having trouble clicking on a javascript link for next page. …

javascript python screen-scraping mechanize spidermonkey
CasperJS passing data back to PHP

CasperJS is being called by PHP using an exec() command. After CasperJS does its work such as retrieving parts of …

php web-scraping screen-scraping phantomjs casperjs
Puppeteer waitForSelector on multiple selectors

I have Puppeteer controlling a website with a lookup form that can either return a result or a "No records …

javascript puppeteer screen-scraping
What's the best approach for parsing XML/'screen scraping' in iOS? UIWebview or NSXMLParser?

I am creating an iOS app that needs to get some data from a web page. My first though was …

iphone ios uiwebview screen-scraping nsxmlparser
Super-fast screen scraping techniques?

I often find myself needing to do some simple screen scraping for internal purposes (i.e. a third party service …

html screen-scraping
Screen scraping with Python

Does Python have screen scraping libraries that offer JavaScript support? I've been using pycurl for simple HTML requests, and Java's …

python screen-scraping htmlunit pycurl
The way to detect web scraping

I need to detect scraping of info on my website. I tried detection based on behavior patterns, and it seems …

algorithm security screen-scraping detection