Crawling and Scraping iTunes App Store

language-agnostic itunes screen-scraping web-crawler
Senseful picture Senseful · Jun 23, 2010 · Viewed 16.9k times · Source

I noticed that iTunes preview allows you to crawl and scrape pages via the http:// protocol. However, many of the links are trying to be opened in iTunes rather than the browser. For example, when you go to the iBooks page, it immediately tries opening a url with an itms:// protocol.

Are there any other methods of crawling the App Store or is this the only way?

Can the itms:// protocol links themselves be crawled somehow?

Answer

philipp picture philipp · Oct 2, 2010

I would have a decent look at the iTunes Search API and the iTunes Enterprise Partner API

You might get most/all of the information you need in a nice JSON file format.

If you can't get the information you need with the API, I would be interested what it is :)

Related questions

What and where are the stack and heap?

Programming language books explain that value types are created on the stack, and reference types are created on the heap, without explaining what these two things are. I haven't read a clear explanation of this. I understand what a stack …

Read More
What is dependency injection?

There have been several questions already posted with specific questions about dependency injection, such as when to use it and what frameworks are there for it. However, What is dependency injection and when/why should or shouldn't it be used?

Read More
Regex to match only letters

How can I write a regex that matches only letters?

Read More