Accessing Metacritic API and/or Scraping

api scrape scraper
boblikesoup picture boblikesoup · Jan 6, 2016 · Viewed 14.4k times · Source

Does anybody know where documentation for the Metacritic api is/if it still works. There used to be a Metacritic API at https://market.mashape.com/byroredux/metacritic-v2#get-user-details which disappeared today.

Otherwise I'm trying to scrape the site myself but keeping getting a blocked by a 429 Slow down. I got data like 3 times this hour and haven't been able to get anymore in the last 20 minutes which is making testing difficult and application possibly useless. Please let me know if there's anything else I can be doing to scape I don't know about.

Answer

nao picture nao · Jan 17, 2016

I was using that API as well for an app I wrote a while ago. Looks like the creator removed it from Mashape. I just sent him an email to ask whether it'll be back up. I did find this scraper online. It only has a few endpoints but following the examples given you could easily add more. Let me know if you make any progress!

Edit: Looks like CBS requested it to be taken down. The ToS prohibits scraping:

[…] you agree not to do the following, or assist others to do the following: Engage in unauthorized spidering, “scraping,” data mining or harvesting of Content, or use any other unauthorized automated means to gather data from or about the Services;

Related questions

No 'Access-Control-Allow-Origin' header is present on the requested resource—when trying to get data from a REST API

I'm trying to fetch some data from the REST API of HP Alm. It works pretty well with a small curl script - I get my data. Now doing that with JavaScript, fetch and ES6 (more or less) seems to …

Read More
How do I make calls to a REST API using C#?

This is the code I have so far: using System; using System.Collections.Generic; using System.Linq; using System.Text; using System; using System.Net.Http; using System.Web; using System.Net; using System.IO; namespace ConsoleProgram { public class Class1 { …

Read More
How do you set the Content-Type header for an HttpClient request?

I'm trying to set the Content-Type header of an HttpClient object as required by an API I am calling. I tried setting the Content-Type like below: using (var httpClient = new HttpClient()) { httpClient.BaseAddress = new Uri("http://example.com/"); httpClient.DefaultRequestHeaders.…

Read More