Scraping html tables into R data frames using the XML package

html r xml parsing web-scraping
Eduardo Leoni picture Eduardo Leoni · Sep 8, 2009 · Viewed 116.3k times · Source

How do I scrape html tables using the XML package?

Take, for example, this wikipedia page on the Brazilian soccer team. I would like to read it in R and get the "list of all matches Brazil have played against FIFA recognised teams" table as a data.frame. How can I do this?

Answer

user225056 picture user225056 · Dec 4, 2009

…or a shorter try:

library(XML)
library(RCurl)
library(rlist)
theurl <- getURL("https://en.wikipedia.org/wiki/Brazil_national_football_team",.opts = list(ssl.verifypeer = FALSE) )
tables <- readHTMLTable(theurl)
tables <- list.clean(tables, fun = is.null, recursive = FALSE)
n.rows <- unlist(lapply(tables, function(t) dim(t)[1]))

the picked table is the longest one on the page

tables[[which.max(n.rows)]]

Related questions

2 column Section in R Markdown

I'm very new at R Markdown and I'm putting together an R Markdown HTML page for some new R users at my work to give them an intro and walk them through some simple demos. While showing off things like …

Read More
How can I read and parse the contents of a webpage in R

I'd like to read the contents of a URL (e.q., http://www.haaretz.com/) in R. I am wondering how I can do it

Read More
Include HTML files in R Markdown file?

Quick Summary How do I place HTML files in place within an R Markdown file? Details I have created some nice animated choropleth maps via choroplethr. As the link demonstrates, the animated choropleths function via creating a set of PNG …

Read More