What options does R have for reading files in the native SAS format, sas7bdat, into R?
The NCES Common Core, for example, contains an extensive repository of data files saved in this format. For concreteness, let's focus on trying to read in this file from LEA Universe in 1997-98, which contains education-agency-level demographics for entities in all states beginning A through I.
Here's a preview from SAS of the data:
What's the simplest way to bring this data in to my R environment? I don't have any version of SAS available and am not willing to pay, so simply converting it to .csv would be a hassle.
Answer
sas7bdat worked fine for all but one of the files I was looking at (specifically, this one); in reporting the error to the sas7bdat developer, Matthew Shotwell, he also pointed me in the direction of Hadley's haven package in R which also has a read_sas method.
This method is superior for two reasons:
1) It didn't have any trouble reading the above-linked file
2) It is much (I'm talking much) faster than read.sas7bdat. Here's a quick benchmark (on this file, which is smaller than the others) for evidence:
microbenchmark(times=10L,
read.sas7bdat("psu97ai.sas7bdat"),
read_sas("psu97ai.sas7bdat"))
Unit: milliseconds
expr min lq mean median uq max neval cld
read.sas7bdat("psu97ai.sas7bdat") 66696.2955 67587.7061 71939.7025 68331.9600 77225.1979 82836.8152 10 b
read_sas("psu97ai.sas7bdat") 397.9955 402.2627 410.4015 408.5038 418.1059 425.2762 10 a
That's right--haven::read_sas takes (on average) 99.5% less time than sas7bdat::read.sas7bdat.
minor update
I previously wasn't able to figure out whether the two methods produced the same data (i.e., that both have equal levels of fidelity with respect to reading the data), but have finally done so:
# Keep as data.tables
sas7bdat <- setDT(read.sas7bdat("psu97ai.sas7bdat"))
haven <- setDT(read_sas("psu97ai.sas7bdat"))
# read.sas7bdat prefers strings as factors,
# and as of now has no stringsAsFactors argument
# with which to prevent this
idj_factor <- sapply(haven, is.factor)
# Reset all factor columns as characters
sas7bdat[ , (idj_factor) := lapply(.SD, as.character), .SDcols = idj_factor]
# Check equality of the tables
all.equal(sas7bdat, haven, check.attributes = FALSE)
# [1] TRUE
However, note that read.sas7bdat has kept a massive list of attributes for the file, presumably a holdover from SAS:
str(sas7bdat)
# ...
# - attr(*, "column.info")=List of 70
# ..$ :List of 12
# .. ..$ name : chr "NCESSCH"
# .. ..$ offset: int 200
# .. ..$ length: int 12
# .. ..$ type : chr "character"
# .. ..$ format: chr "$"
# .. ..$ fhdr : int 0
# .. ..$ foff : int 76
# .. ..$ flen : int 1
# .. ..$ label : chr "UNIQUE SCHOOL ID (NCES ASSIGNED)"
# .. ..$ lhdr : int 0
# .. ..$ loff : int 44
# .. ..$ llen : int 32
# ...
So, if by any chance you need these attributes (I know some people are particularly keen on the labels, for instance), perhaps read.sas7bdat is the option for you after all.