Which is the best method to apply a script repetitively to n .csv files in R?

n.e.w picture n.e.w · Apr 27, 2011 · Viewed 7.4k times · Source

My situation:

  1. I have a number of csv files all with the same suffix pre .csv, but the first two characters of the file name are different (ie AA01.csv, AB01.csv, AC01.csv etc)
  2. I have an R script which I would like to run on each file. This file essentially extracts the data from the .csv and assigns them to vectors / converts them into timeseries objects. (For example, AA01 xts timeseries object, AB01 xts object)

What I would like to achieve:

  1. Embed the script within a larger loop (or as appropriate) to sequentially run over each file and apply the script
  2. Remove the intermediate objects created (see code snippet below)
  3. Leave me with the final xts objects created from each raw data file (ie AA01 to AC01 etc as Values / Vectors etc)

What would be the right way to embed this script in R? Sorry, but I am a programming noob!

My script code below...heading of each column in each CSV is DATE, TIME, VALUE

    # Pull in Data from the FileSystem and attach it
AA01raw<-read.csv("AA01.csv")
attach(AA01raw)
#format the data for timeseries work
cdt<-as.character(Date)
ctm<-as.character(Time)
tfrm<-timeDate(paste(cdt,ctm),format ="%Y/%m/%d %H:%M:%S")
val<-as.matrix(Value)
aa01tsobj<-timeSeries(val,tfrm)
#convert the timeSeries object to an xts Object
aa01xtsobj<-as.xts(tsobj)
#remove all the intermediate objects to leave the final xts object
rm(cdt)
rm(ctm)
rm(aa01tsobj)
rm(tfrm)
gc()

and then repeat on each .csv file til all xts objects are extracted.

ie, what we would end up within R, ready for further applications are:

aa01xtsobj, ab01xtsobj, ac01xtsobj....etc

any help on how to do this would be very much appreciated.

Answer

Thomson Comer picture Thomson Comer · Apr 27, 2011

Be sure to use Rs dir command to produce the list of filenames instead of manually entering them in.

filenames = dir(pattern="*01.csv")
for( i in 1:length(filenames) )
{
  ...