COPY with dynamic file name

Ben Gosack picture Ben Gosack · Apr 15, 2013 · Viewed 7.1k times · Source

I am trying to write a function to load csv data into a table. I want the input argument to be the path to the file.

CREATE OR REPLACE FUNCTION public.loaddata(filepathname varchar)
  RETURNS void AS
$BODY$
BEGIN
COPY climatedata(
    climatestationid, 
    date,
    prcp,
    prcpqflag,
    prcpmflag,
    prcpsflag,
    tmax,
    tmaxqflag,
    tmaxmflag,
    tmaxsflag,
    tmin,
    tminqflag,
    tminmflag,
    tminsflag)
  FROM $1
  WITH csv header;
END;
$BODY$
  LANGUAGE plpgsql VOLATILE
  COST 100;
ALTER FUNCTION public.filltmaxa(character varying)
  OWNER TO postgres;

When I try to create this function I get

syntax error at $1

What's wrong with it?

Answer

Erwin Brandstetter picture Erwin Brandstetter · Apr 15, 2013

You need dynamic SQL:

CREATE OR REPLACE FUNCTION loaddata(filepathname text)
  RETURNS void AS
$func$
BEGIN
   EXECUTE format ('
   COPY climatedata(
         climatestationid
       , date
         ... -- more columns 
       , tminsflag)
   FROM %L (FORMAT CSV, HEADER)'  -- current syntax
           -- WITH CSV HEADER'    -- tolerated legacy syntax
   , $1);  -- pass function parameter filepathname to format() 
END
$func$ LANGUAGE plpgsql;

format() requires PostgreSQL 9.1+.
Pass the file name without extra set of (escaped) single quotes:

SELECT loaddata('/absolute/path/to/my/file.csv')

format() with %L escapes the file name safely. Would be susceptible to SQL injection without it.


Aside, you have a function name mismatch:

CREATE OR REPLACE FUNCTION public.loaddata(filepathname varchar)
...
ALTER FUNCTION public.filltmaxa(character varying)