python netcdf: making a copy of all variables and attributes but one

Bruno von Paris picture Bruno von Paris · Feb 28, 2013 · Viewed 12.9k times · Source

I need to process a single variable in a netcdf file that actually contains many attributes and variable. I think it is not possible to update a netcdf file (see question How to delete a variable in a Scientific.IO.NetCDF.NetCDFFile?)

My approach is the following:

  1. get the variable to process from the original file
  2. process the variable
  3. copy all data from the original netcdf BUT the processed variable to the final file
  4. copy the processed variable to the final file

My problem is to code step 3. I started with the following:

def  processing(infile, variable, outfile):
        data = fileH.variables[variable][:]

        # do processing on data...

        # and now save the result
        fileH = NetCDFFile(infile, mode="r")
        outfile = NetCDFFile(outfile, mode='w')
        # build a list of variables without the processed variable
        listOfVariables = list( itertools.ifilter( lamdba x:x!=variable , fileH.variables.keys() ) )
        for ivar in listOfVariables:
             # here I need to write each variable and each attribute

How can I save all data and attribute in a handfull of code without having to rebuild a whole structure of data?

Answer

Rich Signell picture Rich Signell · Mar 31, 2018

Here's what I just used and worked. @arne's answer updated for Python 3 and also to include copying variable attributes:

import netCDF4 as nc
toexclude = ['ExcludeVar1', 'ExcludeVar2']

with netCDF4.Dataset("in.nc") as src, netCDF4.Dataset("out.nc", "w") as dst:
    # copy global attributes all at once via dictionary
    dst.setncatts(src.__dict__)
    # copy dimensions
    for name, dimension in src.dimensions.items():
        dst.createDimension(
            name, (len(dimension) if not dimension.isunlimited() else None))
    # copy all file data except for the excluded
    for name, variable in src.variables.items():
        if name not in toexclude:
            x = dst.createVariable(name, variable.datatype, variable.dimensions)
            dst[name][:] = src[name][:]
            # copy variable attributes all at once via dictionary
            dst[name].setncatts(src[name].__dict__)