I'm attempting to use National Weather Service (U.S.) data, but something has changed recently and the GZip file no longer opens.
.NET 4.5 complains that...
Message=The magic number in GZip header is not correct. Make sure you are passing in a GZip stream.
Source=System
StackTrace:
at System.IO.Compression.GZipDecoder.ReadHeader(InputBuffer input)
at System.IO.Compression.Inflater.Decode()
at System.IO.Compression.Inflater.Inflate(Byte[] bytes, Int32 offset, Int32 length)
at System.IO.Compression.DeflateStream.Read(Byte[] array, Int32 offset, Int32 count)
I don't understand what has changed, but this is becoming a real show-stopper. Can anyone with GZip format experience tell me what has changed to make this stop working?
A file that works:
A file that doesn't work:
const string url = "http://www.srh.noaa.gov/ridge2/Precip/qpehourlyshape/2015/201505/20150505/nws_precip_2015050505.tar.gz";
string appPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
string downloadPath = Path.Combine(appPath, Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "nws_precip_2015050505.tar.gz");
using (var wc = new WebClient())
{
wc.DownloadFile(url, downloadPath);
}
string extractDirPath = Path.Combine(appPath, "Extracted");
if (!Directory.Exists(extractDirPath))
{
Directory.CreateDirectory(extractDirPath);
}
string extractFilePath = Path.Combine(extractDirPath, "nws_precip_2015050505.tar");
using (var fsIn = new FileStream(downloadPath, FileMode.Open, FileAccess.Read))
using (var fsOut = new FileStream(extractFilePath, FileMode.Create, FileAccess.Write))
using (var gz = new GZipStream(fsIn, CompressionMode.Decompress, true))
{
gz.CopyTo(fsOut);
}
It appears that this service SOMETIMES returns tar
format files disguised as .tar.gz
. This is very confusing, but if you check that the first two bytes are 0x1F
and 0x8B
, you can detect if the file is a GZip by checking its magic numbers manually.
using (FileStream fs = new FileStream(downloadPath, FileMode.Open, FileAccess.Read))
{
byte[] buffer = new byte[2];
fs.Read(buffer, 0, buffer.Length);
if (buffer[0] == 0x1F
&& buffer[1] == 0x8B)
{
// It's probably a GZip file
}
else
{
// It's probably not a GZip file
}
}