Sox, "the Swiss Army knife of audio manipulation", can generate accurate PNG spectrograms from sound files. It plays pretty much anything, and binaries are available for Windows. At the most basic level, you'd use something like this:
sox my.wav -n spectrogram
If you want a spectrogram with no axes, titles, legends, and a light background that's 100px high:
sox "Me, London.mp3" -n spectrogram -Y 130 -l -r -o "Me, London.png"
Sox accepts a lot of options if you only want to analyze a single channel for example. If you need your visuals to be even cooler, you could post-process the resulting PNG.
Here is a short overview from the commandline about all available parameters, the manpage has more details:
-x num X-axis size in pixels; default derived or 800
-X num X-axis pixels/second; default derived or 100
-y num Y-axis size in pixels (per channel); slow if not 1 + 2^n
-Y num Y-height total (i.e. not per channel); default 550
-z num Z-axis range in dB; default 120
-Z num Z-axis maximum in dBFS; default 0
-q num Z-axis quantisation (0 - 249); default 249
-w name Window: Hann (default), Hamming, Bartlett, Rectangular, Kaiser
-W num Window adjust parameter (-10 - 10); applies only to Kaiser
-s Slack overlap of windows
-a Suppress axis lines
-r Raw spectrogram; no axes or legends
-l Light background
-m Monochrome
-h High colour
-p num Permute colours (1 - 6); default 1
-A Alternative, inferior, fixed colour-set (for compatibility only)
-t text Title text
-c text Comment text
-o text Output file name; default `spectrogram.png'
-d time Audio duration to fit to X-axis; e.g. 1:00, 48
-S time Start the spectrogram at the given time through the input