USHCN Code Released

My USHCN code is available here as a zip file.

Once you unzip it you will see these files.

Main.cpp  Makefile  USHCN.cpp  USHCN.exe  USHCN.h

USHCN.exe is a Windows executable. Makefile is a Gnu makefile. If you aren’t a C++/STL programmer, you will probably have a difficult time understanding how it works.

In order to run it, you will need two files in the same directory as the executable.

http://cdiac.ornl.gov/ftp/ushcn_daily/ushcn-stations.txt

and any one of the the data files below. You will have to unzip the data file after you download it using gunzip.

Usage is : ushcn.exe USHCN_DATA_FILE_NAME [month=0-12] [max] [min]

Examples. Open up a command prompt window. Navigate to the directory with USHCN.exe, ushcn-stations.txt, and state41_TX.txt

ushcn.exe state41_TX.txt  > Texas.csv   – dumps out csv information for Texas to file Texas.csv

ushcn.exe state08_FL.txt  month=5 max > Florida_May_Max.csv   – dumps out csv information for Florida May maximum temperatures

ushcn.exe us.txt > US.csv – dumps out all US information

The code makes no attempt to do any adjustments or corrections. All it does is find all valid temperature recordings for all stations on all days, calculates averages – and finds min/max records per day of year for each station. There is no time bias. Earlier years are neither favored nor disadvantaged.

The reason I don’t do any corrections is because I am assuming a Monte Carlo distribution of error, which is the normal way that real scientists handle large data sets. There is no valid reason to believe that a huge data set created by thousands of observers over 130 years would have any systematic time-dependent bias to it – other than UHI. People read thermometers the same way in 1980 as they did in 1934.

If you mess the code up, take responsibility for it and don’t try to blame your mistakes on me.

[   ] state01_AL.txt.gz 23-Apr-2012 14:21 2.1M
[   ] state02_AZ.txt.gz 23-Apr-2012 14:21 2.9M
[   ] state03_AR.txt.gz 23-Apr-2012 14:21 2.3M
[   ] state04_CA.txt.gz 23-Apr-2012 14:21 6.8M
[   ] state05_CO.txt.gz 23-Apr-2012 14:21 3.8M
[   ] state06_CT.txt.gz 23-Apr-2012 14:21 521K
[   ] state07_DE.txt.gz 23-Apr-2012 14:21 598K
[   ] state08_FL.txt.gz 23-Apr-2012 14:21 2.8M
[   ] state09_GA.txt.gz 23-Apr-2012 14:21 3.3M
[   ] state10_ID.txt.gz 23-Apr-2012 14:21 3.9M
[   ] state11_IL.txt.gz 23-Apr-2012 14:21 6.3M
[   ] state12_IN.txt.gz 23-Apr-2012 14:21 5.4M
[   ] state13_IA.txt.gz 23-Apr-2012 14:21 4.2M
[   ] state14_KS.txt.gz 23-Apr-2012 14:21 5.0M
[   ] state15_KY.txt.gz 23-Apr-2012 14:21 2.0M
[   ] state16_LA.txt.gz 23-Apr-2012 14:21 2.6M
[   ] state17_ME.txt.gz 23-Apr-2012 14:21 1.7M
[   ] state18_MD.txt.gz 23-Apr-2012 14:21 1.9M
[   ] state19_MA.txt.gz 23-Apr-2012 14:21 1.5M
[   ] state20_MI.txt.gz 23-Apr-2012 14:21 4.0M
[   ] state21_MN.txt.gz 23-Apr-2012 14:21 5.2M
[   ] state22_MS.txt.gz 23-Apr-2012 14:21 4.4M
[   ] state23_MO.txt.gz 23-Apr-2012 14:21 3.9M
[   ] state24_MT.txt.gz 23-Apr-2012 14:21 6.1M
[   ] state25_NE.txt.gz 23-Apr-2012 14:21 7.5M
[   ] state26_NV.txt.gz 23-Apr-2012 14:21 1.8M
[   ] state27_NH.txt.gz 23-Apr-2012 14:21 812K
[   ] state28_NJ.txt.gz 23-Apr-2012 14:21 1.8M
[   ] state29_NM.txt.gz 23-Apr-2012 14:21 3.8M
[   ] state30_NY.txt.gz 23-Apr-2012 14:21 8.5M
[   ] state31_NC.txt.gz 23-Apr-2012 14:21 4.4M
[   ] state32_ND.txt.gz 23-Apr-2012 14:21 3.8M
[   ] state33_OH.txt.gz 23-Apr-2012 14:21 4.7M
[   ] state34_OK.txt.gz 23-Apr-2012 14:21 6.1M
[   ] state35_OR.txt.gz 23-Apr-2012 14:21 5.7M
[   ] state36_PA.txt.gz 23-Apr-2012 14:21 3.6M
[   ] state37_RI.txt.gz 23-Apr-2012 14:21 363K
[   ] state38_SC.txt.gz 23-Apr-2012 14:21 4.2M
[   ] state39_SD.txt.gz 23-Apr-2012 14:21 4.1M
[   ] state40_TN.txt.gz 23-Apr-2012 14:21 2.5M
[   ] state41_TX.txt.gz 23-Apr-2012 14:21 6.4M
[   ] state42_UT.txt.gz 23-Apr-2012 14:21 5.4M
[   ] state43_VT.txt.gz 23-Apr-2012 14:21 1.2M
[   ] state44_VA.txt.gz 23-Apr-2012 14:21 2.7M
[   ] state45_WA.txt.gz 23-Apr-2012 14:21 6.0M
[   ] state46_WV.txt.gz 23-Apr-2012 14:21 1.6M
[   ] state47_WI.txt.gz 23-Apr-2012 14:21 4.0M
[   ] state48_WY.txt.gz 23-Apr-2012 14:21 3.8M
[TXT] station_file_format.txt 07-Jul-2009 16:17 2.9K
[   ] us.txt.gz 23-Apr-2012 14:21 178M

About stevengoddard

Just having fun
This entry was posted in Uncategorized. Bookmark the permalink.

11 Responses to USHCN Code Released

  1. Anthony Watts says:

    Nicely done, thank you!

  2. John B., M.D. says:

    Thanks for contributing to the scientific process.
    I’ll just sit back and observe.

  3. Lorne50 says:

    nicely done indeed

  4. Brian G Valentine says:

    I used to think, that an indication of GHG warming would be, a decrease of DELTA of (daytime high – night time low) temperatures. Then I realized there is a huge water bias to this, impossible to make a uniform correction to make sense of it. (UHI correction would be nothing compared with something like that).

    Nevertheless I saw some study that did exactly that, and the conclusion was: Demonstration of GHG warming.

    The point is, people can and will do anything at all to demonstrate GHG warming. Meaningful or not, the whole Government supported thing has become, demonstrate GHG warming. It is such a joke, but it is not one bit funny.

  5. Imagine your critics actually downloading the data and doing their own analysis. None of them would be bright enough to understand what to do. I suspect downloading music onto their iPod’s is already pushing it.

  6. Nice!

    $ dos2unix Main.cpp USHCN.cpp USHCN.h && clang++ -o USHCN.run Main.cpp USHCN.cpp
    $ file USHCN.run
    USHCN.run: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses
    shared libs), for FreeBSD 9.0 (900044), not stripped

    😀

    • A stripped version built with -O3 -march=k8-sse3 is only 45K or so (-Os gets it down to 40K). The unoptimised version takes ~2.5x as long to run the full US dataset as the -O3 version. Yow!

      Unoptimised, no -march= setting:
      378.48 real 354.72 user 6.03 sys
      -O3 -march=k8-sse3
      137.33 real 125.97 user 5.01 sys
      ditto after strip -s
      137.70 real 126.41 user 5.05 sys
      -Os -march=k8-sse3 (& strip -s, cos it dinnae seem to make any runtime differences)
      146.48 real 132.38 user 5.19 sys

      This is all while compiling chromium in the background (it takes a couple of hours & uses 1.8G disk space while building, bleh).

      I don’t have LTO figured out just yet, or I’d fiddle with it, too.

  7. Bob Tisdale says:

    Steven: And I’ll second or third the Nicely Done.

    For those visitors who aren’t C++/STL literate, please consider providing the Contiguous U.S. temperatures as a single time series. I’d like to compare it to the sea surface temperatures of the U.S. Coastal Waters.

  8. RobJ says:

    Has the govt. quit updating the raw data sets? I can’t find data subsequent to 4/2012.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s