How to take data out for a test drive, part 1

I wear many hats including data curator and data educator.   I'd like to share a couple of videos I made to help people access weather data from the National Center for Atmospheric Research's Research Data Archive (NCAR RDA).

As I mentioned in Flying Solo, I answer help desk questions about the most popular dataset at the archive, NCEP FNL (Final) Operational Global Analysis. NOAA NCEP creates a representation of the full atmospheric state every six hours using all the calibrated satellite, weather balloon, aircraft, ship, and surface data available at analysis time.  Anyone in the world (except in a few embargoed countries) can ftp to their real-time servers and obtain the data for free.  Most commercial weather services rely on this data and repackage it up as their own branded content.

If you have read my statistics and bullshit threads, you know that I am passionate about empowering people to perform their own data analysis.  I hope that spreading data knowledge helps the public become  savvier consumers of branded content and recognize analysis that doesn't pass the sniff test.

I'm creating a data course to teach people how to find, access and utilize free weather and climate data.  I'm posting a couple of proof of concept videos and I'd love to hear your feedback.

If you want to follow along and try this at home (and I hope you do!), then you need to sign up for a free account first. You can choose the appropriate type or organization or even select no affiliation.  (In that case, write "self" under "Organization Name".)

I demonstrated using a Mac with a browser window and a terminal running tcsh. The first method works on any OS, with any browser, not just Chrome.  Watch them full-screen so you can read the type.



After you take a file or two out for a test spin, and are ready to sift through a lot of of data, I recommend you use an automated batch script. You don't need to be able to write your own csh or perl script. Our software generates one for you based on your custom data request. You do need to know enough UNIX/Linux to work at the command line.  If you are using Windows, try the perl script instead.



Enjoy! And please give me your suggestions on how I can improve them.

I do need to explain that global grids of atmospheric data can become very large; they are packed in a gridded binary format (GRIB1 or GRIB2) agreed upon by all the nations belonging to the World Meteorological Organization.  You need specialized software to unpack and use them.  For these exercises, select the newer GRIB2

Next up, "GRIB1 or GRIB2? What's the difference and how do I choose?" and "What the heck do I do with this binary data?"

In case you can't wait for the next episode, download NASA's Panopoly viewer and you will be slicing and dicing data in your kitchen in no time.