Component-based post-processing system

(1) Getting started

(2) Namelists

(3) Verification

(4) Input data

(5) New schemes

Adding a dataset

COMPS support several formats for input data: Each dataset is located in its own folder in $COMPS/input/. This folder contains a data folder (e.g. $COMPS/input/gfs/data/) where files containing data are stored. In addition, one or more special metadata files with the extension .nl are also located here.

Format	Description	Filename
Flat file	Each row is an offset and each column an ensemble member (just one if observation dataset)	[date]_[location]_[variable].txt
GRIB	Yes	[date].grb2
NetCDF	No	[date].nc

Flat files

Flat files are the easiest format to convert your dataset to, but can be inefficient if large. We show here an example dataset, which contains three ensemble members, two locations (Vancouver and Abbotsford airports), two variables (temperature and relative humidity), for the dates Jan 1, 2012 to Dec 31, 2012. We have forecasts available for forecast horizons 0h, 12h, and 24h.

Data files

The flat files are The contents of $COMPS/data/gfs/data would be:

20120101_597_Hum.txt
20120101_597_Temp.txt
20120101_604_Hum.txt
20120101_604_Temp.txt
20120102_597_Hum.txt
20120102_597_Temp.txt
20120102_604_Hum.txt
20120102_604_Temp.txt

File 20120101_597_Temp.txt looks like:

0.31 0.22 0.12
0.92 2.41 -999
3.14 2.52 5.71

Note that the missing value indicator in COMPS is -999.

Metadata files

In addition to the datafiles, 4 metadata files must be created:

The members.nl file contains information about the ensemble members and might look like this:

0 model=WRF3 resolution=4
1 model=MM5  resolution=4
2 model=WRF3 resolution=12

Both model and resolution are optional, but must be supplied if you want the Selector to select ensemble members based on model name or resolution.

locations.nl would look like:

597 code=YVR lat=49 lon=-123 elev=0
604 code=YXX lat=49 lon=-122 elev=10

Both lat and lon are required, because the downscaler needs to know where the locations are.

offsets.nl stores the offset corresponding to each line in the datafiles in hours:

0
12
24

variables.nl stores a map that relates the variable names in the dataset to standard variable names in COMPS. The first column in the file is the COMPS variable name and the variable in the name attribute is the variable name internal to this dataset. The variable definitions in COMPS is located in $COMPS/namelists/default/variables.nl.

T name=Temp
RH name=Hum

Tutorials