The WOUDC extended Comma Separated Values (extCSV) Format

The WOUDC extended Comma Separated Values (extCSV) Format File Header Description

by

E.W. Hare and E.J. Carty

What is the WOUDC Header?

The WOUDC header presents a set of variables that uniquely describes the data to follow in that particular file, and that are common to all data sets independent of the specific WOUDC data. The uniqueness of each field within the header is required such that if a duplicate data file was received, one or more of the header variables would have to have changed in order for this file to be accepted in to the data archive.

The WDC Metadata Header Format and Content Model

STATIC META DATA *(Only one occurrence or instance per file.)

Table Name	Condition	Field (Column) Names

CONTENT	Required	<Class,Category,Level,Form>
DATA_GENERATION	Required	<Date,Agency,Version,ScientificAuthority>
PLATFORM	Required	<Type ID,Name,Country,GAW_ID>
INSTRUMENT	Required	<Name,Model,Number >

DYNAMIC META DATA **(At least one required per file, multiple occurrences are permitted.)

LOCATION	Required	<Latitude,Longitude,Height >
TIMESTAMP	Required	<UTCOffset,Date,Time>

Metadata Header Format Rules

The order of the static metadata tables is fixed.
A table name is in UPPERCASE and denoted by the pound symbol (#).
A table is comprised of three sections: a table name, field (column) names with units (where applicable) and a data record(s). Refer to Example 1.
Columns are delimited by commas within each record which includes each field (column) name and datum.
Comments may appear anywhere within the file and are denoted by an asterisk (*).

*Example 1

#TABLENAME

Field1,Field 2,Field3

Datum1,Datum2,Datum3

Metadata Header Content Rules

Only ONE instrument may be represented in a file.
To report changes in time and space, more than one #LOCATION and #TIMESTAMP table is permitted. Since most data submitted are considered from a stationary source, a single #LOCATION table is all that is required.
Further information about site conditions, meteorology, sky images etc. may be included as comments or by other tables generated at the discretion of the data originator(s). For example, if a data originator wishes to indicate an occurrence of a special note ("instrument was bumped while sampling") than a comment is probably reasonable. Dynamic variations occurring instantaneously such as meteorology: eg. sky or wind conditions) or the reporting of calibration data; these examples would be represented in a table.

*Example 2

#SITE METEOROLOGY

Date,Time,Temperature,Pressure,Wind Direction,Wind Speed

Description of the Tables and Fields

TABLE	Column	Description

CONTENT	Class	The general type of data to follow. WOUDC would be used here.
	Category	Sub group of data submitted. For example: OzoneSonde or TotalOzone.
	Level	Data level. The level refers to the data product. Raw data would be Level 0, processed data Level 1 etc..
	Form	The version of the data format to follow. The specification for the Form (initial value=1) refers to the table descriptions for that index value. If a data table is already defined (form=1) with a specific number of columns, then the addition of one or more columns to that table would require the Form value to be indexed by 1 (form=1+i, where i=1) for every change.

DATA_ GENERATION	Date	Date the file was processed and/or generated. Date is represented in the ISO 8160 format standard, i.e. yyyy-mm-dd. Example 1998-09-21.
	Agency	The acronym of the submitting agency.
	Version	Data version specified by the submitting agency. These versions have the form *major.minor* (eg. 3.2) where major values are incremented with changes to the processing algorithm and minor values are incremented when the characterisation or calibration values have changed.. Note, minor values are reset to zero with changes to the processing algorithm.
	ScientificAuthority	The ScientificAuthority is the person(s) responsible for the data quality.

PLATFORM	Type	Type of observing platform Stationary (STN) or Moving (Examples: Airborne (FLT) Ship borne, (SHP) etc.). The default is. This field, for example, will accommodate the future inclusion of an onboard GPS on sonde flights which records exact location throughout a flight. Thus, FLT would be selected instead of STN.
	ID	Unique station or flight ID assigned by the WOUDC. The station number is a pointer to other information about the station such as the observation programs, the different instruments in operation and related site information like the proximity to an urban centre and other background effects.
	Name	Station Name
	Country	The country where the station is located. Country is the 3-letter ISO-3166 code. For example, Finland is FIN, Switzerland is CHE etc.
	GAW_ID	Applicable only to stations with the 5-digit WMO number.

INSTRUMENT	Name	Common name of instrument. For example, the Brewer spectrophotometer is called the "Brewer" whereas the Brewer-Mast ozonesonde is called the "Brewer-Mast".
	Model	Model ID where applicable. For the Brewer, a typical corresponding model is MKIII and a Dobson would be Beck or Japanese..
	Number	Serial number of the instrument.

LOCATION	Latitude	Benchmark latitude of the instrument. Typically an instrument is located at a fixed station location and shares the station latitude. The format is decimal degrees (deg.).
	Longitude	Benchmark longitude of the instrument. Typically an instrument is located at a fixed station location and shares the station longitude. The format is decimal degrees (deg.).
	Height	Benchmark height of the instrument. Typically an instrument is located at a fixed station location and shares the station elevation. The format is metres above sea level (m.a.s.l.)

TIMESTAMP	UTCOffset	The UTCOffset is the amount of time SUBTRACTED from the local time in order to obtain UTC time (where East is positive and Local Time as defined by the data originator). This is equivalent to UTC=time-UTCOffset. UTCOffset is represented in the ISO-8601 standard of hh:mm:ss. The default is to report time in UTC, thus UTCOffset=+00:00:00.
	Date	The Date corresponding to the time code represented in the ISO 8160 standard.
	Time	Time is represented in the ISO-8601 standard of hh:mm:ss.

An Example of an extCSV File Format Header

*SOFTWARE: SNDPRO 1.3

*TROPOPAUSE IN MB 320

*A.E.S.-O3 T1200

#CONTENT

Class,Category,Level,Form

WOUDC,OzoneSonde,1.0,1

#DATA_GENERATION

Date,Agency,Version,ScientificAuthority

1999-05-07,AES,1.0,Jonathan Davies

#PLATFORM

Type,ID,Name,Country,GAW_ID

STN,315,Eureka,CAN

#INSTRUMENT

Name,Model,Number

ECC,6a,6a2355

#LOCATION

Latitude,Longitude,Height

79.99,-85.94,10

#TIMESTAMP

UTCOffset,Date,Time

+00:00:00,1999-04-28,23:15:00