Real World Examples

I have included several real world examples to demonstrate the features and concepts discussed on the documentation. I have added comments to the tables to emphasize the important ideas in each step. Through out the examples several things should be noticed, 1: the importance of consistent parsing table entries across the entire DBNet infrastructure. 2: the isolation of the cray super computer from the communications support needed to feed many different sites. 3: the extensive use of regular expressions for power and flexibility. There are also some policies that effect the data flow decisions, like the level of timeliness required by a recipient. NCDC does not get data from das1 or the cray because they do not need the data in real-time. Another goal here is to minimize the number of machines interacting with the cray, leave the cray cpu cycles for running the model not servicing ftp connections.

1. Data flow from the crays to NCDC. Several data files generated on the cray are sent to NCDC for processing. The remote site is can not access the cray directly. Therefore we have used an intermediate step for this data stream. The general flow is: cray -> das1 -> ftp -> dew (ncdc). The appropriate parsing table entries are shown below.

Cray:

HOST host das1 0120 # HOST id for das1

HOST host cray 1001 # host id for local machine

BUFRDUMP adpupa .* BFD0 # message code for this file

QUECMD ALERT ^1001BFD0 CLNT 120 das1 # command to send the alert to das1

das1:

HOST host das1 0120 # HOST id for das1

# host id for cray with login ID and password

HOST host cray 1001 anonymous dbnet@das1 # host id for local machine

# message code for this file including local directory and file info, required by FTPGET command

BUFRDUMP adpupa .*/(.*) BFD0 /pub/data/NCDC $1 0664 06 21 1

# command to get data from cray

QUECMD ALERT ^1001BFD0 FTPGET 60

# command to send new alert ( for data on das1 ) to NCDC

QUECMD ALERT ^0120BFD0 CLNT 240 ftp.ncep.noaa.gov

ftp:

# HOST ID for ftp

HOST host ftp 0110

# host id for das1 with login ID and password

HOST host das1 0120 anonymous DBNet@ftp

# message code for this file including local directory and file info, required by FTPGET command

BUFRDUMP adpupa .*/(.*) BFD0 /pub/data/NCDC $1 0664 06 21 1

# command to get data from cray

QUECMD ALERT ^0120BFD0 FTPGET 60

# command to send new alert ( for data on das1 ) to NCDC

QUECMD ALERT ^0110BFD0 CLNT 240 dew.ncdc.noaa.gov

NCDC:

# host id for das1 with login ID and password

HOST host das1 0120 anonymous dbnet@dew # HOST id for das1

# message code for this file including local directory and file info, required by FTPGET command

BUFRDUMP adpupa .*/(.*) BFD0 /pub/das1_data/bufr $1 0664 06 21 1

# command to get data from cray

QUECMD ALERT ^0120BFD0 FTPGET 60

RUC2 dataflow from the cray to hp47, generate metafiles and send them to das1. The ruc2 model output is used in NAWIPS in the form of metafiles. HP47 is used to generate the metafiles from the grib files. The process is divided into 2 steps, 1: get the grib files to hp47, 2: generate the metafile and ship it to das1.

Cray:

# local machine (cray )

HOST host cray 1001

#message code for remote machine

HOST host das1 0120

# message code for ruc2 grib files

MODEL RUC2_PGB .*/(ruc2a\.\d\d\d\d\d\d)/(.*) MRUC2PGB

# command to send ruc2 to das1

QUECMD ALERT 1001MRCU2PGB CLNT 120 das1

das1:

# host code with user id and password for cray

HOST host cray 1001 anonymous dbnet@das1

# host code for local machine

HOST host das1 0120

# message code for ruc2 grib files with directory and file info for local storage

MODEL RUC2_PGB .*/(ruc2a\.\d\d\d\d\d\d)/(.*) MRUC2PGB /pub/data1/ruc2/$1 $2 0664 6 21 1

# command to get ruc2 data from cray

QUECMD ALERT 1001MRUC2PGB FTPGET 120

# command to send new alert to HP47

QUECMD ALERT 0120MRUC2PGB CLNT 60 hp47