|
|
|
|
|
The need for a reliable data transport system was the driving force behind the development of DBNet. Many of the systems within NCEP were receiving and sending the data now processed by DBNet. Each system had been developed independently. Most of the systems were simple scripts that would ftp the data to or from one computer to another. While these systems were easy to implement, they were difficult, if not impossible, to maintain. They were not expandable and did not handle adverse conditions of network communications well. With this in mind the first goal of DBNet was to be reliable. Network and computer outages are no reason for data to be lost or become corrupt. The system should run in the most hostile situations. The second goal was maintainability. DBNet is written in the Perl and C programming languages. We followed the POSIX standard for C. The Perl language, while not a standard is used by a wide audience and supports all the common Unix platforms. The third goal was ease of implementation. This goal was the primary reason for choosing Perl. The second decision dictated by this goal was to use the Unix file system for data storage. This choice has proven to be a good decision, but not without drawbacks.
With these goals in mind we can discuss the internal operation of DBNet. Figure 1 shows the flow of data within DBNet. There are three main programs that run continuously. The first program, IFSP (InterFace Server-Paranoid) handles receiving data from other DBNet systems. The second program is dbn_external.pl. This program is responsible for filtering and queuing data from other DBNet systems. The third program is the queue handler, dbn_qh.pl. The queue handler spawns all the data processing for DBNet. The pending, foreign and dbnet queues are directories on the local filesystem. Each entry in the queue consists of a file in the queue directory. The file names used in each queue have very specific formats.
 |