Following are utilities to assist in monitoring satellite data ingest. These should be helpful for any data issues which come up at CEMSCS. First: 1) cd to /dcom/us007003 on prodsp 2) Type: ls -lt *out* 3) Note the time stamps indicating the last time these files were written to; for example: -rw-rw-r-- 1 nwprod prod 2400535 Jun 17 15:19 ingest_script_atovs1b.out -rw-rw-r-- 1 dbnprod dcdprod 2412498 Jun 17 15:19 tranjb.out -rw-rw-r-- 1 nwprod prod 2841021 Jun 17 15:19 bufr_transst.out -rw-rw-r-- 1 nwprod prod 2258556 Jun 17 15:10 bufr_transatw.out -rw-rw-r-- 1 nwprod prod 2167951 Jun 17 15:03 bufr_transsnd.out -rw-rw-r-- 1 nwprod prod 2347966 Jun 17 14:51 bufr_tran1x1s.out -rw-rw-r-- 1 nwprod prod 1822667 Jun 17 14:37 bufr_trantmi.out -rw-rw-r-- 1 nwprod prod 3017565 Jun 17 14:10 bufr_tranimgr.out -rw-rw-r-- 1 nwprod prod 1451645 Jun 16 19:00 ingest_sncvgrib.out -rw-rw-r-- 1 nwprod prod 2481728 Jun 16 09:26 bufr_transsmi.out -rw-rw-r-- 1 nwprod prod 2652933 Jun 16 09:23 bufr_tranquik.out -rw-rw-r-- 1 nwprod prod 2197886 Jun 16 09:22 bufr_traners2.out -rw-rw-r-- 1 nwprod prod 2249937 Jun 16 09:06 bufr_transbuv.out -rw-rw-r-- 1 nwprod prod 1547668 Jun 16 09:05 ingest_script_o3.out -rw-rw-r-- 1 nwprod prod 1800633 Jun 15 21:59 ingest_script_1b.out -rw-rw-r-- 1 nwprod prod 5193823 Mar 19 06:08 errlog.out where: ingest_script_atovs1b.out refers to the NOAA-15, -16, -17 ATOVS 1B radiance data (AMSU-A, AMSU-B, HIRS-3) tranjb.out refers to ALL data (decoder and satellite ingest) that is written into /dcom via tranjb processing bufr_transst.out refers to SST data bufr_transatw.out refers to GOES satellite wind data bufr_transsnd.out refers to GOES sounding/radiance data (5x5 field of view) bufr_tran1x1s.out refers to GOES cloud-top data (1x1 field-of-view) bufr_trantmi.out refers to TRMM/TMI rainfall data bufr_tranimgr.out refers to GOES imager radiance data ingest_sncvgrib.out refers to snow data bufr_transsmi.out refers to SSM/I data bufr_tranquik.out refers to QuikSCAT data bufr_traners2.out refers to ERS data bufr_transbuv.out refers to NOAA POES SBUV data in BUFR ingest_script_o3.out refers to NOAA POES SBUV data in IEEE format (soon to be turned off) ingest_script_1b.out refers to the NOAA 14 TOVS 1B radiance data (MSU,HIRS-2) errlog.out refers to errors coming out of TRANJB run Except for errlog.out (which we hope hasn't been written to for a long time), we want to see recent time stamps for these output files. Notice when this was run at 3:20 pm local that the ATOVS 1B has started running again, but the other CEMSCS types still show a time stamp of ~9:30 am local yesterday. Second: 1) cd to /com/logs on prodsp 2) type: grep "NESDIS INGEST: No data" jlogfile | grep "processed" 3) note the length of time since various data types have been processed; for example: 06/17 17:10:47Z LLssmi_1705.o29084-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 27 hrs. 06/17 17:12:33Z LLtovs_1652.o8250-NESDIS INGEST: No data from NSS.AMBX.NK processed in the last 28 hrs. 06/17 17:12:33Z LLtovs_1652.o8250-NESDIS INGEST: No data from NSS.AMBX.NL processed in the last 28 hrs. 06/17 17:12:34Z LLtovs_1652.o8250-NESDIS INGEST: No data from NSS.AMBX.NM processed in the last 31 hrs. 06/17 17:15:05Z LLtovs_1652.o8250-NESDIS INGEST: No data from NSS.HIR*.NJ processed in the last 39 hrs. 06/17 17:17:35Z LLtovs_1652.o8250-NESDIS INGEST: No data from NSS.MSU*.NJ processed in the last 39 hrs. 06/17 17:18:24Z LLssmi_1715.o43976-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 27 hrs. 06/17 17:18:24Z LLsst_1715.o30186-NESDIS INGEST: No data from PRD.SST.NCEP.NAVYOBS processed in the last 30 hrs. 06/17 17:20:55Z LLsst_1715.o30186-NESDIS INGEST: No data from PRD.SST.NCEP.NESDOBS processed in the last 30 hrs. 06/17 17:20:55Z LLssmi_1715.o43976-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 27 hrs. 06/17 17:22:36Z LLtovs_1652.o8250-NESDIS INGEST: No data from PRD.OZONE.PMF3. processed in the last 39 hrs. 06/17 17:22:36Z LLtovs_1652.o8250-NESDIS INGEST: No data from PRD.OZONE.PMF1. processed in the last 28 hrs. 06/17 17:23:35Z LLqscatter_1721.o33024-NESDIS INGEST: No data from NPR.QBUF.QS processed in the last 28 hrs. 06/17 17:23:35Z LLairs_1721.o29058-NESDIS INGEST: No data from NPR.AIBT.AQ processed in the last 28 hrs. 06/17 17:23:36Z LLeuscatter_1721.o59680-NESDIS INGEST: No data from NPR.URAX.E2 processed in the last 28 hrs. 06/17 17:24:27Z LLssmi_1721.o54634-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 27 hrs. 06/17 17:26:07Z LLeuscatter_1721.o59680-NESDIS INGEST: No data from NPR.UWAX.E2 processed in the last 28 hrs. 06/17 17:26:58Z LLssmi_1721.o54634-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 28 hrs. 06/17 17:30:06Z LLtovs_1652.o8250-NESDIS INGEST: No data from PRD.OZONE.PMF4. processed in the last 30 hrs. 06/17 17:30:06Z LLtovs_1652.o8250-NESDIS INGEST: No data from PRD.OZONE.PMF3. processed in the last 39 hrs. 06/17 17:30:06Z LLtovs_1652.o8250-NESDIS INGEST: No data from PRD.OZONE.PMF1. processed in the last 28 hrs. 06/17 17:30:31Z LLssmi_1727.o45488-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 28 hrs. 06/17 17:33:01Z LLssmi_1727.o45488-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 28 hrs. 06/17 17:35:07Z LLtovs_1652.o8250-NESDIS INGEST: No data from NPR.ATNC.NK processed in the last 28 hrs. 06/17 17:35:07Z LLtovs_1652.o8250-NESDIS INGEST: No data from NPR.ATNC.NL processed in the last 28 hrs. 06/17 17:43:39Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.HIRX.NK processed in the last 29 hrs. 06/17 17:43:39Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.HIRX.NL processed in the last 28 hrs. 06/17 17:43:39Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.HIRX.NM processed in the last 31 hrs. 06/17 17:48:40Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.AMAX.NK processed in the last 29 hrs. 06/17 17:48:40Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.AMAX.NL processed in the last 28 hrs. 06/17 17:52:56Z LLqscatter_1750.o81210-NESDIS INGEST: No data from NPR.QBUF.QS processed in the last 28 hrs. 06/17 17:52:57Z LLeuscatter_1750.o52954-NESDIS INGEST: No data from NPR.URAX.E2 processed in the last 28 hrs. 06/17 17:55:28Z LLeuscatter_1750.o52954-NESDIS INGEST: No data from NPR.UWAX.E2 processed in the last 28 hrs. 06/17 17:56:10Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.AMBX.NK processed in the last 29 hrs. 06/17 17:56:10Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.AMBX.NL processed in the last 28 hrs. 06/17 17:56:10Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.AMBX.NM processed in the last 31 hrs. 06/17 17:58:01Z LLssmi_1755.o54896-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 28 hrs. 06/17 17:58:41Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.HIR*.NJ processed in the last 39 hrs. 06/17 18:00:31Z LLssmi_1755.o54896-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 28 hrs. 06/17 18:01:11Z LLtovs_1736.o24644-NESDIS INGEST: No data from NSS.MSU*.NJ processed in the last 40 hrs. 06/17 18:06:12Z LLtovs_1736.o24644-NESDIS INGEST: No data from PRD.OZONE.PMF3. processed in the last 40 hrs. 06/17 18:06:12Z LLtovs_1736.o24644-NESDIS INGEST: No data from PRD.OZONE.PMF1. processed in the last 29 hrs. 06/17 18:13:43Z LLtovs_1736.o24644-NESDIS INGEST: No data from PRD.OZONE.PMF4. processed in the last 31 hrs. 06/17 18:13:43Z LLtovs_1736.o24644-NESDIS INGEST: No data from PRD.OZONE.PMF3. processed in the last 40 hrs. 06/17 18:13:43Z LLtovs_1736.o24644-NESDIS INGEST: No data from PRD.OZONE.PMF1. processed in the last 29 hrs. 06/17 18:18:22Z LLsst_1815.o41384-NESDIS INGEST: No data from PRD.SST.NCEP.NAVYOBS processed in the last 31 hrs. 06/17 18:18:48Z LLtovs_1736.o24644-NESDIS INGEST: No data from NPR.ATNC.NK processed in the last 28 hrs. 06/17 18:18:49Z LLtovs_1736.o24644-NESDIS INGEST: No data from NPR.ATNC.NL processed in the last 28 hrs. 06/17 18:20:52Z LLsst_1815.o41384-NESDIS INGEST: No data from PRD.SST.NCEP.NESDOBS processed in the last 31 hrs. 06/17 18:23:27Z LLairs_1820.o63130-NESDIS INGEST: No data from NPR.AIBT.AQ processed in the last 29 hrs. 06/17 18:23:27Z LLqscatter_1820.o33146-NESDIS INGEST: No data from NPR.QBUF.QS processed in the last 29 hrs. 06/17 18:23:35Z LLeuscatter_1821.o27978-NESDIS INGEST: No data from NPR.URAX.E2 processed in the last 29 hrs. 06/17 18:26:10Z LLeuscatter_1821.o27978-NESDIS INGEST: No data from NPR.UWAX.E2 processed in the last 29 hrs. 06/17 18:27:26Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.HIRX.NK processed in the last 29 hrs. 06/17 18:27:26Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.HIRX.NL processed in the last 29 hrs. 06/17 18:27:26Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.HIRX.NM processed in the last 32 hrs. 06/17 18:28:32Z LLssmi_1826.o43166-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 29 hrs. 06/17 18:31:02Z LLssmi_1826.o43166-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 29 hrs. 06/17 18:32:27Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.AMAX.NK processed in the last 30 hrs. 06/17 18:32:27Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.AMAX.NL processed in the last 29 hrs. 06/17 18:39:58Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.AMBX.NK processed in the last 30 hrs. 06/17 18:39:58Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.AMBX.NL processed in the last 29 hrs. 06/17 18:39:58Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.AMBX.NM processed in the last 32 hrs. 06/17 18:42:29Z LLtovs_1819.o32488-NESDIS INGEST: No data from NSS.HIR*.NJ processed in the last 40 hrs. 06/17 18:45:43Z LLssmi_1843.o36646-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 29 hrs. 06/17 18:48:13Z LLssmi_1843.o36646-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 29 hrs. 06/17 18:50:50Z LLqscatter_1848.o33168-NESDIS INGEST: No data from NPR.QBUF.QS processed in the last 29 hrs. 06/17 18:50:52Z LLeuscatter_1848.o49110-NESDIS INGEST: No data from NPR.URAX.E2 processed in the last 29 hrs. 06/17 18:52:49Z LLtovs_1845.o30250-NESDIS INGEST: No data from NSS.HIRX.NK processed in the last 30 hrs. 06/17 18:52:49Z LLtovs_1845.o30250-NESDIS INGEST: No data from NSS.HIRX.NL processed in the last 29 hrs. 06/17 18:52:49Z LLtovs_1845.o30250-NESDIS INGEST: No data from NSS.HIRX.NM processed in the last 32 hrs. 06/17 18:52:57Z LLssmi_1850.o52918-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 29 hrs. 06/17 18:53:22Z LLeuscatter_1848.o49110-NESDIS INGEST: No data from NPR.UWAX.E2 processed in the last 29 hrs. 06/17 18:55:27Z LLssmi_1850.o52918-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 29 hrs. 06/17 18:57:50Z LLtovs_1845.o30250-NESDIS INGEST: No data from NSS.AMAX.NK processed in the last 30 hrs. 06/17 18:57:50Z LLtovs_1845.o30250-NESDIS INGEST: No data from NSS.AMAX.NL processed in the last 29 hrs. 06/17 18:59:05Z LLssmi_1856.o27336-NESDIS INGEST: No data from NPR.EDRR.S processed in the last 29 hrs. 06/17 19:01:36Z LLssmi_1856.o27336-NESDIS INGEST: No data from NPR.SDRR.S processed in the last 29 hrs. where the following refer to POES data types retrieved from the CEMSCS server (non-operational in italics, otherwise, operational): LLssmi and NPR.SDRR.S refer to SSM/I brightness temperatures and Neural Net products generated from the brightness temps LLssmi and NPR.EDRR.S refer to SSM/I operational FNMOC products LLtovs and NSS.AMAX.NK refer to NOAA-15 AMSU-A brightness temperatures LLtovs and NSS.AMAX.NL refer to NOAA-16 AMSU-A brightness temperatures LLtovs and NSS.AMAX.NM refer to NOAA-17 AMSU-A brightness temperatures (not currently ingested) LLtovs and NSS.AMBX.NK refer to NOAA-15 AMSU-B brightness temperatures LLtovs and NSS.AMBX.NL refer to NOAA-16 AMSU-B brightness temperatures LLtovs and NSS.AMBX.NM refer to NOAA-17 AMSU-B brightness temperatures LLtovs and NSS.HIRX.NK refer to NOAA-15 HIRS-3 brightness temperatures LLtovs and NSS.HIRX.NL refer to NOAA-16 HIRS-3 brightness temperatures LLtovs and NSS.HIRX.NM refer to NOAA-17 HIRS-3 brightness temperatures LLtovs and NSS.HIR*.NJ refer to NOAA-14 HIRS-2 brightness temperatures (considered "data of opportunity" and are often very late) LLtovs and NSS.MSU*.NJ refer to NOAA-14 MSU brightness temperatures (considered "data of opportunity" and are often very late) LLtovs and NPR.ATNC.NK refer to NOAA-15 ATOVS temperature retrievals LLtovs and NPR.ATNC.NL refer to NOAA-16 ATOVS temperature retrievals LLtovs and NPR.ATNC.NM refer to NOAA-17 ATOVS temperature retrievals (not currently ingested) LLtovs and PRD.OZONE.PMF3. refer to NOAA-14 SBUV (ozone) data LLtovs and PRD.OZONE.PMF1. refer to NOAA-16 SBUV (ozone) data LLtovs and PRD.OZONE.PMF4. refer to NOAA-17 SBUV (ozone) data LLairs and NPR.AIBT.AQ refer to AIRS IR and AMSU brightness temperatures LLqscatter and NPR.QBUF.QS refer to QuikScat scatterometer products LLeuscatter and NPR.URAX.E2 refer to ERS radar altimeter data LLeuscatter and NPR.UWAX.E2 refer to ERS SAR data LLeuscatter and NPR.UWIX.E2 refer to ERS scatterometer data (not currently ingested) LLsst and PRD.SST.NCEP.NAVYOBS refer to NAVY SST data LLsst and PRD.SST.NCEP.NESDOBS refer to NESDIS SST data The following are not on the list above because they were not affected by an actual outage refer to GOES data types retrieved from gp12.ncep.noaa.gov server (non-operational in italics, otherwise, operational): LLradsnd and /pub/sndbfr/satsnd.goesW refer to GOES-10 5x5 soundings/radiances LLradsnd and /pub/sndbfr/satsnd.goesE refer to GOES-12 5x5 soundings/radiances LLradsnd and /pub/awips/dpisdev/goesW.RMD. refer to GOES-10 1x1 cloud-top data LLradsnd and /pub/awips/dpisdev/goesE.RMD. refer to GOES-12 1x1 cloud-top data LLsatwnd and /pub/bufrwinds/satwnd.bufrCD.goesW refer to GOES-10 IR channel "cloud drift" wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrCD.goesE refer to GOES-12 IR channel "cloud drift" wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV.goesW refer to GOES-10 water vapor imager channel wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV.goesE refer to GOES-12 water vapor imager channel wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrpt.goesW refer to GOES-10 picture triplet derived wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrpt.goesE refer to GOES-12 picture triplet derived wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrVZ02.goesW refer to GOES-10 visible channel wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrVZ02.goesE refer to GOES-12 visible channel wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV11.goesW refer to GOES-10 water vapor sounder channel 11 wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV10.goesW refer to GOES-10 water vapor sounder channel 10 wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV11.goesE refer to GOES-12 water vapor sounder channel 11 wind data LLsatwnd and /pub/bufrwinds/satwnd.bufrWV10.goesE refer to GOES-12 water vapor sounder channel 10 wind data The following were not on the list above because they were not affected by an actual outage refer to data types retrieved from servers other than CEMSCS and gp12 (non-operational in italics, otherwise, operational): LLtmi and /pub/data/mirrored_sites/trmmrt.gsfc.nasa.gov/2A12 refers to TRMM/TMI rainfall data (considered "data of opportunity") - from NASA/GSFC via ncosrv server LLradsnd and /pub/data/mirrored_sites/ftp.ssec.wisc.edu/gicsbt.*******.****.goes10NB refer to GOES-10 imager radiance data - from UW/SSEC via ncosrv server LLradsnd and /pub/data/mirrored_sites/ftp.ssec.wisc.edu/gicsbt.*******.****.goes12NB refer to GOES-12 imager radiance data- from UW/SSEC via ncosrv server