File Processor Agent Example – YSI-6

<< Click to Display Table of Contents >>

 >Live > Agents > File Agents > File Processor > Examples >

File Processor Agent Example – YSI-6

1.Add YSI-6 to the *.config file's processors setting, and ensure the following Solinst settings:

 

Setting

Value

Description

YSI-6.dateTimeColumnHeaderRegex

Date|Date Time

DT_LOGGER_DATUM.DATUM_UTC_DT date; column header text contains Date, or Date Time.

YSI-6.dateTimeFormats

M/d/y|d-M-y H:m


YSI-6.folderWatcherPath

C:\EarthSoft\Live\YSI-6


YSI-6.headerLinesRegex

Date.*Time\n.*

The header line contains Date + anything (.*) + Time.

YSI-6.loggerCodeFileNameRegexGroups

((?'a'.*)_|(?'b'.*)\.|(?'c'.*))

DT_LOGGER.LOGGER_CODE = file name text before last x, where x =

1.underscore (_), else

2.dot (.), else if no underscore or dot,

3.everything.

YSI-6.loggerDescPrefix

YSI-6


YSI-6.seriesNameColumnHeaderRegexGroups

^(?'a'.*)\n

DT_LOGGER_SERIES.SERIES_NAME

YSI-6.seriesUnitColumnHeaderRegexGroups

\n(?'a'.*)$

DT_LOGGER_SERIES.SERIES_UNIT

YSI-6.timeColumnHeaderRegex

Time

DT_LOGGER_DATUM.DATUM_UTC_DT time; column header text contains Time.

 

And default (non-YSI-6) settings (or add YSI-6.dateTimeFormats, etc., settings):

 

Setting

Value

Description

delimiters

,

The fields are separated by commas.

utcOffsetHrs

-5

DT_LOGGER.UTC_OFFSET_HRS

 

2.Start the agent, and copy the following file, named SpringField_A_2019-02-01.csv, to folder C:\EarthSoft\Live\YSI-6.

 

 

"Date","Time","Temp","SpCond","Depth","pH","pHmV","Turbidity+","ODO%","ODO Conc"

"M/D/Y","hh:mm:ss","C","mS/cm","m","","mV","NTU","%","mg/L"

"10/11/11","12:21:06",24.32,0.399,0.362,7.59,-55.2,0.0,92.5,7.73

"10/11/11","12:21:43",24.36,0.400,1.112,7.59,-55.1,-0.1,92.3,7.71

"10/11/11","12:22:14",24.37,0.400,2.028,7.59,-54.8,0.0,92.1,7.70

 

 

3.The file's name contains the the DT_LOGGER.LOGGER_CODE. (The location/logger code is not in a YSI 6-series file's header or data lines; it can only be obtained from the file's name, or a default value.) Find/add the logger row, for example:

 

LOGGER_CODE

LOGGER_DESC

LIVE_DATA_SOURCE

UTC_OFFSET_HRS

SpringField_A

YSI-6 SpringField_A (= YSI-6.loggerDescPrefix + LOGGER_CODE)

folder://COMPUTER_NAME/
C:/EarthSoft/Live/YSI-6

-5 (= utcOffsetHrs)

 

4.The file's Date,Time line, and the line that follows it, are the header lines; they contain the DT_LOGGER_DATUM.DATUM_UTC_DT columns Date and Time. The other columns are the series; they contain DT_LOGGER_SERIES.SERIES_NAMEs and SERIES_UNITs.

 

5.For each subsequent line:

a.For each series (each column, excluding Date and Time columns):

i.Find/add the series row for the SERIES_NAME.

ii.Add the datum row, e.g., for the first three series in the first data line:

 

SERIES_NAME

DATUM_UTC_DT

DATUM_VALUE

SERIES_UNIT

Temp

2011-10-11 17:21:06 (= 2011-10-11 12:21:06 - UTC_OFFSET_HRS)

24.32

C

SpCond

2011-10-11 17:21:06

0.399

mS/cm

Depth

2011-10-11 17:21:06

0.362

m

 

Example File 2

 

The following file has one date and time column, instead of two.

 

 

Date Time,Temp,SpCond,DOsat,DO,Depth,pH,pH,Baro,Battery,TDS

D-M-Y HH:MM:SS,C,uS,%,mg/L,meters,,mV,mmHg,volts,g/L

26-07-13 14:18,19.36,492,108.6,9.99,0.082,7.71,-89.6,768.7,11.9,0.32

26-07-13 14:18,19.36,492,109.6,10.08,0.093,7.71,-89.5,768.6,11.9,0.32

26-07-13 14:18,19.37,492,110.5,10.16,0.145,7.71,-89.6,768.7,11.9,0.32

 

 

Note: The date and time formats in the file's header, line:
 
"M/D/Y","hh:mm:ss","C","mS/cm","m","","mV","NTU","%","mg/L"
 
are not used/supported. Instead, the dateTimeFormats are used.
 
Also, in Example File 2, the file's header includes seconds, but the data lines do not have seconds.