File Processor Agent Example – RST-DT2055

<< Click to Display Table of Contents >>

Live > Agents > File Agents > File Processor > Examples >

File Processor Agent Example – RST-DT2055

1.Add RST-DT2055 to the *.config file's processors setting, and ensure the following RST-DT2055 settings:

 

Setting

Value

Description

RST-DT2055.dateTimeColumnHeaderRegex

TIMESTAMP

DT_LOGGER_DATUM.DATUM_UTC_DT; column header text contains TIMESTAMP.

RST-DT2055.folderWatcherPath

C:\EarthSoft\Live\
RST-DT2055


RST-DT2055.headerLinesRegex

TIMESTAMP\nUNITS

The header has 2 lines that look like this:

1.contains TIMESTAMP

2.contains UNITS

 

Note: Lines starting with ,,, are ignored, so the TIMESTAMP and UNITS lines are 'read' one after the other.

RST-DT2055.headerLinesReplacementRegex

(?'a'[^,]+),,,,,(?'b'[^,]+)

Replace ,,,,, with ,,,, in the file's header.

 

Note: The example file has an extra comma in the TIMESTAMP line, which is not present in the UNITS and data lines.

RST-DT2055.headerLinesReplacement

${a},,,,${b}

RST-DT2055.ignoreColumnHeadersRegex

RECORD

Do not add series or datum for columns whose header text contains RECORD.

RST-DT2055.ignoreLinesRegex

^,,,

Ignore lines starting with ,,,.

RST-DT2055.logDuplicateDatumRowErrors

false

See Example File 2, below.

RST-DT2055.loggerCodeHeaderLinesRegexGroups

^Name,(?'a'[^,]*)

DT_LOGGER.LOGGER_CODE

RST-DT2055.loggerDescPrefix

RST-DT2055


RST-DT2055.seriesNameColumnHeaderRegexGroups

^(?'a'[^\n]*)

DT_LOGGER_SERIES.SERIES_NAME

RST-DT2055.seriesUnitColumnHeaderRegexGroups

\n(?'a'.*)$

DT_LOGGER_SERIES.SERIES_UNIT

 

And default (non-RST-DT2055) settings (or add RST-DT2055.dateTimeFormats, etc., settings):

 

Setting

Value

Description

dateTimeFormats


Empty; determined automatically.

delimiters

,

The fields are separated by commas.

utcOffsetHrs

2

DT_LOGGER.UTC_OFFSET_HRS; Solwezi, Zambia

 

2.Start the agent, and copy the following file to folder C:\EarthSoft\Live\RST-DT2055.

 

 

Model,DT2055 VW Multi Channel Logger

Name,CGT086

Serial Number,02593

Firmware Version,v1.20,Software Version,v1.31.0

Sampling Status,LOGGING,APPEND

Sampling Rate,FIXED

Current Interval,12 Hours  0 Minutes  0 Seconds

Start Time,Mon  Oct  22  18  12:00:00  PM

Current Time,Tue  Nov  27  18  10:12:42  AM

Number Of Sensors,8

Number Of Records,72

Sensor Upload Num / Error Code,7 / 80000

 

"TIMESTAMP","RECORD","BATTERY","VW 1","Therm 1","VW 2","Therm 2","VW 3","Therm 3","VW 4","Therm 4",,,,,"VW 1","Therm 1","VW 2","Therm 2","VW 3","Therm 3","VW 4","Therm 4","LOGGER TEMP","RESET"

,,,,,,,,,,,,,Unit Type,"Pressure","Disabled","Pressure","Disabled","Pressure","Disabled","Pressure","Disabled"

,,,,,,,,,,,,,Input Units,"kPa","Disabled","kPa","Disabled","kPa","Disabled","kPa","Disabled"

,,,,,,,,,,,,,Output Units,"m H2O","Disabled","m H2O","Disabled","m H2O","Disabled","m H2O","Disabled"

,,,,,,,,,,,,,Offset,"8.84600e+001","Disabled","5.69600e+001","Disabled","3.94600e+001","Disabled","2.24600e+001","Disabled"

,,,,,,,,,,,,,Cal A =,"5.64600e-004","Disabled","2.73700e-004","Disabled","2.89500e-004","Disabled","2.98200e-004","Disabled"

,,,,,,,,,,,,,Zero B =,"7.28600e+003","Disabled","7.01000e+003","Disabled","7.70800e+003","Disabled","8.34800e+003","Disabled"

,,,,,,,,,,,,,Cal C =," ","Disabled"," ","Disabled"," ","Disabled"," ","Disabled"

,,,,,,,,,,,,,Temp CF,"4.35600e-001","Disabled","9.30800e-002","Disabled","7.44500e-002","Disabled","1.09500e-001","Disabled"

,,,,,,,,,,,,,Temp Initial,"2.50000e+001","Disabled","2.30000e+001","Disabled","2.50000e+001","Disabled","2.50000e+001","Disabled"

,,,,,,,,,,,,,Linked Channel,"1","Disabled","3","Disabled","5","Disabled","7","Disabled"

"UNITS",,"Volts","m H2O","deg C","m H2O","deg C","m H2O","deg C","m H2O","deg C",,,,"B Units","Ohms","B Units","Ohms","B Units","Ohms","B Units","Ohms"

2018/10/22 12:00:00,1,2.9,88.32,22.86,56.92,22.39,39.42,22.14,22.41,22.11,,,,8110.05,3297.91,8102.74,3368.05,8493.80,3406.23,8994.10,3410.74,29.50,0

2018/10/23 00:00:00,2,2.9,88.32,22.86,56.92,22.39,39.42,22.14,22.41,22.11,,,,8109.56,3297.91,8101.93,3368.05,8493.53,3406.23,8993.73,3410.74,16.60,0

2018/10/23 12:00:00,3,2.9,88.32,22.86,56.92,22.39,39.42,22.14,22.41,22.11,,,,8109.31,3297.91,8102.43,3368.05,8493.80,3406.23,8993.81,3410.74,32.30,0

 

 

 

3.The file's Name line contains the DT_LOGGER.LOGGER_CODE. Find/add the logger row, e.g.:

 

LOGGER_CODE

LOGGER_DESC

LIVE_DATA_SOURCE

UTC_OFFSET_HRS

CGT086

RST-DT2055 CGT086 (= RST-DT2055.loggerDescPrefix + LOGGER_CODE)

folder://COMPUTER_NAME/
C:/EarthSoft/Live/RST-DT2055

2

 

4.The line that contains TIMESTAMP, followed by the line that contains UNITS (lines starting with ,,, are ignored), are the header lines; they contain the DT_LOGGER_DATUM.DATUM_UTC_DT column. The RECORD column is ignored. The other columns are the series; they contain DT_LOGGER_SERIES.SERIES_NAMEs and DT_LOGGER_SERIES.SERIES_UNITs.

 

5.For each subsequent line:

a.For each series (each column, excluding TIMESTAMP and RECORD columns):

i.Find/add the series row for the SERIES_NAME and SERIES_UNIT.

ii.Add the datum row, e.g., for the first three series in the first data line:

 

SERIES_NAME

DATUM_UTC_DT

DATUM_VALUE

SERIES_UNIT

BATTERY

2018-10-22 10:00:00 (= 2018-10-22 12:00:00 - UTC_OFFSET_HRS)

2.9

Volts

VW 1

2018-10-22 10:00:00

88.32

m H2O

Therm 1

2018-10-22 10:00:00

22.86

deg C

 

Note: The offsets, between the TIMESTAMP and UNITS, lines:
 
,,,,,,,,,,,,,Offset,"8.84600e+001","Disabled","5.69600e+001","Disabled","3.94600e+001","Disabled","2.24600e+001","Disabled"
 
are not added to the datum values.
 
The Data Computation Agent or Data Deriver Agent may be used to add the offsets.

 

Example File 2

 

The following file contains two VW 1, 2, and 3 columns, with the same units (B Units), and values (and dates and times). RST-DT2055.logDuplicateDatumRowErrors = false, does not log errors for these.

 

 

Model,DT2055 VW Multi Channel Logger

Name,CPD-002

Serial Number,02574

Firmware Version,1.20,Software Version,1.15.4

Sampling Status,LOGGING

Sampling Rate,FIXED

Current Interval,12:0:0

Start Time,2018/09/24,08:50:33

Current Time,2018/11/27,11:43:40,Logger Time,2018/11/27,11:41:55

Number Of Sensors,6

Number Of Records,129,USB

Sensor Upload Num / Status Code,7,80000,FFFFFFFF,1048704,Bat Type Not set,,WD 4,PO 0,EXT 0,BO 0,SW 255,PDI 255

 

"TIMESTAMP","RECORD","BATTERY","VW 1","Therm 1","VW 2","Therm 2","VW 3","Therm 3",,,,"VW 1","Therm 1","VW 2","Therm 2","VW 3","Therm 3","LOGGER TEMP","RESET","USB","RSSI","STATUS 0x"

,,,,,,,,,,,Unit Type,"Disabled"," ","Disabled"," ","Disabled"," ",,,,,"[code][P#]"

,,,,,,,,,,,Input Units,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Output Units,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Offset,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Cal A =,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Zero B =,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Cal C =,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Temp CF,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Temp Initial,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Linked Channel,"Disabled"," ","Disabled"," ","Disabled"," "

,,,,,,,,,,,Sensor,"U User","3K NTC","U User","3K NTC","U User","3K NTC"

"UNITS->",,"Volts","B Units","deg C","B Units","deg C","B Units","deg C",,,,"B Units","Ohms","B Units","Ohms","B Units","Ohms","deg C"

2018/09/24 08:50:41,1,3.65,8420.08,23.28,8668.99,22.71,9063.68,22.70,,,,8420.080078,3236.779053,8668.990234,3320.150879,9063.679688,3321.922363,23.30,1,0,0,000

2018/09/24 20:50:33,2,3.00,8421.59,23.28,8669.47,22.70,9064.20,22.69,,,,8421.589844,3237.213867,8669.469727,3321.922363,9064.200195,3323.694336,23.00,0,0,0,000

2018/09/25 08:50:33,3,3.00,8422.64,23.27,8669.40,22.69,9064.05,22.68,,,,8422.639648,3238.083740,8669.400391,3323.251221,9064.049805,3325.023926,23.40,0,0,0,000