Live File Processor Agent File – EBatch

<< Click to Display Table of Contents >>

Navigation:  Live > Agents > File Agents > File Processor > Settings > File >

Live File Processor Agent File – EBatch

Live File Processor Agent's data are assigned an ebatch number when loaded into the database. Thus the data's source file is known, and if the data are bad, it can be removed from the database using the EDD Rollback option in EDP.

 

Note: This functionality requires the Live Schema (schema.LIVE.xme) for EQuIS 7.20.2 Build or later, which adds the ebatch columns to the DT_LOGGER tables.

 

 

Setting

Type

Description

ebatchEddFile

Composite String

The ST_EDD_BATCH.EDD_FILE value.

If an ST_EDD_BATCH table row exists where EDD_FILE = ebatchEddFile (and ROLLBACK_DATE is null), then this row is used; else, a new ST_EDD_BATCH table row is added.

ebatchEddType

Composite String

The ST_EDD_BATCH.EDD_TYPE value.

 

 

Example 1 – Path and File Name

 

The ebatchEddFile's default value is:

 

Setting

Value

ebatchEddFile

{folderWatcherPath}/{fileNameWithExtension}

HOBO.folderWatcherPath

C:\EarthSoft\Live\HOBO

 

If a file is named rainfall-2020-05-26.csv, ST_EDD_BATCH.EDD_FILE is C:/EarthSoft/Live/HOBO/rainfall-2020-05-26.csv.

 

The same ebatch number is used for a file (path and name), regardless of how often the file gets data appended to it.

 

 

Example 2 – Path and Part of File Name

 

A date and time, or other info (e.g., everything after the first dash), can be removed from the file name. An example is provided in the table below.

 

Setting

Value

HOBO.ebatchEddFile

{folderWatcherPath}/{fileNameWithoutExtension => ebatchEddFileRegexReplacements}.{fileExtension}

HOBO.ebatchEddFileRegexReplacements

^(?'a'[^-.]*).* => ${a}-*

HOBO.folderWatcherPath

C:\EarthSoft\Live\HOBO

 

If a file is named rainfall-2020-05-26.csv, ST_EDD_BATCH.EDD_FILE is C:/EarthSoft/Live/HOBO/rainfall-*.csv.

 

The same ebatch number is used for all files named like rainfall-*.csv.

 

 

Example 3 – Path and File Name and Date

 

A date time variable (e.g., the file's last write date) can be added to the file name. An example is provided in the table below.

 

Setting

Value

HOBO.ebatchEddFile

{folderWatcherPath}/{fileNameWithoutExtension}-{fileLastWriteTimeUtc:yyyy-MM-dd}.{fileExtension}

HOBO.folderWatcherPath

C:\EarthSoft\Live\HOBO

 

If a file is named rainfall.csv, ST_EDD_BATCH.EDD_FILE is C:/EarthSoft/Live/HOBO/rainfall-2020-05-26.csv.

 

A new ebatch number is used for file rainfall.csv for each day.

 

 

Example 4 – Not Used

 

If ebatchEddFile is empty, ebatch is not used.

 

Setting

Value

ebatchEddFile


 

 

EDD Rollback

 

The EDD Rollback option in EDP can be used to remove the data, but it fails if all the data for a selected ebatch number cannot be removed. A temporary workaround may be to use an SQL statement (which can be executed using SQL Server Management Studio) to delete the DT_LOGGER_DATUM (e.g., delete from DT_LOGGER_DATUM where EBATCH = 1234), and then delete the DT_LOGGER, DT_LOGGER_SERIES, etc., rows manually.