EFD Management Configuration – File Monitor

<< Click to Display Table of Contents >>

Navigation:  Configuration > ST_CONFIG Entries > Enterprise Workflow Service > EFD Management >

EFD Management Configuration – File Monitor

The EarthSoft workflow service File Monitor agent loads EDD files to an EQuIS database in preparation for processing by Enterprise EDP. The File Monitor agent scans folders for new EDDs. The folders are determined by combining the EQuIS work folder with the channels (e.g., web, email, FTP) defined by the efdManagementConfiguration.FileMonitor configuration settings.

 

FileMonitor agent configuration includes definition of EDD delivery channels and the client assembly associated with them. The defined client assembly determines what the FileMonitor agent does with any EDD files found in the specified channel folder. Three options are available for the default client assembly (EarthSoft.Workflow.Clients.Default):

EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

EarthSoft.Workflow.Clients.Default.DataFileMonitor

EarthSoft.Workflow.Clients.Default.SplitByFacilityFileMonitor

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.DefaultFileMonitor, files with a file name extension that matches an inboundFileType are loaded into the ST_FILE_REGISTRATION table and removed from the channel folder. Files with a file name extension that do not match an inboundFileType are moved to the .\archive\ folder.

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.DataFileMonitor, a new EDD is created for each file in the channel folder. The EDD contains the data file and user certificated as specified in the configuration. The original file is removed from the channel folder and the newly created EDD is loaded into the ST_FILE_REGISTRATION table. With this channel configuration all files are processed and no files are moved to the .\archive\ folder. See the DataFileMonitor Configuration section below for more details.

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.SplitByFacilityFileMonitor, the EDD file name must specify a format that is registered. Files with names that do not contain a registered format or have file name extensions that do not match configured inboundFileTypes are moved to the .\archive\ folder. If the EDD file name specifies a FACILITY_CODE or the format does not support SplitByFacility, then the file is loaded into the ST_FILE_REGISTRATION table. For EDD files named with a registered format that supports SplitByFacility and do not specify a facility, SplitByFacilityFileMonitor will split the file into a separate EDD for each facility denoted in the designated FACILITY_CODE column.

 

The agent can be completely customized through development of a custom client assembly (DLL). The default client assembly (EarthSoft.Workflow.Clients.Default) supports all the capabilities of other client assemblies. The major difference between the Default and other Client Assemblies is the EDD file naming convention.

 

 

EQuIS Version: 7

 

Namespace: EarthSoft.Workflow.Clients

 

Products Affected:

oEnterprise

oProfessional

oEDGE

oEDP Stand Alone

 

Required Modules: Enterprise Workflow Service

 

Channel Options

 

By default, EarthSoft defines three channels for the File Monitor agent:

WEB – EDDs uploaded via EQuIS Enterprise EDP EDD Upload widget or Sign and Submit web service.

EML – EDDs retrieved from email box monitored by Email Retrieval agent.

FTP – EDDs retrieved from remote FTP server by FTP Retrieval agent.

 

Additional channels may be declared for the File Monitor agent so long as the channel designation is no more than three characters long.

 

ST_CONFIG

VALUE

REMARK

VALID VALUES

CONFIG_SECTION

efdManagementConfiguration

Required


CONFIG_KEY

FileMonitor

Required


OBJECT_TYPE

channel

Required


OBJECT_VALUE

EML

FTP

WEB

<custom>

OBJECT_VALUE defines a three character channel designation that maps to a sub-folder of the workfolder setting.

Choose One

STRING_VALUE

<class name>

Define how EDD will be processed for the specified channel.

EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

EarthSoft.Workflow.Clients.Default.DataFileMonitor

EarthSoft.Workflow.Clients.Default.SplitByFacilityFileMonitor

or as directed by EarthSoft.

 

 

Example: DefaultFileMonitor Configuration

 

CONFIG_SECTION

CONFIG_KEY

OBJECT_TYPE

OBJECT_VALUE

STRING_VALUE

efdManagementConfiguration

FileMonitor

channel

EML

EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

efdManagementConfiguration

FileMonitor

channel

WEB

EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

efdManagementConfiguration

FileMonitor

channel

FTP

EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

 

 

DataFileMonitor Configuration

 

Enterprise EDP expects incoming EDDs to be compressed (zipped) files containing the EDD data files and the data submitter's user certificate (*.usr). In some scenarios, such as system integration, it may be desirable to have individual data files processed by Enterprise EDP. For example, a LIMS or other tool may produce flat files, but not have the ability to produce *.zip files with the user certificate. DataFileMonitor is designed to accommodate this scenario.

 

The DataFileMonitor agent will look for files in the channel and repackage them in ZIP containers with a designated user certificate. To accomplish this, DataFileMonitor requires two additional entries in the ST_CONFIG table (see examples below). Those entries combine the text "FileMonitor." with the channel name from OBJECT_VALUE.

<USER_ID> – Enterprise EDP requires that an EDD contain a user certificate identifying the data submitter. <USER_ID> is the USER_ID (from the ST_USER table) that DataFileMonitor will use for creating the user certificate. This user will become the data submitter of the EDDs as the EDDs are processed by Enterprise EDP.

<FILE_NAME> – Enterprise EDP assumes a specific file naming convention for each EDD. The name of the EDD needs to identify the facility to which the data will be loaded and the name of the EDD format. <FILE_NAME> is a string that will be used as a template for the EDD file name that DataFileMonitor produces. It may contain the following (case sensitive) place holders:

{fileDate:yyyyMMdd_HHmm} – This place holder will be replaced with the last write time of the incoming data file (based on server time). This place holder is typically used to ensure uniqueness in the EDD file name.

{fileName} – This place holder will be replaced with the name (without file extension) of the incoming data file.

 

The newly created EDD file will use the specified naming convention.

 

Example: DataFileMonitor Configuration

 

An example scenario might be:

A Laboratory Information Management System (LIMS) produces comma-separated value files (*.csv)

The CSV files are in the EZEDD format

EDDs to be "submitted" by a user with USER_ID = 329

EDDs to be loaded into the SPRINGFIELD facility (aka FACILITY_CODE = 'SPRINGFIELD')

Workfolder setting in ST_CONFIG is C:\Windows\Temp\equisworkflow

 

Enterprise EDP can automatically process these files by adding the following records to the ST_CONFIG table and restarting the workflow service:

 

CONFIG_SECTION

CONFIG_KEY

OBJECT_TYPE

OBJECT_VALUE

STRING_VALUE

efdManagementConfiguration

FileMonitor

channel

LMS

EarthSoft.Workflow.Clients.Default.DataFileMonitor

efdManagementConfiguration

FileMonitor.RMT

user_id

329


efdManagementConfiguration

FileMonitor.RMT

file_name

{fileDate:yyyyMMdd_HHmm}.SPRINGFIELD.EZEDD.zip

 

This example creates a new channel named LMS that accepts files uploaded to the application server via third party tools (i.e., the LIMS). The workflow service will create a folder called C:\Windows\Temp\equisworkflow\LMS. As the LIMS exports CSV files to this folder, the File Monitor agent will repackage each file as a ZIP archive with a user certificate for USER_ID 329 and submit it to the Springfield facility using the EZEDD format. The agent will register the file in ST_FILE_REGISTRATION, where it will continue to be processed as a normal EDD.