<< Click to Display Table of Contents >> Navigation:  Administration and Configuration > ST_CONFIG Entries > Enterprise Workflow Service > EFD Management > EFD Management Configuration – File Monitor

Overview

Channel Options

DataFileMonitor Configuration

 

Overview

 

The EarthSoft workflow service File Monitor agent loads EDD files to an EQuIS database in preparation for processing by Enterprise EDP. The File Monitor agent scans folders for new EDDs. The folders are determined by combining the EQuIS work folder with the channels (e.g., web, email, FTP) defined by the efdManagementConfiguration.FileMonitor configuration settings.

 

FileMonitor agent configuration includes definition of EDD delivery channels and the client assembly associated with them. The defined client assembly determines what the FileMonitor agent does with any EDD files found in the specified channel folder. Three options are available for the default client assembly (EarthSoft.Workflow.Clients.Default):

•EarthSoft.Workflow.Clients.Default.DefaultFileMonitor

•EarthSoft.Workflow.Clients.Default.DataFileMonitor

•EarthSoft.Workflow.Clients.Default.SplitByFacilityFileMonitor

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.DefaultFileMonitor, files with a file name extension that matches an inboundFileType are loaded into the ST_FILE_REGISTRATION table and removed from the channel folder. Files with a file name extension that do not match an inboundFileType are moved to the .\archive\ folder.

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.DataFileMonitor, a new EDD is created for each file in the channel folder. The EDD contains the data file and user certificated as specified in the configuration. The original file is removed from the channel folder and the newly created EDD is loaded into the ST_FILE_REGISTRATION table. With this channel configuration all files are processed and no files are moved to the .\archive\ folder. See the DataFileMonitor Configuration section below for more details.

 

When the channel is configured as EarthSoft.Workflow.Clients.Default.SplitByFacilityFileMonitor, the EDD file name must specify a format that is registered. Files with names that do not contain a registered format or have file name extensions that do not match configured inboundFileTypes are moved to the .\archive\ folder. If the EDD file name specifies a FACILITY_CODE or the format does not support SplitByFacility, then the file is loaded into the ST_FILE_REGISTRATION table. For EDD files named with a registered format that supports SplitByFacility and do not specify a facility, SplitByFacilityFileMonitor will split the file into a separate EDD for each facility denoted in the designated FACILITY_CODE column.

 

The agent can be completely customized through development of a custom client assembly (DLL). The default client assembly (EarthSoft.Workflow.Clients.Default) supports all the capabilities of other client assemblies. The major difference between the Default and other Client Assemblies is the EDD file naming convention.

 

Channel Options

 

By default, EarthSoft defines three channels for the File Monitor agent:

•WEB – EDDs uploaded via EQuIS Enterprise EDP EDD Upload widget or Sign and Submit web service.

•EML – EDDs retrieved from email box monitored by Email Retrieval agent.

•FTP – EDDs retrieved from remote FTP server by FTP Retrieval agent.

 

Additional channels may be declared for the File Monitor agent so long as the channel designation is no more than three characters long.

 

 

Example: DefaultFileMonitor Configuration

 

 

DataFileMonitor Configuration

 

Enterprise EDP expects incoming EDDs to be compressed (zipped) files containing the EDD data files and the data submitter's user certificate (*.usr). In some scenarios, such as system integration, it may be desirable to have individual data files processed by Enterprise EDP. For example, a LIMS or other tool may produce flat files, but not have the ability to produce *.zip files with the user certificate. DataFileMonitor is designed to accommodate this scenario.

 

The DataFileMonitor agent will look for files in the channel and repackage them in ZIP containers with a designated user certificate. To accomplish this, DataFileMonitor requires two additional entries in the ST_CONFIG table (see examples below). Those entries combine the text "FileMonitor." with the channel name from OBJECT_VALUE.

•<USER_ID> – Enterprise EDP requires that an EDD contain a user certificate identifying the data submitter. <USER_ID> is the USER_ID (from the ST_USER table) that DataFileMonitor will use for creating the user certificate. This user will become the data submitter of the EDDs as the EDDs are processed by Enterprise EDP.

•<FILE_NAME> – Enterprise EDP assumes a specific file naming convention for each EDD. The name of the EDD needs to identify the facility to which the data will be loaded and the name of the EDD format. <FILE_NAME> is a string that will be used as a template for the EDD file name that DataFileMonitor produces. It may contain the following (case sensitive) place holders:

•{fileDate:yyyyMMdd_HHmm} – This place holder will be replaced with the last write time of the incoming data file (based on server time). This place holder is typically used to ensure uniqueness in the EDD file name.

•{fileName} – This place holder will be replaced with the name (without file extension) of the incoming data file.

 

The newly created EDD file will use the specified naming convention.

 

Example: DataFileMonitor Configuration

 

An example scenario might be:

•A Laboratory Information Management System (LIMS) produces comma-separated value files (*.csv)

•The CSV files are in the EZEDD format

•EDDs to be "submitted" by a user with USER_ID = 329

•EDDs to be loaded into the SPRINGFIELD facility (aka FACILITY_CODE = 'SPRINGFIELD')

•Workfolder setting in ST_CONFIG is C:\Windows\Temp\equisworkflow

 

Enterprise EDP can automatically process these files by adding the following records to the ST_CONFIG table and restarting the workflow service:

 

 

This example creates a new channel named LMS that accepts files uploaded to the application server via third party tools (i.e., the LIMS). The workflow service will create a folder called C:\Windows\Temp\equisworkflow\LMS. As the LIMS exports CSV files to this folder, the File Monitor agent will repackage each file as a ZIP archive with a user certificate for USER_ID 329 and submit it to the Springfield facility using the EZEDD format. The agent will register the file in ST_FILE_REGISTRATION, where it will continue to be processed as a normal EDD.