Live Web Response Saving

If an EQuIS Live agent requests web data (e.g., Adcon, Weather Underground, and NOAA NCDC), the web responses that it receives can be saved to files. The files will be added to a zip file when the agent finishes running.

The zip file will be named ‘{0}-V1-{1:yyyy-MM-ddTHHmmss}.zip’, where:

•{0} is the report name.

•V1 is the code version that generated the web response files, and the other files in this zip.

•{1:yyyy-MM-ddTHHmmss} is the date and time that the zip file was created.

The zip file will contain the following files:

•parameters.xml – The report’s parameters, and their values.

•uris.txt – The web request URIs, in the order they were requested.

•1.* – The first web response data file (e.g., 1.json); its URI is on the 1st line in uris.txt.

•n.* – The nth web response data file; its URI is on the nth line in uris.txt.

Enabling

The web response saving functionality is disabled by default. To enable it, add one or more rows to the ST_CONFIG table, with:

•CONFIG_SECTION = EarthSoft.Live.Reports.*, where * is a class name from the table below
(e.g., EarthSoft.Live.Reports.UsgsWaterServicesDailyValuesAgent).

•CONFIG_KEY = webResponseSaving

•OBJECT_VALUE = save

•STRING_VALUE = true

The zip file will be saved to the EQuIS logs folder (defined in log4net.config) by default, but a different folder can be specified by adding a row to the ST_CONFIG table, with:

•CONFIG_SECTION = EarthSoft.Live.Reports.*, where * is a class name from the table below.

•CONFIG_KEY = webResponseSaving

•OBJECT_VALUE = folder

•STRING_VALUE = the folder in which to save the zip file (e.g., C:\EarthSoft\Live\Reports).

Agent	Report Name	Class Name
Adcon	EQuIS Live: Adcon Data Collection Agent EQuIS Live: Adcon Historical Data Loader	AdconAgent AdconHistoricalDataLoader
Air Monitors	EQuIS Live: Air Monitors Agent EQuIS Live: Air Monitors Historical Data Loader	AirMonitorsAgent AirMonitorsHistoricalDataLoader
Air Ontario	EQuIS Live: Air Ontario Agent EQuIS Live: Air Ontario Historical Data Loader	AirOntarioAgent AirOntarioHistoricalDataLoader
DWD CDC	EQuIS Live: DWD CDC Agent EQuIS Live: DWD CDC Historical Data Loader	DwdCdcAgent DwdCdcHistoricalDataLoader
Eleven-X	EQuIS Live: Eleven-X Agent EQuIS Live: Eleven-X Historical Data Loader	ElevenXAgent ElevenXHistoricalDataLoader
EPA AirNow API	EQuIS Live: EPA AirNow Agent EQuIS Live: EPA AirNow Historical Data Loader	EpaAirNowAgent EpaAirNowHistoricalDataLoader
In-Situ	EQuIS Live: InSitu Data Collection Agent	InSituAgent
In-Situ HydroVu	EQuIS Live: In-Situ HydroVu Agent EQuIS Live: In-Situ HydroVu Historical Data Loader	InSituHydroVuAgent InSituHydroVuHistoricalDataLoader
NOAA NCDC	EQuIS Live: NOAA NCDC Agent EQuIS Live: NOAA NCDC Historical Data Loader	NoaaNcdcAgent NoaaNcdcHistoricalDataLoader
NOAA NWS METAR	EQuIS Live: NOAA NWS METAR Agent	NoaaNwsMetarAgent
Open Data Location Scout	EQuIS Live: Open Data Location Scout	OpenDataLocationScout
Outpost Central	EQuIS Live: Outpost Central Agent EQuIS Live: Outpost Central Historical Data Loader	OutpostCentralAgent OutpostCentralHistoricalDataLoader
UK Gov Flood Monitoring	EQuIS Live: UK Government Flood Monitoring Agent EQuIS Live: UK Government Flood Monitoring Historical Data Loader	UkGovFloodMonitoringAgent UkGovFloodMonitoringHistoricalDataLoader
USGS EDDN	EQuIS Live: USGS EDDN Agent EQuIS Live: USGS EDDN Historical Data Loader	UsgsEddnAgent UsgsEddnHistoricalDataLoader
USGS Water Quality	EQuIS Live: USGS Water Quality Current Values Agent EQuIS Live: USGS Water Quality Daily Values Agent EQuIS Live: USGS Water Quality Historical Data Loader	UsgsWaterQualityCurrentValuesAgent UsgsWaterQualityDailyValuesAgent UsgsWaterQualityHistoricalDataLoader
USGS Water Services	EQuIS Live: USGS Water Services Instantaneous Values Agent EQuIS Live: USGS Water Services Daily Values Agent EQuIS Live: USGS Water Services Groundwater Levels Agent EQuIS Live: USGS Water Services Historical Data Loader	UsgsWaterServicesInstantaneousValuesAgent UsgsWaterServicesDailyValuesAgent UsgsWaterServicesGroundwaterLevels Agent UsgsWaterServicesHistoricalDataLoader
Weather Canada	EQuIS Live: Weather Canada Agent	WeatherCanadaAgent
Weather Underground	EQuIS Live: Weather Underground Agent	WeatherUAgent
Other?		If there is another EQuIS Live agent that requests web data and is not listed here, please let us know.

Note: The CONFIG_SECTION is the namespace and CLASS name, not the REPORT name, to be consistent with other rows in the ST_CONFIG table (e.g., EarthSoft.Common.Utilities.File) and Performance Tracking.

Example: The following SQL enables web response saving for the USGS Water Services Daily Values Agent.

IF OBJECT_ID('tempdb..#set_st_config') IS NOT NULL DROP PROCEDURE #set_st_config

CREATE PROCEDURE #set_st_config(@config_key VARCHAR(128), @config_section VARCHAR(128), @object_value VARCHAR(128), @string_value VARCHAR(2048)) AS

BEGIN

IF NOT EXISTS(SELECT * FROM st_config WHERE config_section = @config_section AND config_key = @config_key AND object_value = @object_value)

BEGIN

INSERT INTO st_config

(config_section, config_key, object_value, string_value)

VALUES

(@config_section, @config_key, @object_value, @string_value)

END

ELSE

BEGIN

UPDATE st_config

SET string_value = @string_value

WHERE config_section = @config_section AND config_key = @config_key AND object_value = @object_value

END

RETURN

END

EXEC #set_st_config 'webResponseSaving', 'EarthSoft.Live.Reports.UsgsWaterServicesDailyValuesAgent', 'save', 'true'

--EXEC #set_st_config 'webResponseSaving', 'EarthSoft.Live.Reports.UsgsWaterServicesDailyValuesAgent', 'folder', 'C:\EarthSoft\temp'