Selecting an Appropriate EDD Format for your Data Needs

<< Click to Display Table of Contents >>

EQuIS 7  >>  EDP > EDP Format Files, Open, Data Review, Tools, EDP Professional and EDP Enterprise > EDP Format Files >

Selecting an Appropriate EDD Format for your Data Needs

Selecting or developing an appropriate EDD Format is one of the first and most important steps in successful data management. EarthSoft publishes and distributes several different EDD Formats that include variations in terms of target-specific data. In many cases, existing, published EDD Formats meet the data criteria required. If so, choosing an existing format can considerably reduce the start-up time necessary to report and load data. In other cases, it may be necessary to select an existing format and modify it to reflect specific data needs. Consider the following:

The selected Format(s) should meet the scope of the data to load. Some workflows may be well suited to using one EDD Format, whereas others may require more than one EDD Format to load different types of data.

The Format(s) should be capable of importing both laboratory and field data as appropriate.

EDD Formats range in complexity. In some cases simplistic EDD Formats such as the single section EZEDD Format can be used if it meets all the required needs. In other cases, formats such as EFWEDD or EQEDD may be appropriate for loading more varieties of data.

Refrain from selecting EDDs that offer unnecessary complexity.

 

EDD Formats - Quick Reference

 

When purchasing Standalone EDP and EQuIS Professional, EarthSoft provides several standard EDD formats for importing data. The formats range from simple single section EDDs to more complex multi-section EDDs. The format selected is dependent on the type of data entered.

 

If the standard, unaltered EQuIS formats do not meet specific company needs, use one of the unencrypted standard formats as a guide for creating a new format or purchase one of the pre-made custom formats that EarthSoft has available, for example, one of the EPA Region formats.

 

Alternatively, EarthSoft can create a custom format, built to exact needs. This requires additional development, and an additional fee.

 

This document is a quick guide for the standard EQuIS formats. It includes a brief description of each.

 

To obtain a blank copy of the EDD or to get the format descriptions, please read EDD Display, Description, and Blank EDD.

 

Standard EarthSoft EDD Formats

 

ActionLevels.xsd

The Action Level format file has two sections:

  1. Action levels

  2. Action level parameters

This format allows you to easily import new action levels or edit existing data as shown in the help article Using the ActionLevels.xsd Format.

EFWEDD.xsd

The EFWEDD format, also known as the 4-file format, is a standard format that is suitable for most labs. It offers two different sections for your sample information (choose either Field or Lab), a test section, a results section, and a batch section.

EQEDD.xse

The EQEDD is one of the most extensive/detailed standard EQuIS formats. It contains sections for location, lab, and field data. It is a good generic format to begin importing data.

ESBasic.xsd

ESBasic is a simple, 1 section lab EDD format. Use to import single sample tests.

EZEDD.xsd

EZEDD is another 1 section lab EDD format. Note that a new SYS_LOC_CODE field has been added to EZEDD, creating the new format EZEDD_LOCATION.

EZLithology.xsd

EZLithology is a 1 section EDD. Use to enter Field/Lithology data.

Facility.xsd

The Facility EDD format allows the importing of facilities with related coordinate and location data as well as associated location groups and group members. The Facility.xsd file contains the following sections that mimic the data table structure:

DT_FACILITY

DT_COORDINATE

DT_LOCATION

RT_GROUP

RT_GROUP_MEMBER

This format can be used to edit existing data by exporting one or more of the above tables from EQuIS Professional using the 'Send to Microsoft Excel' icon, and then opening these from the EDP Facility.xsd format.

GeologyEDD.xsd

The Geology EDD is a more complex location EDD. It contains several sections for importing data related to location, well logs, geology, lithology, geological samples, and water levels.

Historical.xsd

The Historical format contains four sections for importing historical data. The four sections are:

1.Water Levels

2.Geology

3.Chemistry

4.Location Data

Please note that the following changes have been made to the Historical EDD format:

Added default mapping ("NA") for COLUMN_NUMBER

Renamed TOTAL_OR_DISSOLVEDto fraction

Increased LOC_TYPE to 20

Changed WELL_ID to non-required, added conditional mapping, and added custom check "If TOP_CASING_ELEV or DEPTH_TO_BOTTOM_OF_WELL is populated then WELL_ID is required"

Refvals.xsd

This format contains an EDD section for each of the reference value tables within the EQuIS Database Schema. Use this format to import reference values.

WaterLevels.xsd

The Water Levels format is a single section EDD format to enter Water Level Data. This includes SYS_LOC_CODE, measurement date, water level elevation and depth, and remarks.

EuroEQEDD.xse

The EuroEQEDD is a multiple section format that includes sections for Locations, Wells, Water Levels, Field Samples, Field Results, Soil, Gas, and Lab Data.

 

Notes:

EDP now allows for a custom facility check per format. The custom check will scan file names within the zipped EDD (e.g., txt, csv, etc.) for a facility code and match against the facility code in the zipped file name. The files that do not match within the zipped EDD will be ignored. Please contact support@earthsoft.com for more information.

A file section has been added to the EQEDD format to allow uploading of attachments.

The EZEDD format now allows for loading negative result values. The RESULT_VALUE column data type is changed from numeric to string, and a new check allows only numeric values.

 

Data Generator or Data Receiver Considerations

 

Consider EDD format selection from the perspective of a data generator or a data receiver:

A data generator typically acquires data through field observation, populates the EDD, and prepares the data for submission.

A data receiver generally receives the data, loads it into a database, and uses the data for reporting.

 

Consultants often act as the data generator. Industrial users can be both generators and receivers of data. Regulatory agencies are usually receivers of data, although they can also be data generators. The distinction between generator and receiver is not a clear-cut separation. To choose an EDD, consider the requirements from both perspectives.

 

Data Generators

 

Consider the following when selecting an EDD:

Project or Client Requirements

QA/QC Requirements

Regulatory Requirements (EPA, State Agency, Local Agency)

Laboratory Capability

Risk-based Corrective Action (RBCA) or statistical cleanup?

 

Data Receivers

 

Consider the following when selecting an EDD:

Existing Data

oWhat type of data is currently being collected (soil, water, air, etc.)?

oIs field data being collected (lithology, water levels, sample parameters, purge data, well construction, etc)?

oIs analytical data collected? If so, how much QC is collected?

Field Data Requirements

Analytical/Monitoring Data Required

Long-term Objectives

Ability of Community to Produce EDD

 

An essential issue for data receivers, such as a regulatory agency, is the ability of the regulated community to produce the EDD. If the staff cannot or do not use the EDD, the technical requirements are irrelevant. The tools that are provided should make the EDD easy to use. It should include:

Clear documentation with the EDD formats.

The use of tools such as the EDP to check data, and make sure it conforms to requirements before submitting it.

Organized public forums to educate users and address concerns.

 

Comprehensive vs. Simple Formats

 

While single formats can meet all data requirements, a single format may not promote EDD usability.

 

Generally it is possible to input simple data into a comprehensive format. However, if only a few formats are used in a comprehensive EDD or a few fields in a large format, a simpler format is often more efficient.

 

Often, a comprehensive format is used for current data, whereas a simple format is used for historical data where there is little or no control over what data is available.

 

The ordering of the EDD formats in either a comprehensive or simple EDD is arbitrary. Consider ordering by the most important or commonly used format first, followed by others in order of importance.

 

Required Fields

 

Required fields are generally designated in EarthSoft EDDs with header information in red. These fields are required by the database and cannot be changed. However, in developing an EDD you may want to consider requiring additional fields. For example, in the Location format, only sys_loc_code is required by the database. Technically, it is possible to successfully load a data set that includes nothing more than location codes. However, for any type of analysis or modeling, more information is required than just the location code. For use in any type of GIS, the X and Y coordinates are needed at minimum. If the users intend to use GIS or any other modeling tools, it may be necessary  to require additional fields such as X and Y in your EDD.

 

Custom Fields

 

Several of the major tables in the EQuIS database include custom fields. While these fields may be used essentially to accommodate any information, their use should be coordinated among all users within the organization. This is particularly important in regards to the EDD.

 

Custom fields are intended to be used for storing data for which there is no other field. However, without proper consideration, any use of the data once loaded into the database will be difficult at best. If different users put different types of data into Custom Field 1, for example, it becomes a hodgepodge of unusable data because it cannot be queried or reported. However, if it is determined beforehand that Custom Field 1 will be used for atmospheric conditions at the time of surveying (for example) — and the EDD is modified to reflect this rather than a generic CUSTOM_FIELD_1 — then this data becomes usable. Reports will consistently be able to draw the expected information from this field.

 

The use of custom fields should be consistent not only within a single database project, but also across all projects. This issue becomes even more critical for users who may eventually use EQuIS Enterprise where data from many projects are uploaded into a single Enterprise database.

 

Format Variations

 

As EDD formats change over time, it is recommended that the various formats include a version number. For example, if a format is developed for location data called and is named Location, it is recommended that the format be introduced with the name Location_01. As the format evolves, a new version may be called "Location_02". Avoid eliminating one format when a new one is introduced. Thus, data populated with the old format can still be loaded, and new data can be loaded with the new format.