Data Sets can be used to maintain uniqueness or continuity in the data collected in a form and can facilitate the automation of indexes or interval generation on form records. The configuration tools allow the user to define the parameters that the data set must follow.
Note: Data Sets are required to be configured on all form sections, for both parent and child forms, of new and edited templates in order to use the Sync Data functionality. The Sync Data check box will only be enabled for legacy templates that have not been edited or new/edited templates that have Data Sets configured. |
|---|
Warning: All records in data forms with pre-population enabled must conform to the configured data set constraints to be available in Collect Mobile or the Web Forms widget. During pre-population the data set will, in combination with the Key fields in the pre-population mapping, define the record uniqueness. If there are two records with the same Key field values or data set value, only one will appear in the resulting data form during pre-population. |
Example uses of Data Sets:
•Ensure that borehole measurements (well development, mining, etc.) are recorded without over-lapping depths, keeping a contiguous series of measurements.
•Automate assignment of unique index numbers to data (e.g., observation numbers).
•Prevent the creation of duplicate records in Collect mobile; required for the Sync Data feature to keep records unique when synchronizing data submitted from other users.
To define a data set, select the Data Sets button on the Fields section of the Form Setup page on the Template Designer page.
Available data sets types are:
•Single – User-defined incrementation on a field chosen from the form. When a new record is created, the value in the Start field is incremented according to logic set in an Increment setting. Start must fall within Minimum and Maximum parameters also set by the template designer.
•Interval – Ensures no overlap between data. This is based on fields chosen to act as Start and End values for a data record. When a new record is created in Collect mobile, its Start field is set a value equal to the End value of the previous record. Its length can be defined by a formula entered in the Increment setting.
•Unique – Defines a unique value with no specific start, end or increment; instead, uses one or more selected fields as a key to keep records unique.
Warning: When using data sets in conjunction with pre-population and the Allow Form Refresh functionality, ensure the parameters defined for the data set return unique values. If the values defined by the data set, in the pre-populated records, are not unique they will be deleted when the data form is refreshed in Collect Mobile or the Web Forms widget. |
Select the Data Set type and then configure as desired in the Data Sets Organization window.
For Single and Interval types, configuration options include:
•Start – Defines which field from the current form serves as the Start value for a data record.
•End – (Interval only) Defines which field from the current form serves as the End value for a data record.
•Minimum – Defines the minimum value as a distinct number or by more advanced formulas. Selecting the Minimum button opens the Formula Builder to allow definition of the minimum value.
•Maximum – (Optional) Defines the maximum value as a distinct number or by formula. Selecting the Maximum button opens the Formula Builder to allow definition of the maximum value. If no value is set, data sets will continue incrementing without end, based on Increment.
•Increment – Defines the increase in value based on a distinct number or by more advanced formulas. Selecting the Increment button opens the Formula Builder to allow definition of the increment.
For the Unique type, all fields on the current form will display. Select the field or fields which together should constitute a unique record on the form (e.g., combination of location code and sample code should be unique). When saved, Collect will automatically create a new, hidden field called “Unique Data Set” that will allow uniqueness checks in Collect mobile to take place.
Operations available on the Data Sets Organization window are:
•Cancel – Exits the Data Sets Organization window without saving.
•Clear – Clears the defined data set.
•OK – Saves the defined data set and closes the Data Sets Organization window.
In Collect Mobile, when a user inputs data that causes the Unique Data Sets to match, the "Partition value is not unique" error will be thrown in the Collect Mobile application. To resolve this, ensure the user inputs data that results in a "Unique Data Set" value.
Data Sets and Sync Data
For Data Sync, it is important that records can be uniquely identified. Data Sets are used to define unique records to help Collect determine which records to merge or keep separate during the Sync Data process. The Unique Data Set type is commonly used for Sync Data.
During the syncing of records between two Collect data forms, Collect looks for records where the data sets match up to merge together. If the record's data sets do not match, Collect will keep the records separate during syncing. Data sets that are not unique enough are the leading cause of unwanted behavior during the syncing process. For more information on troubleshooting Sync Data behaviors, see the Sync Issues article.
Note: A Data Set is required for every form in a Collect template if you wish to use Data Sync. |
|---|
Example:
Below is a list of fields for a basic Sample Form:
•Location (sys_loc_code)
•Sample Code (sys_sample_code)
•Sample Date
•Sample Time
•Sample Matrix
•Sample Type
•Start Depth
•End Depth
•Depth Unit
•Sample Remark
•Sampler
•Weather
Only selecting the Location field in the Unique Data Set would not be enough to keep the records unique in the Sample Form because it is possible that each location can have more than one type of sample (e.g., method blanks, field duplicates, etc.). Therefore, for the Sample Form it would be recommended that the Unique Data Set be a concatenation of both the Location field and Sample Type field, or some other combination of fields that ensures the uniqueness of each record.
Unique Data Set = CONCAT([Location],[Sample Type])
The scenario below would occur during the syncing process between two devices in the field. Indicated by the green arrows, the three records on Device 1 match the three records on Device 2. In the resulting data form, each record will merge accordingly and result in three merged records, on for each Unique Data Set.
Records with Data Sets that matched between the two devices were merged, while records with unique Data Sets remain distinct after the sync process.
Volatile Fields in Data Sets
A volatile field is a field in which the value can change. Volatile fields are often the cause of duplicate records as changes in the record's Data Set will result in the records remaining independent during Sync Data rather than being merged.
When choosing fields to define a Data Set, it is important to consider if and how the field's value may change with input from field users in Collect Mobile. Some scenarios to consider are:
•Fields that have a Form Lookup type. Form Lookup fields refer to the #id of a record. During data sync, the order and number of records can change leading to changes in the #id value of a record.
•Formula Fields with volatile functions, such as JOIN or CONCAT, where the final list of values can change during data sync as new data is merged with the local data.
•Pre-populated fields which can change with user input.
Example:
In the scenario below, the SYS_SAMPLE_CODE is a concatenation of the following fields:
TASK_CODE + SYS_LOC_CODE + SAMPLE_TYPE + MATRIX
The SYS_SAMPLE_CODE, TASK_CODE, and SYS_LOC_CODEs are pre-popuated, while the SAMPLE_TYPE and MATRIX fields are populated by the field user.
Unique Data Set = CONCAT([SYS_SAMPLE_CODE])
The user on Device 1 fills out the B-30 record SAMPLE_TYPE and MATRIX code, meaning the SYS_SAMPLE_CODE is now TaskA_B-30_GW_N. Device 2's SYS_SAMPLE_CODE is still the pre-populated value of TaskA_B-30 because the SAMPLE_TYPE and MATRIX fields were not populated. When Device 1 and Device 2 sync their records, Collect cannot recongize that the two B-30 records are supposed to sync which each other to produce a single record because their data sets no longer match. In the resulting data form, the two B-30 records remain distinct.
