Overview
The Data Hygiene dashboard is designed to surface issues in your data and help you fix them. The dashboard covers three main areas of concern on the Person record, attributes, applications and milestones. Use the downloadable file from the dashboard to fix errors in your preferred spreadsheet app. Finally, Import the corrected data to Element451.
Accessing the Data Hygiene Dashboard
The Data Hygiene dashboard can be found via the Insights sub-menu. The Insights module can be accessed from the Data + Automations dropdown in the top navigation.
Dashboard Features
Use the Data Hygiene dashboard to find errors in commonly used data and correct them. The dashboard will provide you with a downloadable file that you can then modify and Import back into Element451, correcting the data issues.
Tabs
The Overview Tab
The Overview tab provides a quick summation of data quality across three key dimensions of the Person record: their direct attributes, Applications, and Milestones.
Person Record Errors Summary: Shows the number of errors across 7 commonly used attributes:
Intended Term
Intended Major
Intended Degree
Intended Campus
Intended Student Type
Citizenship Status
Citizenship Country
Each card shows the number of errors. A total card shows the total number of errors, and the percent of Person records with Application errors against all Person records.Application Data Errors Summary: Shows the number of errors across 7 commonly used attributes:
Application Term
Application Major
Application Degree
Application Campus
Application Student Type
Application Status
Application Type
Each card shows the number of errors. A total card shows the total number of errors and the percent of Person records with errors against all Person records.Milestone Data Errors Summary: Shows the number of errors across 3 commonly used attributes:
Milestone Term
Milestone Major
Milestone Student Type
Each card shows the number of errors. A total card shows the total number of errors, and the percent of Person records with Milestone errors against all Person records.
The Error Report Tab
The Error Report tab allows you to select a field, observe errors for that field and it's expected values, then download a file of errors. Additionally, lookup up Import tasks that may have created the error.
Fixing Data Errors
The Data Hygiene dashboard is designed to give you a file of Person records with errored data. You can then correct the data in a spreadsheet application of your choice and import the corrected data via the Import module.
The following instructions assume that you are familiar with Importing Data into Element451.
Field Mappings
Use these mappings when Importing:
Person Record Attributes
Field | Mapping Slug |
Citizenship Country | user-citizenship-country |
Citizenship Status | user-citizenship-status |
Intended Campus | user-education-campus |
Intended Degree | user-education-degree |
Intended Major | user-education-preferred-major |
Intended Student Type | user-education-student-type |
Intended Term | user-education-term |
Application Attributes
Field | Mapping Slug |
Application Campus | user-applications-campus-* |
Application Degree | user-applications-degree-* |
Application Major | user-applications-major-* |
Application Status | user-applications-status-* |
Application Student Type | user-applications-student-type-* |
Application Term | user-applications-term-* |
Milestone Attributes
Field | Mapping Slug |
Milestone Major | user-milestones-major-* |
Milestone Student Type | user-milestones-student-type-* |
Milestone Term | user-milestones-term-* |
Example 1: Fixing Intended Term
Watch the following video as we walk through an example:
Intended Term is a direct attribute of the Person record. It does not repeat. This makes it straightforward to fix, compared to Application and Milestone attributes.
Review the Step-by-Step Process
Review the Step-by-Step Process
Select Intended Term as the Active Field: On the "Error Report" tab, find the Person Record Errors section at the top. Set the Active Field control to "Intended Term."
Observe the Error Values and Expected Values: With Intended Term selected, observe the "Records with Errors" table, and contrast it to the "Expected Values" table. Notice that the expected values are "GUIDs", the unique identifier for Terms in the Element451 database.
Download the Records with Errors Table: Download the "Records with Errors" table using the three-dot menu in the top right corner of the table. CSV and Excel options are available, either should work fine.
Open the Downloaded File in Your Spreadsheet App: In your spreadsheet app, use find and replace to select error data and replace it with the correct value. Replacing the error with the Term GUID is preferable, but replacing the error with Term code or Term name is also acceptable.
Save the Corrected File as a CSV: After errored data has been corrected, save the file as a .csv. This will be required when uploading to Element451.
Create an Import Task in Element451 and Upload the Correct File as the Source: On the Source tab of the Import Task, selected the corrected file from your local computer.
Map the Columns of the Corrected File: On the Mapping tab, map the Element ID column to
user-elementid. Map the Intended Term column touser-education-term.
Set Intended Term Column Settings: In Intended Term column settings, set "interpret as" to be the value you corrected your error data to. If you found errors and replaced them with the Term GUID, set "interpret as" to "guid". If you replaced with Term code, set "interpret as" to "code."
IMPORTANT: This step is critical to fixing the data. Data errors will persist if imported data is not correctly transformed.
Match the Element ID Column: On the Matching tab, match entities by "Element ID". This will match imported data to the existing Person records.
Preview the Mappings: On the Preview tab, observe the right column, "Database", to verify that incoming data is being transformed properly. For Intended Term, Term GUID should be shown in the right column.
Import the Corrected Values and Verify: Run the Import task, then verify that Intended Term is correct on the Person Profile. The Data Hygiene dashboard will also update, but may take longer than an individual Person Profile. Please allow up to 12 hours for changes to be visible on the dashboard.
Example 2 : Fixing Application Major
Watch the following video as we walk through an example:
Application Major is an attribute of the Person record's Applications. Applications have a many-to-one relationship with the student record, and are stored as an array on the Person Record. To target a specific Application Major, we'll need to know it's "position" in the array. The position is provided in the "Records with Errors" table, and is selected along with Active Field.
Review the Step-by-Step Process
Review the Step-by-Step Process
Select Application Major as the Active Field: On the "Error Report" tab, find the "Application Errors" section in the middle. Set the Active Field control to "Application Major". Set the Application Position control to "1."
Observe the Error Values and Expected Values: With Application Major selected, observe the "Records with Errors" table, and contrast it to the "Expected Values" table. Notice that the expected values are "GUIDs", the unique identifier for Majors in the Element451 database.
Download the Records with Errors Table: Download the "Records with Errors" table using the three-dot menu in the top right corner of the table. CSV and Excel options are available, either should work fine.
Open the Downloaded File in Your Spreadsheet App: In your spreadsheet app, use find and replace to select error data and replace it with the correct value. Replacing the error with the Major GUID is preferable, but replacing the error with Major code or Major name is also acceptable.
Save the Corrected File as a CSV: After errored data has been corrected, save the file as a .csv. This will be required when uploading to Element451.
Create an Import Task in Element451 and Upload the Correct File as the Source: On the Source tab of the Import Task, selected the corrected file from your local computer.
Map the Columns of the Corrected File: On the Mapping tab, map the Element ID column to
user-elementid. Map the Application Major column touser-applications-major-*. Leave the "Position" column unmapped.
Set Application Term Column Settings: In Application Major column settings, set "interpret as" to be the value you corrected your error data to. If you found errors and replaced them with the Term GUID, set "interpret as" to "guid". If you replaced with Term code, set "interpret as" to "code."
Additionally, set the Position Scope of the column to the Position selected on the dashboard. In this example, we're working with Position 1.
IMPORTANT: This step is critical to fixing the data. Data errors will persist if imported data is not correctly transformed. New data errors may occur if Position is incorrectly configured.
Match the Element ID Column: On the Matching tab, match entities by "Element ID." This will match imported data to the existing Person records.
Preview the Mappings: On the Preview tab, observe the right column, "Database", to verify that incoming data is being transformed properly. For Application Major, Major GUID should be shown in the right column.
Import the Corrected Values and Verify: Run the Import task, then verify that Application Major is correct on the Person Profile. The Data Hygiene dashboard will also update, but may take longer than an individual Person Profile. Please allow up to 12 hours for changes to be visible on the dashboard.
Additional Considerations
Transformation Errors
Errors take many forms, but it's common for errors to simply be incorrectly transformed data. For example, when importing Intended Term into Element451, it's common to accidentally import the Term Code, instead of transforming the code to GUID in the column settings. Learn more about transforming codes to GUIDs during Import.
In this case, we can modify our steps for fixing the issue. Referring to the "Fixing Intended Term" example, we can skip Step 4. Simply download the file from the dashboard and upload it straight into the Import task. Set the "Interpret as" settings to correctly identify the Term code. Then run the task.
Finding Data Errors in Import Tasks
Import tasks are the most common place where data errors occur. Imports are highly customizable to fit a variety of needs, but if incoming data is not properly interpreted and transformed, data can be incorrectly saved to the database. Learn more about data errors.
The Data Hygiene dashboard makes it easy to find imports that could have caused data errors. Use the "Import Tasks Mapping to.." table on the "Error Report" tab to see Import tasks that could have caused data errors for the selected.
Troubleshooting and FAQ
What constitutes a data "error"?
A data error occurs when an unexpected data value is saved to the database. For each field on the Person record, Element451 anticipates a certain type of data in a certain format. For Data-Source-controlled fields, like Term, Major and Student Type, Element451 also anticipates only specific values as defined by the Data Source.
As shown in the examples above, a data error occurred for the Intended Term field when the value "Fall 2022" was saved to the database. The database anticipated the Term GUID.
How did my instance get so many data errors?
Data errors for Data-Source-controlled fields usually occur during an Import. It's common to accidentally mis-configure the column settings for these fields, forgetting to set their interpretation to the correct value. Pay special attention to the transformations when mapping.
How can I fix Application Type errors?
Application Type cannot be changed via an Import or through the Element451 interface. Please reach out to Element451 support for assistance with these errors.
How often does the Data Hygiene dashboard update?
The Data Hygiene dashboard currently refreshes twice a day at 6:00 am and 12:00 pm Eastern. Depending on the size of the dataset, the refresh can take up to 30 minutes. You will see changes to the dashboard shortly after the refresh times.