Help Center

It is important to validate your data before pushing it to a production environment so that anomalies, errors, and misconfigurations can be detected. Data validation also helps prevent impacts on profile merging errors that can be hard to undo. To complete validation on a data source, the specialized Data Validation Connection can be used in your BlueConic tenant. This connection adds profile data from CSV files and BlueConic segments to SQLite tables to enable you to define queries comparing the source data with the imported profiles in BlueConic.

The primary goal of the Data Validation Connection is to analyze discrepancies when the numbers from the source system do not match the numbers in BlueConic. For example, if you import data from an SFTP source with 100,000 users, but only 30,000 profiles appear in BlueConic, this connection can help identify why certain users are not being imported. By importing the source file into a database table, you can run queries to pinpoint the differences and address any inconsistencies.

Installing BlueConic plugins

The first step in using this connection is to install the Data Validation Connection plugin to your BlueConic tenant. By default, this plugin is not installed in BlueConic.

To install, navigate to BlueConic settings > Plugins, click the Add plugin button at the top right, and register the external URL below.

https://plugins.blueconic.net/connection_data_validation/index.xml

For more detailed instructions, review the article Overview: What are BlueConic plugins? If you need further assistance, reach out to BlueConic Support.

Creating a Data Validation Connection

1. Once the Data Validation Connection is added to your tenant, you can create one by navigating to the Connections tab and clicking Add Connection. When the pop-up window appears, enter "Data validation” in the search bar and click the Data Validation Connection.
2. The Data Validation Connection page will open. You can expand or collapse metadata fields (favorite, labels, and description) by clicking the gray chevron at the top of the page. Enter a name for the connection and Save your settings.

Defining sources for validation

To configure the functionality of this connection, click Define your files/segments you want to use for data validation underneath the "Define sources" header in the right menu.

1. Select file to download.
   If using a file for data validation, first choose the BlueConic connection where the file exists from the dropdown menu. Then, enter the file path on the left side and a custom table name on the right (this name will be used in the SQL queries below). To add multiple files, click the “Add file” button. Once you've added all of your files, you can click the cogwheel icon next to the table name field to load the data into tables.
   
2. Select segment to download.
   If using a BlueConic segment for data validation, enter the segment on the left side and a table name on the right. You can also choose specific profile properties from the segments to include in the table. To add multiple segments, click the “Add segment” button. Once you've added all of your segments, you can click the cogwheel icon next to the table name field to load the data into tables.
   
3. Define your SQL queries.
   Enter the SQL queries you want to apply to the data from tables created in the previous steps.
   
4. Run your SQL queries.
   To run your queries on the data sources defined in steps one and two, click the cogwheel icon next to the query field. The tables will appear underneath each query once they are applied to the data sources.