What are BlueConic data processors?
Data processors are used in certain BlueConic connections to transform or filter data being exchanged between BlueConic and other systems. Data processors ensure that BlueConic imports and exports the data correctly by cleaning up or transforming the data, to store data in normalized format, or to remove variations in addresses, for example. This helps you to ensure that data being exchanged is validated, corrected, and cleansed.
BlueConic data processors
Here are some examples of available data processors that transform or filter data being imported to or exported from BlueConic:
| Processor | Description |
| Email cleansing | Cleans up email addresses to fix mistakes. For example, correcting first.lastname@gmial.com to first.lastname@gmail.com |
| Add a fixed value | Adds a field with a fixed value when importing data. For example, to set the data source. |
| Convert a value | Allows you to convert values to ones that are clearer for segmentation (e.g., “F” becomes “Female”). |
| Concatenate fields | Links together values from different fields and makes them available as a new field. |
| Add current date | Adds a field with the current date. |
| Address aggregation (U.S. only) | Takes multiple address fields and returns an aggregated address field. This processor supports U.S. addresses only. |
| Name normalization | For example, to change Bob Smith, Bobby Smith, and Robt. Smith to Robert Smith. |
| Log | Creates a history log file of all data in a stringified format for group and profile imports. |
| Filter data | Filters lines in the data set for processing. Configure a regular expression to match values that are stored in the configured profile property and then have the import goal only import lines that have a matching value or instead, only import lines that do not have a matching value. |
| Remove null data | Removes string null values from data from all fields. For example, removing "null" or "NULL" entries. |
| First X lines | Imports or exports the set number of spreadsheet rows only. You can configure the number of lines and the rest is ignored. |
| Phone number cleansing | Normalizes phone numbers to fix errors using an open library. Enter the country code in the settings. |
| Casing data |
Changes the casing of the selected field. For example, if data is provided in uppercase but needs to be stored in lowercase or first upper. |
| Date conversion |
Converts date fields to a specified date format. |
| Hash values |
Hashes incoming values using MD5, SHA-1, SHA-256, or SHA-512 formats. |
| JavaScript |
Applies custom-written JavaScript code functionality to a data source. |
| Split multi-value fields |
Takes a data field with multiple values and places them into a unique field. |
| Duplicate check |
Identifies and skips the import of a data record if it was already imported within the same run. |
Which BlueConic Connections use data processors?
The following BlueConic connections enable you to use data processors:
- Amazon Web Services (S3) Connection
- Azure Blob Storage Connection
- Facebook Leads Connection (for data imports)
- Google BigQuery Connection
- Kafka Connection (for data imports)
- Salesforce Marketing Cloud Connection (for data imports)
- SFTP Connection
- Snowflake Connection
How to use data processors in a connection
Important: To have one or more data processors added to your BlueConic environment, contact your Customer Success Manager.
- Once you have had a data processor added to your BlueConic environment, open the Import or Export goal settings of your connection. See the list of connections that support data processors.
- In the configuration step for processing, transforming, or filtering your data, select each processor you want to use for the data import or export. For example, in the SFTP Connection, you select data processors in step 3:
- For data processors that require configuration, click the configuration (cog wheel) icon and select the relevant data fields from your CSV file. For example, if you select the Email cleansing processor, you would choose the email field. (See also: Best practices for exchanging data via comma-separated values (CSV) files.)
- Some data processors affect the selection of which profiles are exported/imported. In these cases, you'll see a message in the segment selection step, or the Timeline event mappings step alerting you that a data processor might have impact on this aspect of the export.
- Complete the configuration steps, turn the connection on, save your changes, and schedule or run the connection.
For more information, or to install data processors for your connections, contact your BlueConic Customer Success Manager.
Data processors in the BlueConic plugin gallery
You can access and add available data processors via the BlueConic plugin gallery. Open the Settings > Plugins page and click the Add plugin button. This opens a lightbox window with several options for add a new data processor plugin: via the plugin gallery, by uploading a Zip file, or by registering an external URL.
Find related data processors
In a connection window, you can find which data processors are used for the current connection by checking the Related items widget on the right side of the BlueConic window. Learn more about navigating your Related items in BlueConic.