Validating and enforcing data types in profile properties

With the release of R60 in October 2018, BlueConic introduced data type validation for profile properties. This helps guarantee that profiles contain valid data, which leads to higher data quality, making your data both more useful and better able to sync with other systems.

Data is validated for the number, decimal, currency, date time, and email address types. At the time of the upgrade, BlueConic analyzed the values of all existing profile properties to find the percentage of valid values for each property.

BlueConic Profile Property Data Types

Profile properties are set to one of the following types: 

  • Text — The value stored in the profile property is in text format. Valid entries include any type of text. Examples: ブルークルー, Blue Crew 
  • Email address — The value contains a valid email address. Example: anyuser@domain.com
  • Number — The value stored in the profile property is in numerical format: 0 to 9 and -. If you select this format, select the units for the number below. The maximum number that can be held by BlueConic is 2,147,483,647. Examples: 1234 and -9876
  • Decimal — The value contains numbers from 0 to 9, - and . with a maximum of 4 decimal places. Examples: 12.3456 and -98
  • Currency — The value contains numbers from 0 to 9, - and .. If you select this format, you can select from over 150 currencies, with the corresponding units. Examples: 12.34 and -98.76
  • Date time — The value stored in the profile property is in Unix millisecond timestamp format: the number of milliseconds since 1-1-1970 UTC. Example: 1538073598692 represents September 27th, 2018 6:39:58 PM UTC

Note: When you create a new profile property using the number, date time, currency, decimal, or email address type, validation is always enabled. (Content stored as text is not validated.) When you set these data types, BlueConic will enforce them.

Once there is at least one profile with a value for the profile property, you can no longer change the data type (except for "Text" profile properties, which can be changed to the type "Email address" at all times).

What happened at the R60 upgrade?

BlueConic analyzed each of your profile properties by sampling up to 10,000 random values for each profile property. It automatically applied validation for profile properties where the analysis showed greater than 99% validation for the data type.

If fewer than 99% of the values for a profile property matched the data type at the time R60 upgraded data, the profile property remained as is with validation disabled. If you want to enable validation or enforce a specific data type, you can enable the validation for these properties manually. You can either change the data type (for example, from text to email address) or clean the data of invalid entries.

Enabling validation for existing number and date time properties

When BlueConic analyzes profile properties set to the number or date time data type, validation will be enabled automatically only for profile properties with more than 99% type validation. 

For profile properties whose type is number or date time, if validation is not yet enabled, the profile property view shows the option to Open validation analysis.

Open_validation_analysis.png

BlueConic will analyze a random sample of up to 10,000 profiles to evaluate the percentage of valid values. For example, if you enable validation for a profile property called "Age" to make sure the values are all valid numbers, BlueConic reports the scope of invalid values, with details on the first 500 invalid values in the sample set, as shown below. You can choose to manually fix the invalid entries (click Cancel) or click Yes, enable to enable validation and remove invalid profile properties.

Number_Age_Validation.png

Enabling validation for email addresses previously stored as text

BlueConic can help you identify email addresses previously stored as text. BlueConic analyzes your content to identify the percent of values of the valid data type. If more than 80% of the values in a profile property are valid email addresses, the text "Not yet enabled" is shown in the Validation column of the profile properties overview. Profile properties with fewer than 80% valid values will show the text "N/a" in the Validation column.

All profile properties of type text can be converted to type email address. In the settings for a profile property, select the "Email address" option from the Type drop-down menu. When you select this option, a pop-up window appears, where BlueConic analyzes the values stored in the profile property. Click Yes, convert to enforce validation. This cleans out invalid email addresses, and invalid entries will no longer be accepted. Click Cancel to leave the profile property as is; the type remains as text, and the invalid data stays in the profiles.

email-analysis.png