Amazon Web Services (S3) Connection

Amazon Web Services S3

What: The Amazon Web Services (S3) connection allows you to synchronize data from various marketing platforms and databases into BlueConic via delimited flat files on servers. User profiles can be created or enhanced with this data and BlueConic information can be shared with marketing platforms. This connection supports scheduled batch import and export.

About BlueConic: BlueConic is a Customer Data Platform that harnesses the data required to power the recognition of an individual at each interaction, and then synchronizes their intent across the marketing ecosystem. Click here to learn more.

Why: Transfer customer data from and to internal data warehouses, CRM databases, analytics platforms and other marketing systems to enrich BlueConic profiles with purchase history, predictive analytics segmentation (e.g. propensity modeling and lifecycle signals). This data can then be utilized directly within BlueConic or synchronized out to other marketing platforms to interact with users with relevancy and enable right-time messaging, or to enrich data and enhance capabilities with behavioral data collected by BlueConic and profile data integrated from other systems.

Concepts

CSV file format

Before we delve into the CSV file and field handling settings, let's first have a look at the actual text format of the file that we're about to import. Let's assume we're about to import the following data:

This would be exported as the following CSV text file:

id,ib_avg,ib_ddg,ib_z1,ib_z2
aghvww32,4123551,GK,open,3
agkkemw3,4233215,GK,open,0
alfk3oor,4135111,GR,open,
bezz5las,4009852,GR,closed,0
c9uuz5g0,4823507,K,closed,1
cc2rkes6,4222941,G,open,1

The first line of the CSV file is expected to contain column names, separated by a comma. Each following line is expected to contain values with the same amount of columns.

Adding an Amazon Web Services (S3) connection

  1. Click Connections in the Navigation bar.
  2. Click Add Connection.
  3. A pop-up window appears. Check the Show all box. Enter "S3" in the Search bar.

    Click Amazon Web Services (S3) connection.

  4. The Amazon Web Services (S3) connection page opens.

    You can expand or collapse metadata fields (favorite, labels, and description) by clicking the gray + or .

Configuring the connection

In the side bar on the left are a number of tabs. There is a setup section and there are import and export goals. Goals can be activated or deactivated with the checkbox in front of the name. Multiple goals can be added by using the [Add goal] button at the bottom of the page and you can delete or copy goals by hovering over them and selecting the "x" respectively copy icon. 

Amazon-S3-Setup.png 

Click on a goal to open its tab and see the configuration options for that goal. The name of the goal can be changed by editing it at the top of the goal configurations.

Set up and run

This is where you set up the authentication that allows BlueConic to connect to the Amazon Web Services (S3) platform. Look up and enter the configuration parameters:

Access key ID
Enter the Amazon Web Services (S3) access key ID that you can retrieve via the security credentials page.
Secret access key
Enter the Amazon Web Services (S3) secret access key. You can create a new access key for your account by going to the security credentials page. In the "Access Key" section, click "Create New Access Key". After saving the connection, the secret key will be hidden. You can click "Change secret access key" to change the existing key.

For more information on working with credentials, read AWS Security Credentials.

Amazon-S3-access.png

Optional: Add email addresses for email notifications. Check the Notifications settings box if you would like BlueConic to send an email if an error occurs. You can then also check the box for BlueConic to send emails on successful runs.

Import goals

Import goals allow you to import data from Amazon Web Services (S3) into BlueConic. To do this, complete the steps that are outlined in the goal.

1. Select BlueConic domain group for the import.

BlueConic supports domain groups to ensure profiles are exchanged between the right domains. Select a domain group for the import to ensure that the correct profiles are updated or added.

2. Define files to import data from.

Start by entering or selecting the S3 bucket where the files are stored that will be imported.

In order for BlueConic to be able to scan the buckets, the S3 account needs to have the "listAllBuckets" permission enabled.

Your S3 bucket can contain many files. You can specify exactly which file (or files) will be imported from by either entering a name or using wildcards to match multiple names, e.g. 10-import.csv, *.csv or  *import*.

Click [Scan now] to scan the selected S3 bucket for the specified files. Matches are shown in the table below.

amazon-define-files.png

To get a preview of how a file will be scanned, hover over the file and click "Preview fields." Here you can verify that the file and field handling has been set up correctly; are the columns matching what you thought they would be?

3. Define file and field handling.

This is where you define how the contents of the imported files must be interpreted. Define the field separator -- the character that separates the column values on every line. By default this will be the ,, but you can also use other characters such as ; or |, depending on the contents of your file.

Specify which character your files use as a decimal separator, either a point . or a comma , .

You can also set more advanced options:

Multi value separator
If columns contain multiple values, what character separates them within the column? This character has to be different from the field separator.
Date format
Help BlueConic to correctly interpret date values by configuring the date format used in the file.
Time zone
Enter the time zone used to export date values in the file.

field_separators.png

After saving the connection, try clicking the Date and time preview button to see if the date is formatted as expected.

4. Link identifiers between files and BlueConic.

BlueConic can only map imported data to profiles if at least one of the imported columns should be imported to a profile property that is a unique identifier. So, you need to determine how BlueConic can recognize matches:

Select a column name on the left and select the BlueConic unique identifier profile property it should match on the right hand side.

Information will only be imported if an exact match for the linked identifiers is found in an existing profile, unless the checkbox is activated to allow the creation of new profiles. In the latter case a new profile will be created in BlueConic.

5. Map Amazon Web Services (S3) data to BlueConic.

Finally, click Add mapping to configure which CSV values should be mapped to which BlueConic profile properties. If you have multiple fields to map, BlueConic detects the fields and offers the option to map them all at once.

Bulk mapping: Click "Add all detected fields" to set up a bulk mapping between Amazon S3 and BlueConic.
Add a prefix: You can optionally add a prefix for the BlueConic profile properties. If you set the prefix and then add an empty mapping, the right side automatically adds the prefix as well. 

Bulk-Map-Amazon-S3.png

Select from the menu how the value(s) should be imported:

  • Set: Always overwrite the profile property value in BlueConic.
  • Set if empty: Import the value in BlueConic only if the profile property does not contain a value yet.
  • Add: Add the imported value to the list of values for the profile property. The list will only contain unique values, doubles are not imported.
  • Sum: Numerically add the imported value to the value of the profile property in BlueConic. If the profile property was empty, it will be treated as 0. If the imported value is not a number, it is not imported.

You do not have to create a mapping and import every field of the CSV into BlueConic; there is no need to clutter BlueConic's data storage with values that serve no purpose in BlueConic or other connected systems. It is a best practice to only import values that are of interest to BlueConic use cases.

6. Run the connection.

To run the connection, you need to Save your changes and turn the connection On. See Running the Amazon Web Services (S3) Connection below for details.

Check the box for Import data into BlueConic in the left panel. A green arrow appears, going from the S3 icon to the BlueConic icon.

If this is the first time you are running the connection, it is a good idea to do a test run with a small example file, to see if everything works as intended. If you are satisfied with the result, you change the import to the actual, full size file.

Export goals

Export goals enable you to send data from BlueConic to Amazon Web Services (S3). To do this, complete the steps that are outlined in the goal.

1. Select a BlueConic segment to export profile data from.

Using the Select segment Search bar, choose a BlueConic segment. Only profiles in this segment with a value for at least one of the selected profile properties (from Step 4) are exported.

In the top right-hand corner, the number of profiles available for the export appears.

profiles.png

This number shows how many profiles meet the conditions for the selected segment (for example, they have an email address and gave required consent for an objective) and therefore are available for the export. BlueConic tries to export only the profiles that have changed since the last export. This number of profiles exported may be lower than the number displayed here. 

2. Set up directory to export to.

Enter the Amazon Web Services (S3) directory for the exported files, or click the folder icon to select a directory. View the files already in the directory in the Files detected table.

Optional: Click Scan now to rescan the directory to retrieve updated files and data fields.

3. Determine how to set up the files and their fields.

Select how to name the exported files. Click the Segment name + timestamp radio button to add a new file every time the export runs. Click the Custom name radio button to replace the file with a newer version every time the export is run.

Specify how the files' fields should be handled by entering a field separator. This is usually ,;, or \t (for tab-separated files).

Specify whether your files use a point or a comma as a decimal separator.

Optional: Click Show advanced settings for additional options. Here, you can enter a multi-value separator, replace BlueConic IDs with the file name, compress the exported files, set a custom date format, and select your time zone. Click Date and time preview to test if the date values in the detected files match your selected date format.

4. Map BlueConic data to Amazon Web Services (S3).

Select the BlueConic data you want to export into Amazon Web Services (S3).

Select a BlueConic profile property by entering a search term and enter the data field value it should populate. Order mappings by grabbing the drag bar on the left side; this will be reflected in the order of the fields in the exported file.
Click the BlueConic icon to open a drop down menu where you can switch between the type of information you want to export. Pick one of:

  • BlueConic profile identifier: The unique identifier for a profile
  • Associated segments: All segments, or a selection of one or more specific segments that the profile is associated with
  • Profile property: The value of a specific profile property
  • All viewed interactions (all time): The interactions that the profile has seen
  • Permission level: The permission level set in the profile
  • Text value: Enter any static text
    text_export_options.png

Click Save on the top right corner of the screen to save your Export goal.

If you want to create another Export goal, click Add goal and then Export goal in the left panel. 

In the left panel, check the box next to Export data into BlueConic. A green arrow appears, going from the BlueConic icon to the Amazon Web Services (S3) icon.

 

Running the Amazon Web Services (S3) connection

Tip: Make sure you have added Import and Export goals before running the connection.

  1. Turn the Amazon Web Services (S3) connection on.
  2. Go to the Authentication and run page. Scroll down to the Run history section.
  3. To start the import and export manually, click the Run now button. 
  4. To schedule the import and export for a future date, click the Settings icon Screen_Shot_2018-03-15_at_2.14.53_PM.png. Select how to schedule the import by choosing an option from the drop-down list:
    • Every X minutes
    • Number of times per day
    • Days of the week
    • Days of the month
    • Weekday of the month

    Set a time for the import.

    Click OK.

Data exchange shows you the total number of unique profiles that have been imported or exported via the connection. Note that these are unique profiles. If the same profile is exported multiple times, it is counted as 1.

Privacy management

Connections can be added to Objectives, allowing for privacy management of the information that is being picked up. A connection will only process the profiles of visitors who have consented to at least one of the objectives that the connection is linked to.

Frequently asked questions:

If a file has already been imported, will it be imported again? 

If the connection configuration has not been changed, and a file has already been imported, it will not be imported again. If the file has changed, or the configuration has changed, files will be reimported.

The Amazon Web Services (S3) Connection is a standard plugin.