When importing timeline events from a data source, especially when dealing with multiple events for the same profile, optimizing the process can significantly enhance performance. This guide provides best practices for configuring connection import setups efficiently to consolidate events and improve import speed.
Consolidating Events
Typically, importing multiple events for the same profile may result in individual bulk operations for each record, which can impact performance. To enhance efficiency, it's advisable to combine multiple timeline events into the same bulk operation by following these steps:
- Let’s assume the following feed is being imported:
- In order to keep seven bulk operations from being imported when there are really two profiles, select the same file twice and link based on the grouping property. In this case, the grouping property is the email address.
- When the file is selected twice, the first file can be used for the profile property mapping and the second file (which is a one to many relationship) can be used for the timeline event mapping.
- Lastly, apply the “Duplicate check” processor to the connection import. Based on an identifier, this data processor skips the record if it was already imported.
Order Bundling
Similarly, the same process can be applied to bundle order and order line data into a single event. By grouping order lines under the same order, you can streamline the import process and avoid duplicate entries.
- Let’s assume the following feed is being imported and you would like to combine all entries that start with “OL-1” to be combined into one order event.
- Select the file twice and link based on the grouping property, which in this case is the order ID.
- This will result in the order lines being connected to the same order, which can then be used for mapping as pictured below
- Add the “Duplicate check” processor.