Skip to main content
All CollectionsBringing Data InConnectionsAnalytics and Big Data Connections
Overview of streaming firehose connections in BlueConic
Overview of streaming firehose connections in BlueConic
Updated over 2 weeks ago

What are BlueConic firehose connections?

Amazon, Google, and Microsoft event queueing systems provide key capabilities including many-to-many, asynchronous messaging that decouples senders and receivers. These systems work at scale and can handle the potentially millions of events coming their way in real-time via a BlueConic firehose streaming connection.

BlueConic captures data and events at the customer level for known and anonymous users. Handing over this information to your cloud data store in real time gives your team complete freedom in how to process, enrich, and improve existing reports and applications and invent new ones. You can use BlueConic firehose connections to stream BlueConic profiles and event data to other systems in real time.

BlueConic firehose connections:

Use cases

BlueConic customers use big cloud vendor firehose technologies to:

  • Enrich profiles, for example, using machine learning algorithms and then sending the data back to BlueConic in new profile properties for segmentation and targeting

  • Send data to data analytics tools for customer journey analytics, multi-touch attribution, and next-best actions (NBAs)

  • Send profiles to data lakes and data warehouses such as Google BigQuery or Amazon Redshift for modeling and analysis

Customers use these tools to further enrich their customer data, gaining insight into customer segments, conversion events, and next-best actions (NBAs).

What kinds of data can you stream?

  • Unified profiles or only selected profile properties

  • Dialogue views, clicks, and conversions

  • Events from the BlueConic dialogues 'When' tab

  • UTM parameters of landing pages

  • Record of all dialogues a customer saw before completing a specific conversion goal

  • Page view events

Note: Firehose connections do not send BlueConic Timeline events or associated lifecycle information.

Why use a firehose connection instead of batch uploads?

In a word, timeliness. Batch uploads run on a schedule, but firehose streaming connections run in real time, as events occur. When a visitor's profile is updated or an event is triggered, the firehose connection sends a message.

Real-time customer data

The phrase "real time" can mean different things in different contexts, so let's be specific about the timing details:

  • Changes to profile property data (and therefore the segments the profiles are part of) are sent to the stream after being "in rest" for 2 minutes, meaning that no changes occurred in the past 2 minutes.

  • Events are sent every 10 seconds.

Monitoring firehose traffic data

With the potential for passing large amounts of data to the firehose, you'll want to monitor your firehose traffic data. In the BlueConic settings > General section, you can monitor your traffic metrics by clicking Firehose traffic breakdown. A pop-up window opens with a list of all streaming goals and the firehose traffic they generated. See the article on General Settings for more details.

Account usage limits

Note: By using a firehose connection, you will create a stream of data on the network and infrastructure of BlueConic and its third-party suppliers. To prevent excessive use of this capacity by users, certain fair use policies (subject to change by BlueConic) apply to the use of this connection. The usage limits on this traffic that are applicable under this fair use policy can be found by clicking on the Settings icon in your BlueConic tenant and by selecting the General settings. The aggregated maximum amount of traffic you are allowed to generate under our fair use policy is specified in the column “Subscription” of the “Firehose traffic” table. Your actual usage is also specified.

If at any time you need to generate more traffic than the specified fair use maximum in the Subscription column and you want to prevent additional charges, stop using this connection or contact your BlueConic Account Manager for a quote immediately. Either way, your continued use of this connection constitutes your acceptance of any additional charges or fees as they may apply.

Sample streaming messages in BlueConic firehose connections

Streaming updated profile properties

This example shows and sends updated user profiles through a firehose to a cloud system. In the example below, the values of the profile properties "Last Visited Date" and "Email Address" will be sent to the firehose if the profile has changed. (Even if these two profile properties themselves have not changed, these values will be sent to the stream if other values of the profile have changed.)

How do I stream profile properties from BlueConic via firehose connections to Amazon Kinesis, Google Cloud Pub/Sub, or Microsoft Azure Event Hubs?

Message sent through the firehose: You'll notice the profile property IDs sent are 'lastvisitdate' and 'email' for the changed visitor profile. Because the Export segments box is checked, the two segments that the visitor belongs to are sent as well.

{    "eventDateTime":"1538119536237",    "eventType":"PROFILE_CHANGED",    "profile":{       "id":"e6fe897a-0e76-934c-8dd0-844fc380af72",       "properties":{          "lastvisitdate":[             "1538119531026"          ],          "email":[             "[email protected]"          ],       },       "segments":[          {             "id":"0e3e6219-108c-47b4-8342-3bc2fb32f451",             "name":"2 - 4 visits"          },          {             "id":"2caadd28-a4d1-4177-b4c7-7a87b55d58ff",             "name":"Visitors with an email address"          }       ]    } }

Details on timing

Profile changes are sent to the firehose 2 minutes after the last change to the BlueConic profile. Learn about real-time data for firehose streaming.

Streaming customer click data with full profile properties

In this example, we want to send firehose messages when dialogue clicks occur and send full profile properties, the name of the dialogue clicked, and dialogue variants.

How do I stream profile property data from BlueConic via firehose event streaming to Amazon Google and Microsoft event hubs?

Sample message: Note: we removed the full list of profile properties for brevity.

 { "eventDateTime":"1538119536237", "eventType":"DIALOGUE_CLICK_EVENT", "profile":{ "id":"e6fe897a-0e76-934c-8dd0-844fc380af72" }, "dialogue":{ "dialogueId":"09c6816f-e366-5c5b-b73f-2f741e0196ed", "dialogueName":"Subscription Lightbox", "variantId":"e3ae6e06-73be-38bc-bb4f-9e977fccb7a8", "variantName":"Variant A" } }

Streaming customer conversion data with details on variants

It can be valuable to analyze customer conversion events by examining messages -- which dialogues did a customer see or click before the conversion happened?

How do I send customer conversion data via Firehose connections in BlueConic?

Sample message: In this example, BlueConic sends the dialogue that 'won' the conversion (Content Meter ONE), as well as the two dialogue variants that 'lost' the conversion (which are placed in the "otherCandidates" section of the message). The dialogues that lost the conversion have a 'last viewed' or 'last clicked' moment that is older for this profile compared to the winning dialogue.

{ "eventDateTime":"1538119536237", "eventType":"DIALOGUE_CONVERSION_EVENT", "profile":{ "id":"e6fe897a-0e76-934c-8dd0-844fc380af72" }, "dialogue":{ "dialogueId":"09c6816f-e366-6c6d-b73f-2f741e0196ed", "dialogueName":"Content Meter ONE", "variantId":"e3ae6e06-73be-38bc-bb4f-9e977fccb7a8", "variantName":"Variant A" }, "otherCandidates":[ { "dialogueId":"781c7642-ca2c-60b3-8c8c-ee7df348ca37", "dialogueName":"Content Meter TWO", "variantId":"09810450-dfb7-59d8-932f-9ee5220f3abd", "variantName":"Variant A" }, { "dialogueId":"09c6816f-e366-6c6d-b73f-2f741e0196ed", "dialogueName":"Content Meter ONE", "variantId":"856ae2bc-018c-445c-de23-cc91aa6edf34", "variantName":"Variant B" } ] }

Streaming custom event details

BlueConic allows you to trigger custom events that pick up on user behavior or changes. For example, if a visitor puts an item in a shopping cart or signs up for a newsletter, a custom event can be triggered. You can stream these events to the firehose by selecting the events you want to stream (all custom events or selected ones).

How do I stream custom event details via BlueConic Firehose connections?

Sample message: In this message, the custom event measures the customer's exit intent. The message also sends dialogue variants that were shown to this customer after the custom event occurred.

 { "eventDateTime":"1538119536237", "eventType":"CUSTOM_EVENT", "customEventId":"exitintent", "profile":{ "id":"e6fe897a-0e76-934c-8dd0-844fc380af72" }, "dialogues":[ { "dialogueId":"781c7642-ca2c-60b3-8c8c-ee7df348ca37", "dialogueName":"Content Meter TWO", "variantId":"09810450-dfb7-59d8-932f-9ee5220f3abd", "variantName":"Variant C" }, { "dialogueId":"927a8162-cd91-00b1-93ab-8361caed92d6", "dialogueName":"Content Meter THREE", "variantId":"856ae2bc-018c-445c-de23-cc91aa6edf34", "variantName":"Variant A" } ] }
Did this answer your question?