Create a real-time decisioning strategy that helps make smarter, faster, data-driven choices about what to do next. In this use case, you'll use AI Workbench to set goals and let BlueConic determine the most effective next step for the customer. In this article, we will walk through the steps to prepare to execute this use case.
Focus Area | Ideal For | Difficulty | Key Success Metrics |
Customer Engagement |
| Advanced |
|
Skills Required
Prior to beginning, ensure you can:
Create and modify:
Profile Properties
Timeline Events
Listeners
Segments
Connections
Dialogues
The Simulator
AI Workbench*
Identify and involve your internal marketing teams/decision makers
Track KPIs and metrics using Insights and external systems
* This use case relies on AI workbench, which requires knowledge of Jupyter and Python. To execute Next Best Action, you will need either:
Your data science team (or Partner) with the skill set and capacity to develop and test the propensity model.
OR
BlueConic will build and manage the model. Once developed, your data science team will have full access to the code and can modify the notebook as desired.
Implementing this use case
Because of the complexity of this use case, you will be working closely with BlueConic for implementation. As part of that process, you'll complete the NBA Discovery Framework to guide you through working with the BlueConic team to ensure that there are clear, validated inputs and all of the required elements are identified to execute Next Best Action. This guide will walk through the process of the NBA Discovery Framework.
Review the data
The first step is to review the current state of your use case.
Existing CTAs: review your existing CTAs and determine which will be included, or if additional CTAs will be needed.
Conversion paths and decision rules: review your current decisioning rules, including those for:
Audiences for Dialogues
Segments
Current Data Sources: Inbound and outbound connections.
KPIs: Determine what the current KPIs are. These baselines will be used to compare against once the NBA use case is launched. Some examples include:
Paywall
Number of Conversions
Average revenue per user
Paywall exit rate
Paywall interaction rate
Abandon Cart
Conversion rate
Number of abandoners
Exit rate at each step
Abandon rate based on device
Content recommendations
Click-through rate
Time spent on site
Content consumption
Paid media
Conversion
Return on ad spend
Cost per acquisition
Click-through rate
Cost per click
Impressions
Retention
Winback reactivation
Email engagement (opens, click-throughs)
Define your actions, hypotheses, and conversions
Next, you'll work to determine actions, hypotheses, and conversion moments. Actions are what the model will show based on the visitor's behavior and inputs. The model determines what the visitor will see based on scoring and predictive analysis.
Actions and their value
Examples of Actions:
Show a paywall
Show Newsletter sign-up
Prompt Mobile app install
Show Paid subscription Paywall
Make a Discount offer
Advertising impression
Show a product
When you identify the actions, you’ll also need to determine:
The lifetime value of the action
The historical average conversion rate
The eligibility segment- what segment criteria must be true based on this action?
Develop a hypothesis
Identify predictors:
Next, take time and think through what potential actions (predictors) could influence an action value. Below is a catalogue of 20 high‑leverage predictors (features) you can use to form your hypothesis. All of them are:
Collectable in BlueConic through native Listeners or standard Connection imports (CRM, ESP, OMS, loyalty, etc.).
Neutral with respect to privacy - no sensitive health or children’s data.
Mapped to the verticals where they’ve proven most predictive of action value.
Predictor / Feature | Why It Often Predicts Higher (or Lower) Action Value | Typical Data Source(s) | Works Best For* |
DMA / Region (city, state, zip, Designated Market Area) | Geography drives shipping cost, ad CPMs, local news relevance, or blackout rules. | IP‑to‑geo Listener, CRM address, store‑finder data | Publishing, Retail, C‑store |
Distance to Nearest Store / Branch | Shorter distance ↑ likelihood to redeem store coupons or click “Buy Online, Pick Up In Store.” | Store‑locator API, CRM “home store” lookup, “Geolocation” AIWB notebook | Retail, C‑store, Finance |
Visit Hour & Day‑of‑Week | Signals shopper mindset (commuter vs. evening couch‑shopper) and campaign “send‑time” effectiveness. | BlueConic “Visit Listener” | All |
Visit Frequency (last 7/30 days) | High frequency = loyalty or research phase → more receptive to upsell or subscription messages. | Behavioral profile properties | Publishing, Retail |
Scroll Depth / Page Engagement Score | Indicates content relevance and attention span; deeper scrollers accept longer forms. | Scroll‑Depth Listener, Interest Ranker | Publishing, SaaS |
Content / Category Affinity (top 3) | Drives click‑through on contextual CTAs (“More on Oncology” banner, “Matching Shoes” module). | Interest Ranker, Tag‑based metadata | Publishing, Retail, CPG |
Paywall Meter Views Remaining | Scarcity effect—visitors with 0–1 free articles left convert at > 4× average. | Content Meter Listener | Publishing |
Subscriber / Loyalty Tier (yes/no + tier) | Subscribers may skip paywall, but accept cross‑sell (events, newsletters); loyalty gold tier clicks “VIP drop” offer. | CRM, ESP, loyalty file import | Publishing, Retail, C‑store |
Last Purchase Recency (days) | Recency strongly predicts repeat purchase likelihood; decays quickly for CPG replenishment. | Orders feed, POS, OMS import | Retail, CPG |
Average Order Value (AOV) Band | Higher AOV shoppers see bigger basket offers; low AOV get voucher nudges to trade up. | Transaction import, calculated profile property via rollup or AIWB notebook | Retail |
Basket / Cart Size (items, value) | Real‑time cart context triggers financing offers, add‑on suggestions, or free‑shipping thresholds. | Abandon Cart Tracker Connection, Data Layer | Retail, CPG |
Device & Channel (mobile‑web, app, desktop) | Influences form length tolerance, payment options, and creative layout (e.g., mobile paywall with Apple Pay CTA). | User agent parser, SDK property (platform) | All |
Referral Source / Campaign UTM | Paid‑search visitors convert differently than organic; referrer can pre‑qualify finance leads. | Global Listener (UTM) | Retail, Finance, SaaS |
Weather Condition in DMA (rain, >90 °F) | Weather swings store footfall, drink sales, outdoor gear demand, and content appetite. | Weather API enrichment → property import or AIWB notebook | Retail, C‑store, Publishing |
Household Income Band | Higher income correlates with premium product uptake; lower income with coupon affinity. | Third‑party append, CRM survey | Retail, Finance |
Age Range | Age drives creative tone and channel mix; Gen Z engages differently from Boomers. | CRM, registration form, survey import | Retail, Publishing, Finance |
Education Level | Long‑form editorial or complex financial product messaging performs better with higher education segment. | CRM, declared field | Publishing, Finance |
Email Engagement Score (open/click last 30 days) | Highly engaged email readers accept newsletter cross‑sell; dormant readers need reactivation CTAs. | ESP engagement file import | Publishing, Retail |
Product Lifecycle Stage (trial, active, renewal, lapsed) | Dictates next best cross‑sell vs. retention action, especially for SaaS and subscription publishers. | Subscription/CRM feed | Publishing, SaaS, Finance |
Consent Status & Preferences (marketing, tracking) | Ensures model only selects actions that respect privacy choices; lack of consent lowers value of ad‑targeting actions. | Privacy Management listeners, profile flags | All |
*Vertical key—Publishing/Media, Retail/eCommerce, Consumer Packaged Goods (CPG), Financial Services, Convenience Stores / QSR, B2B SaaS. Most predictors work cross‑industry; fourth column indicates where they’re typically most influential.
Create your hypothesis
Identify 10-12 predictors that you believe matter most for each action. Create a hypothesis for the predictor based on what you believe it will predict higher or lower action values for your business. For example:
Predictor | Why It Often Predicts Higher (or Lower) Action Value | Hypothesis |
Scroll Depth / Page Engagement Score | Indicates content relevance and attention span; deeper scrollers accept longer forms. | Visitors who scroll 60% of a page are more likely to subscribe. |
Referral Source / Campaign UTM | Paid‑search visitors convert differently than organic; referrer can pre‑qualify finance leads. | Visitors who visit via newsletter links are more likely to subscribe. |
Determine conversions
What signal tells BlueConic that the conversion happened? What is the trigger? How does that data come into BlueConic? Outline your conversions for each action (e.g. subscribed to newsletter) and how the conversion is tracked either in BlueConic or in an outside system and imported into BlueConic.
Data availability spreadsheet
Create a spreadsheet based on actions, predictors, and conversion events you have chosen above. Identify the hypothesis, profile property the data is mapped to, and the source of the data. This will help validate where the data is coming from and identify any gaps.
When filling out the table, think about:
Is the data already in BlueConic?
How is it captured (Listener, connection)?
Will a new listener or connection be required?
Does the source require any flattening, mapping, cleanup, or Timeline Event Rollup creation?
Example:
Hypothesis | Profile Property | Source | Comments | Expected effect (positive/ negative/ unknown) |
Visitors who scroll 50% of a page twice a week are more likely to subscribe. | Scroll_50 | Scroll depth listener
Scroll Timeline Event rollup |
| Positive |
Visitors who visit via newsletter links are more likely to subscribe. | utm_source | UTM Listener |
| Positive |
Create and modify
The BlueConic team will work with you to implement Next Best Action. Below are examples of some feature items that will need to be created or modified.
Exact items to be created will depend on tech stack, action items, predictors, and conversions.
Listeners
Realtime Model Listener
The Realtime Model Listener will need to be added to your plugins by BlueConic. This will be done in the implementation process.
Add the plugin
The Realtime Model Listener is a plugin that requires an extra step before setting up the listener. To add the plugin to your tenant:
Select BlueConic Settings> Plugins
Click Add Plugin
Search for the Realtime Model Listener plugin with the calendar icon> Add plugin. The plugin will be automatically installed.
Configure the listener
Select Listeners > Add Listener >Realtime Model Listener
Select which channel(s) where the listener is active.
Select the segment of profiles for which a new prediction should be triggered.
Click Add rule to define when a new prediction should be triggered.
Specify S3 Credentials-
Enter your Amazon Web Services (S3) access key ID. You can retrieve this via the security credentials page.
Enter the Amazon Web Services (S3) secret access key. You can create a new access key for your account by going to the security credentials page. In the Access Key section, click Create New Access Key.
Select the profile properties that contain the input data for your model
Configure data transformations and model executions
Select event (Optional): Select the event trigger after the profile has been updated. This will throw an event to the timeline with the action determined by the model.
Turn the listener On and Save.
Additional listeners
Depending on your identified actions and predictors, you may need to set up additional listeners to gather data. Some examples:
Follow the instructions in the knowledge base articles to set up the listeners.
Connections
Review your data availability spreadsheet source column, identify all connections needed. If the connection is not established, follow the steps in the knowledge base to set up the connection.
Email Service Provider Connections: BlueConic integrates with email service providers to ingest important newsletter subscription information. This is dependent on the vendor, but is typically API-based or flat-file batch processing, depending on needs.
Client-side data layer integration: BlueConic can both create a data layer and ingest data from or inject data into an existing data layer on your website to capture important information.
Data from additional sources: These are not necessary, but can be another data point to train on to improve the models. Connections include API-based or flat-file-based (CSV) integrations, depending on need.
Profile Properties
Profile property creation is dependent on the Actions identified.
Review your Data Availability table and confirm that each of the profile properties in the table has been created and mapped correctly to the appropriate listeners or connections.
Create the following profile properties for use by the Next Best Action model:
All next best action dates: the date for each time a new NBA was calculated
All conversion dates: the date for each conversion event
Next best action: the last predicted next best action
Timeline Events
Create a Timeline Event Type to map decisions made as a result of the Next Best Action
Select More > Timeline events from the navigation bar.
Click the Add timeline event type button.
Enter the name Next Best Action Decision for your new Timeline event type.
(Optional) BlueConic creates a unique ID for the event type based on this name. You can change the ID now, but as soon as you save the new event type, the ID becomes read-only.
Adjust the priority setting if needed. Events have high or low priority, and this determines how long they are stored on the timeline.
Set the retention period to indefinite (default for high priority events), or a fixed period of days (for low priority events).In the Properties section, set up the event attributes you want to track for each event. By default, the Event ID and Event date/time are stored for all events.
Add Event properties as needed for your NBA Action Decisions,
Repeat the process to create an NBA Conversion event.
Segments
Create segments that correspond to each Action. The segment logic will become guardrails that the model must respect.
For example:
Action: Newsletter Sign up
Segment: Visitors not subscribed to any newsletter
Action: Paywall
Segment: Not paid subscriber
Action: Ad View
Segment: Not paid subscriber or subscribed to any newsletter
Dialogues
Once the Next Best Action has been determined, the Realtime Model listener will save the action type name to the Next Best Action profile property, then trigger a configurable event such as “NBA Determined Event”.The Next Best Action Event will be the trigger for serving the NBA dialogues. The action name saved to the Next Best Action profile property will then be used in the dialogue’s ‘Who’ logic to determine which action dialogue should be shown.
Modify current dialogues or create new dialogues to match each action item, and set the who criteria to the Next Best Action Profile Property
Modify the Model
AI Workbench Next Best Action Model
When executing Next Best Action, the Realtime Model listener will interface with the trained model to score the NBA propensity predictions in real-time. The propensity predictions as well as the winning action will be stored in the Next Best Action timeline event along with the input values used when making the propensity predictions. The BlueConic team will take the NBA Discovery Framework you have developed together to modify the model.
The Next Best Action AI Workbench Model is based on the Thompson Sampling model for Causal Uplift Modeling, also referred to as a Partitioned Thompson Sampling contextual multi-armed bandit model or TreeHeuristic. It is a reinforcement learning model designed to manage the explore/exploit tradeoff. This tradeoff involves balancing the act of gathering new training data (exploration) with optimizing for immediate revenue or value (exploitation).
Here's how it works:
Causal T-learner: The model operates as a causal T-learner, meaning it aims to identify the action that yields the highest uplift or value.
Oracle Model and Visitor Segmentation:
For each potential action, a distinct oracle model is trained.
This oracle model's function is to segment visitors into various look-alike groups. The underlying assumption is that visitors sharing similar characteristics, whether demographic or behavioral, will exhibit comparable conversion rates.
These oracle models are structured as decision tree classifiers.
Custom Pruning Algorithm:
To optimize the decision tree specifically for Thompson Sampling, a custom pruning algorithm is applied.
This algorithm evaluates the similarity between two "sibling leaves" (sub-groups within the tree) by determining the highest density interval (HDI) or minimum width Bayesian credible interval (BCI) of the difference between their respective beta distributions.
A Region of Practical Equivalence (ROPE) is defined as 10% of the standard deviation of the Beta distribution of the parent of the two leaves.
If the HDI overlaps with the ROPE, the two leaves are considered too similar and are pruned (combined). This process is recursive, continuing until no further leaves can be pruned.
Propensity and Expected Value Calculation:
When determining the Next Best Action for a given visitor, the model predicts the propensity (likelihood) and its associated uncertainty for each valid action. This prediction is based on the behavior of visitors who resemble the current visitor.
Thompson Sampling then selects a random value from within the predicted propensity distribution. If the model lacks sufficient data, a random value between 0 and 1 is used for the propensity score.
This generated propensity score is then multiplied by the predefined Lifetime Value (LTV) for each action. This calculation yields the expected value (or "reward") for each action. The LTV for each action, along with its historical average conversion rate and eligibility segment, is defined during the NBA Discovery Framework's Week 1.
Action Selection:
Finally, the model selects the action that possesses the highest expected value.
The entire process relies on robust input definitions established during the NBA Discovery Framework. This framework ensures alignment on data, actions, conversion signals, governance, and execution before implementation. Key inputs gathered include:
Mapping existing CTAs and current-state logic.
Defining each action's LTV and eligibility rules.
Identifying and validating model predictors, data signals, and ingestion paths. Predictors are attributes expected to influence action choice, such as DMA, metered views, or subscriber tenure. These are validated for availability in BlueConic.
Paywall/Policy Overrides- Do any editorial or legal rules trump NBA decisions?
Confirming signal capture for all conversions and in-session feedback. The authoritative conversion moment and signal for every action are agreed upon, and any necessary development work for tracking is identified.
Learning time has variance and dependence on several factors across our client base:
Number of visitors coming to your website
Number of features you're using
Number of conversions per day
Differences in behavior for people who convert vs. don't convert
Control and test group design
Some customers use a control group to see how a percentage of visitors who do not receive NBA. To do this, you’ll work with BlueConic to:
Determine the size of the control group: typically 10‑20 % of total visitors
Align on the tracking method for the control group. Create segment criteria determining which visitors fall into which group
Agree on evaluation metrics
The size of the control group can affect the time it takes to train the model. The smaller the control group, the faster the model will train. It’s recommended that the model is allowed to observe 100 % of traffic for faster learning.
Measure Success
Create an insights dashboard to measure success
Report on use case success with Dashboard Insights of your campaign by tracking key performance indicators.
Select Insights> Add dashboard
Name the dashboard: e.g., Next Best Action
Click Add insight and choose the Dialogues Table Insight.
Use the Text filter (e.g., "Next Best Action") to display relevant dialogues.
The Insight table will show views, clicks, and conversions for your dialogues.