Dialogue Optimization

A dialogue is a way to get your message across to visitors. Sometimes there are multiple ways to get the same message across. For example: display a lightbox, or change content to display a banner, or use a toaster at the bottom of the page. You will want to use the message that most effectively communicates your message to visitors.

Enter dialogue optimization.

BlueConic will help you figure out the most effective way of reaching a goal that you define. Configure BlueConic to evaluate the scores for the variants of a dialogue and then take a specific action. For example, you can see whether displaying a lightbox consistently leads to more clicks than displaying a banner. If the lightbox leads to more clicks or conversions, for example, then you can tell the optimizer to automatically target website visitors with the more effective lightbox and stop showing the less effective banner.

 


Creating variants

Before you can start optimization, you will first need multiple variants for your dialogue. New variants can be created by clicking the [Add Variant] option under the existing variants under the Optimization tab of the dialogue menu. The normal dialogue is then converted into an optimizer dialogue.

Dialogues with multiple variants subtly change their navigation bar:

  1. Instead of the "What" tab, you now have a menu showing "Variant A" and the other variants. The menu allows you to quickly switch between variants.
  2. The menu bar now also show a "Add Variant" shortcut to quickly add a new variant.

This navigation bar change also implies an interesting tidbit: the "Who", "When", "Where" and "Why" settings are the same for the entire dialogue, regardless of the selected variant. So for example, if you configured a specific segment on the "Who" tab, that segment is being used by all variants. If you want to serve your alternative dialogue to a different segment of visitors ("Who" tab), this cannot be done by adding a variant; you will have to create a new dialogue to do this.

 


Configuring optimization settings

The first thing you notice on the Optimization tab is the table listing all existing variants:

The first row in this table is the Original, in analytics this is called the Control Group. This represents the original content that a visitor would see if none of the variants of the dialogue was shown. It allows you to test whether or not the dialogue is improving results over the original content. After all, why have a dialogue if it isn't effective?

You can add or remove columns to the table by clicking the cog icon on the top right of the table.

The following information is shown in the table: 

Property Description
Name The name of the variant. To navigate to the settings for a variant, click it in the list.
Distribution

The distribution percentage shows what chance a variant presently has of being served to new visitors. This value is read only, except for the original control group variant and for when the optimizer strategy is set to "A/B testing".

For variants with an "A/B testing" Variants Strategy, you can fill in a distribution percentage in this column for all variants. The percentages you enter determine how often the variants are shown to visitors in relation to each other. For example, if you enter 70% for Variant A and 30% for Variant B, 70% of the visitors will be served Variant A and 30% of the visitors will be served Variant B.

If the percentages you enter do not add up to 100% precisely, the values will be normalized so that they do equal 100%. For example, if there are two variants, one with the distribution of 30 and the other with a distribution of 50 (80% in total), BlueConic will normalize the 30 value to 37.5% (30 * (100/80)) and the 50 value to 62.5% (50 * (100/80)). Another example: if there are two variants, one with the distribution of 1 and the other with a distribution of 4 (5% in total), BlueConic will normalize the 4 value to 80% (4 * (100/5)) and the 1 value to 20% (1 * (100/5)). If you add more variants and percentage values, the normalizations will be recalculated.

Type Content, Lightbox, Form etc.
Views The number of times the variant has been viewed.
Clicks The number of times the variant has been clicked.
Conversions The number of times the variant has led to a conversion.
Click / Views Ratio The percentage of times that the variant has led to a click out of the total number of times the variant has been viewed.
Conversions / Views Ratio The percentage of times that the variant has led to a conversion out of the total number of times the variant has been viewed.
Conversions / Clicks Ratio The percentage of times that the variant has led to a conversion out of the total number of times the variant has been clicked.
Lift Clicks / Views Ratio The percentage of times that the variant has led to a click out of the total number of times the variant has been viewed, relative to that same percentage for the control group. Positive lift means the variant is doing better than the control group, negative lift means it is doing worse.
Lift Conversions / Views Ratio The percentage of times that the variant has led to a conversion out of the total number of times the variant has been viewed, relative to that same percentage for the control group. Positive lift means the variant is doing better than the control group, negative lift means it is doing worse.
Lift Conversions / Clicks Ratio The percentage of times that the variant has led to a conversion out of the total number of times the variant has been clicked, relative to that same percentage for the control group. Positive lift means the variant is doing better than the control group, negative lift means it is doing worse.
On/Off With the on/off switch you can switch individual variants on or off and determine whether or not they should be actively used in the optimizer. The On/Off switch can have three values:

 The variant is switched off 
 The variant is switched on, and it is active. (The optimizer containing it is active)
Copy / Delete When you hover over a line in the table in the last column a copy and a delete icon will appear. Using these you can either copy a variant or delete it from the list.

 

Creating a new variant

To create a new variant from the "Variants" tab, click the [Add Variant] - button below the table.

 

Optimization Goal

Select the deciding factor in determining success of a variant from the menu:

These are the available goals:

Goal Description
Clicks/Views Ratio Top variant is based on the best ratio of views that lead to a click. More clicks per view means the variant is considered to be more successful.
Conversions/Views Ratio Top variant is based on the best ratio of views that lead to a conversion. More conversions per view means the variant is considered to be more successful.
Conversions/Clicks Ratio Top variant is based on the best ratio of clicks that lead to a conversion. More conversions per click means the variant is considered to be more successful.

 

Variants Strategy

Select the strategy that will be used by BlueConic to pick the variant of the dialogue that a user will see when encountering the dialogue for the first time. On subsequent encounters the visitor will see the same variant as presented the first time.

The options are:

Strategy Description
Automatic Optimization BlueConic determines which variant it will serve to a visitor with one aim: to achieve the highest score for the selected Optimization Goal. The decision is based on historical values gathered from all variants. At the outset, BlueConic selects variants to show on a random basis. However, over time, the variants that consistently score higher are selected more often. Variants that score lower are still served to visitors, only less often. The process is continuous and at any time any variant can be the one that is served to visitors most often. It all depends on which variants are currently achieving higher scores. If you add a new variant, at first it will be shown more often in order to score its effectiveness compared to the other variants, however, once enough historical data is gathered, it will be served using the same criteria as all other variants.
A/B testing For each visitor, a randomly chosen variant will be targeted at them the first time they trigger the optimizer. For each subsequent time that the optimizer is triggered for the same visitor, they will be targeted with the same variant that they had the first time. You can define the distribution percentage for each variant in the variants overview table lower on the screen. BlueConic will distribute the variants among visitors based on the ratios you define.
Rotating The variants will be targeted to visitors in a round robin fashion. For example, if there are three variants A, B, and C, BlueConic will cycle through these variants for each visitor individually as the optimizer is triggered by them. For example, the first time a unique visitor triggers the optimizer, they are served variant C. The second time the same visitor triggers the optimizer, they are served variant A. The following time they trigger the optimizer, they are served variant B. Once they have seen all the variants, the same rotating pattern will be used for that visitor, in this case (C-A-B-C-A-B, and so forth).

For more information about the available strategies and usage examples, see Optimization use cases.
Select the "Variants Strategy".

 

Evaluate At

The Evaluate At setting makes it possible for you to determine how the variants are performing with respect to each other at a specific point. For example, this allows you to configure optimization to stop showing all variants except the one that performs the best or automatically serve up the variants that score higher more often than those that score lower. The options for the evaluation period are:

Setting Description
Views The "After Evaluation" action will be in force after the number of views specified. The number of views is counted against all variants in the optimizer. For example, if you have an optimizer with four variants (Variant A, Variant B, Variant C, and Variant D) and the number of views is 10, any combination of views between the four variants that adds up to 10 triggers the evaluation.
Clicks The "After Evaluation" action will be in force after the number of clicks specified. The number of clicks is counted against all variants in the optimizer. For example, if you have an optimizer with three variants (Variant A, Variant B and Variant C) and the number of clicks is 5, any combination of clicks between the three variants that adds up to 5 triggers the evaluation.
Conversions The "After Evaluation" action will be in force after the number of conversions specified. The number of conversions is counted against all variants in the optimizer. For example, if you have an optimizer with two variants (Variant A and Variant B) and the number of conversions is 20, any combination of conversions between the two variants that adds up to 20 triggers the evaluation.
Date The effectiveness of the variants will be evaluated on the date specified.

You can define more than one of the above options. When the first is matched, the evaluation will take place.

Enter the desired value(s) in the text field(s). If you want to base the evaluation on a date, click the date picker and select the date from the calendar. For example:

Changing the "Evaluate At" settings during the evaluation period

If you change the "Evaluate At" criteria before any of the goals has been reached (views, clicks, conversions, and/or date), the historical data for the variants is kept and the tracking of values continues until the evaluation period ends.


After Evaluation

This setting allows you to take a specific action after the evaluation period ends. The options are:

Setting Description
Automatic Optimization BlueConic uses the Automatic Optimization strategy to serve variants to visitors.
Stop serving any variant The optimizer will not target any more visitors with variants.
Serve specific variant The optimizer will serve the variant you specify to all website visitors. After you select "Serve specific variant", a second drop-down list will appear from which you can select the variant to be served.
Serve winning variant The optimizer will serve the variant that is determined to be the best according to the Optimization Goal when the evaluation period ends. When the evaluation period ends, the variant that achieved the highest overall score is considered the winning variant and that is the variant that will be served exclusively from this point forward.

Select the "After Evaluation" setting from the drop-down list. For example:

Adding variants during the evaluation period

If you add a variant to an optimizer during the evaluation period, it will be kept track of just as the other variants are and its distribution values can be seen in the variants overview table.

Adding variants after the evaluation period ends

If you add a variant to an optimizer after the evaluation has occurred, what happens next depends on the "After Evaluation" setting. The following table summarizes what will happen with new variants added after an evaluation period ends.

Setting Description
Automatic Optimization The new variant will be served along with the existing variants and its effectiveness will be scored. As time passes, if its effectiveness is higher than the existing variants, it will be served increasingly more often than those that are less effective. The process continues as such with the currently best performing variants always being served more often than those that perform less well.
Stop serving any variant The new variant will not be served to any visitors.
Serve specific variant The new variant will only be served if it is selected as the specific variant to use.
Serve winning variant The new variant will not be shown.


Configuring variants

You can configure a variant by navigating to the variants section. You can do this by either clicking on a variant name in the variants table, or by selecting the variant from the drop down list in the menu.

The appearance of the variants section will depend on the type of variant that is chosen. For complete information on the configuration of a specific type of variant, click the Interaction and Variant types tab above and select the variant type from the drop-down list.

In the right hand sidebar you will find a properties widget for the variant.

In this widget you can:

 

  • Change the name for the variant
  • Switch the variant on or off. The On/Off switch has three states:

     The variant is switched off

     The variant is switched on, and it is active. The dialogue itself is also active.

  • Change the type of plugin for that variant. The default plugin that is selected when a new dialogue is created is "content".
  • For some interaction types additional settings can be made in this properties windows, for example for a lightbox you can specify the size of the lightbox in this widget.

Optimization use cases

Given the multiple combinations possible when mixing Variants Strategies and Optimization Goals, deciding which to use can sometimes pose a challenge. This section describes use cases for each strategy that help to illustrate which strategy/goal pairings are best used in specific circumstances.

Rotating

The "Rotating" strategy serves the defined variants in a rotating order. For example, if you have three variants (A, B, and C), they will be served in the same order each time to every visitor triggering the dialogue. For example, the first time the visitor triggers a variant they will be served variant B. The next time they trigger a variant, they will be served variant C. The following time, variant A. Each successive time, the order B-C-A will be used again. This strategy applies to all visitors.

The "Rotating" strategy is best used when:

  • The messages contained within the variants are seen as equal when compared to each other.
  • You want each visitor to see all the messages in order to maximize the chance that one of them will catch the visitor's attention.
  • You want to spread more than one message to each visitor in the likelihood that two or more messages will complement each other and lead to stronger engagement.
  • You want to target the visitor with more than one type of interaction in order to see which is most effective (banner, lightbox, content, etc.).

When using the "Rotating" strategy, the variants are definitive and you typically will not add any more to the optimizer during the evaluation period.

For example, the following three banner variants have been added to an optimizer and will be shown on a rotating basis at the same position on the web page:



The messages contained in these three variants are all different selling points but at the same time they are equal and do not conflict with each other. Because you do not know which selling point will be most effective, you want each visitor to see all three variants in order to spread all messages equally. Are visitors more influenced by the ability to enhance their website through personalization, comply with online privacy regulations or create long lasting relationships? Or are they influenced by the fact that they can both add personalization and comply with online privacy laws using your product? The "Rotating" strategy makes it possible to engage your website visitors with a differing message for each visit.

A/B testing

The "A/B testing" strategy selects a variant at random from the optimizer and serves the same variant to the unique visitor from then on each time they visit your website. This strategy is best used when:

  • You want to divide your website visitors into separate groups in order to be able to directly compare the effectiveness of each variant with the other variants.
  • The messages contained within the variants conflict with each other and you only want each visitor to see one of them and not all of them.
  • Seeing more than one of the variants could lead to an inconsistent visitor experience.

Consider the following example: you are offering a discount on the sale of an item but do not know whether it is better to offer an exact monetary savings value or a discount based on a percentage of the full price. Which is better?


Variant A


Variant B


Variant A offers a 20% discount off the full price of an item while Variant B offers the concrete amount of $15. By serving each message exclusively to two groups of website visitors, you can determine over time which of the offers is more appealing.

Another example: You have three variants containing messages that range from formal, to informal, to highly personal:

Formal


Informal


Highly personal


Some visitors will be more engaged by the formal, some by the informal and some by the highly personal. However, if the tone of the message that the visitor receives varies with each visit, this could lead to confusion or a degraded experience that could lead to disengagement. Using the "A/B testing" variant, the experience of the visitor remains consistent.

Automatic optimization

Using the "Automatic" optimization, you let BlueConic perform the optimization for you. This strategy is based on a self-learning mechanism that continuously monitors historical data gathered from all variants and serves to visitors more often the variant(s) that are achieving the Optimization Goal at the highest rates at the given time. Over time the performance of variants can vary and the "Automatic" strategy will change the variants it is serving based on the current situation. The "Automatic" strategy is best used in the following scenarios:

  • You want to maximize clicks and conversions.
  • You want to limit the loss of conversions during the evaluation period.
  • You want to continue to modify, add or remove variants during the evaluation period.
  • You do not know how long you need to test the variants in order to achieve significant results.
  • In all situations where the "Rotating" and "A/B testing" strategies are not effective or desired.

Variants comparison line graph

For each optimizer, you can view a line graph that shows how each of the variants is performing. For complete information, see Variants comparison line graph.