Content Collector Connection

The Content Collector Connection collects data about your content and stores it in a BlueConic content store that can be accessed by dialogues making use of Content Recommendations


To set up the connection, click "Collect data from your channels."


1. Select the channel(s) to collect content data from.
Optionally, define URL rules to specify which areas of the sites to collect content data from.

2. Manage which data is collected.
Next, paste an article URL into the "Test URL" area, and click 'Test' to review the metadata that would be collected for your content:


If the "Name", "Description", "Image", "URL", and "Categories" fields are not populated, the webpage may not surface content information in a supported format. BlueConic supports:

To add custom metadata fields click "Add data field". For example, if your content has a sponsorship association, or is tagged based on overarching stories or topics, influencers, or sources, you can use this data in recommendation placement filtering.

3. Set the algorithm timeframe.
Some recommendation algorithms are based on a look-back timeframe (for example, Viral articles or recent high CTR). You can configure that timeframe here, based on hours or days:


 4. Set request headers (optional).
To enable content recommendations to be collected from webpages that are under development or behind a login or paywall, you can add HTTP request headers here. Click "Add request header," choose a channel, and add the custom header name and value used to access the content.




  1. Make sure images are being scraped properly.
  2. Review article names to be sure appendages do not exist, for example " |" may be appended to every title depending where it is being scraped from.