Working with A/B-Testing and Optimization
The consentmanager.net CMP includes the option to perform A/B-Testing and Optimization of the chosen designs. This can help you achieve better results with your website.
Please note, that not all features are available in all packages. See our package comparison here.
A/B-Testing
In order to activate A/B-Testing, simply activate more than one design in your CMP settings:
As soon as more than one design is active, the system will rotate them. This means that for each visitor the system will randomly choose a (different) design to display the consent message.
Reporting
In order to see the results for each design, go to Reports > CMP Report and group the output by Design:
As a result, you will get a report that shows you the numbers per design.
Automatic Optimization
In order to enable the automatic optimization of designs go to Menu > CMPs > Edit and set the Optimization to the desired value:
Once the optimization is active, the system will automatically analyze the data from the A/B-testing. It will do this using machine learning. The system will therefore search for patterns and try to understand which design works best for which group of visitors. Once the system is confident enough, it will start prioritizing the design(s) with the best performance (highest acceptance rate and/or lowest bounce rate). The system will then start to show the design(s) with lower performance less often and therefore show the designs with higher performance more often. As a result only the "good" designs remain and therefore the acceptance-rate increases and/or the bounce rate decreases.
How long will the optimization take?
The system is searching for a confidence level of 95% before it decides that a design is better/worse than another design. In order to achieve this confidence level, either the performance of both designs must be very different or the amount of collected data must be high. If comparing only two designs, the confidence level is usually achieved at around 5000-10000 displays of each design for a single dimension group of visitors (e.g. visitors using Firefox browser). The more data is collected, the more detailed optimization is possible (e.g. visitors using Firefox on a mobile device coming from country X surfing on domain Y on a Friday morning).
How to set the optimization goals?
Usuall Acceptance-Rate and Bounce-Rate are contradictory. Therefore you need to choose the perfered goal for the design optimization. If the setting is set to 100% acceptance rate, the system will only take the acceptance rate into account when comparing the data. If a setting between acceptance and bounce rate is used, the system will take both rates into account and will attach a weight to them when comparing the data.
