Curamando

Client case: Data-driven business development through A/B-testing

By 27 June, 2018 No Comments

Background

A large client (revenue of approximately 30 billion SEK/year) has several ideas of how to improve customer satisfaction and increase online sales. However, these ideas have a high risk of affecting revenue and costs both online and in other sale channels.

Challenge

There were two main ideas in need of investigation:

  • Will the cost be higher than the benefit if we make our online chat more available for online visitors?
    There is an evident interest among the website visitors to contact customer service through a chat solution. By increasing contact points to customer service and making a chat more available we would increase customer satisfaction but also increase the cost of agents.
  • If we were to remove an additional fee in the online checkout, would the increase of volume make the change profitable?
    Customers experienced an additional administrative fee in the checkout flow. The checkout drop off and overall sales were expected to increase if this fee was to be removed. However, this fee generated an income which is collected in all sales channels (both online and offline). There was also a risk of frustration from customers when experiencing different pricing in different sales channels; for example when visiting the online shop prior to continuing the purchase by call and noticing a higher price.

Solution

Lead by consultants from Curamando, two A/B-test projects were initiated.

An A/B test means presenting different variations of an online experience, where users at random will experience one or the other. By using this method you are able to follow up on the behavioral change and ensure data-driven decisions for a positive impact.

In this case; whether these ideas would improve customer satisfaction and if the increase in online sales would be higher than the additional cost. Two tests were executed:

  1. In close collaboration with the customer service department Curamando created an A/B test to expose a chat solution for a small part of the total traffic on the web page. Exposure was increased gradually in order to adapt resources from agents.
  2. In the other A/B test, 50% of the traffic in the checkout was not exposed to the additional administration fee. Prior to, and during the test, other channels such as stores, and call centers were informed to continuously monitor the effect.

Results

The A/B tests showed a significant change in behavior and provided actionable results:

  1. Having the chat available on the whole website sales increased with 14%
  2. Removing the additional administration fee resulted in an increase in sales of 13%. Not only covering the loss of intake by the fee but creating an overall uplift in revenue.

 

Client case: Data-driven business development through A/B-testing

Based on these results Curamando were able to help the client to create credible business cases where these ideas could be evaluated against other initiatives. The additional fee was removed from the online channel and the client is evaluating different chat solutions to satisfy customer need.

Key success factors

  • Close collaboration with key stakeholders in the client organisation
  • Courage to test ideas with high potential and high risk
  • A project manager with experience in A/B testing pushing the ideas forward