10. Benchmarking & Competitive Analysis

2. Benchmarking against business objectives:
   Benchmarking design decisions against products’ business objectives helps to see whether various proposed improvements make the product more or less successful in achieving its goals, thus meeting its business objectives. This is the best way of justifying design changes.The process of benchmarking a product against its business objectives:

1. Decide on the key business objective behind the product or service – how does the organization make or save money with the product. The key objective could be to sell some products, get users sign up for a newsletter, donate, view ads or contact to arrange a service. The key business objective should be clear, not generic such as “become a popular website”.
2. Identify the UX factors that will help to achieve the key objective. Some examples of such factors are the ability to search for products easier, the ability to view a product from all angles or to compare products in a simple way. You often have to do some research to identify them, for example send an online survey to a sample of the target audience members. The UX factors are assumptions that need testing.
3. Propose a way to work on improving the UX factors – specific design activities that need to be carried out to meet the UX objective. For example, you could run a usability test to see if there are any problems at the moment with e.g., the current product comparison tool, create some design ideas to improve it.
4. Measure the benchmark state of each UX factor (see the steps 1-2 and 4-7 in the previous section about standard usability benchmarking). You need to derive some values to assess the current performance, e.g. the success rate or efficiency (refer to lesson 6 for usability metrics). Measure the performance of your product to get the current values which will be used for comparison later. After that set some realistic targets for improvement, e.g. if the current success rate of the product comparison tool is 50%, set it to 75%.
5. Track changes in each UX factor until target values are achieved. Start improving the interface and testing changes with users to see whether you are getting closer to your target values, e.g. whether the success rate your product comparison tool is getting closer to 75%. Use unmoderated user testing.
6. Test if the business objective is being met. Once the targets are met, you need to check whether your assumption that these factors help to achieve the business goal was correct. The best way of doing it is running an A/B test (a comparison of two versions of a product by showing them to similar visitors at the same time), version A being the original interface, version B – the interface with UX improvements. It is important to ensure that version B does not include a lot of new features or many major changes, since you will not be able to tell which ones lead to an improvement, or whether the change is due to an improvement to the UX factors or something else. If it does not help in meeting the business objective, it is likely that wrong UX factors were identified.

Competitive analysis (competitor benchmarking):

It is a process of comparing the performance of a website’s (or another digital product’s) user interface to the performance of its competitor’s interface in order to identify its strengths and weaknesses. The comparison can be holistic – ranking competing products by overall usability metrics, or it could be focused, comparing specific features or elements.

The process of competitive analysis:

1. Identify your goals – what exactly you want to achieve by benchmarking? To compare the overall performance or you would like to focus on how your product search compares to similar features of competitors’ websites?
2. Choose competitors, typically 2 to 3 are chosen, more than that can be expensive and overwhelming to analyze. Choose your direct competitors who provide the best user experience.
3. Carry out steps 1-6 of the section about standard usability benchmarking.
4. Carry out testing. Each participant should complete some tasks on 2-3 products (more is overwhelming), at the end of the session participants should be asked to comment on the products used and on how they compare, what they liked, what they found confusing (most unmoderated testing tools allow creating questions).
5. Analyze results – see how well your product performs compared to its competitors. Unmoderated testing tools will easily provide data on KPIs, however, the main goal should not be to declare a winner, but to improve your design, thus look at the biggest strengths in competing designs, what trends arise across sites. Base your future design decisions on what you discovered to work well and avoid the mistakes your competitors make.

Expert reviews could also be used for competitor benchmarking, it is a cheaper alternative (lesson 8 discuses the limitations of it). A usability expert could evaluate the competing products, looking for relative strengths and weaknesses, trends, patterns and differences. It helps to identify what is missing in your product, where usability is inferior compared to competitors’. Reviews can be broad or narrow (focusing on a particular feature, e.g. checkout).