Competition

In its SEC (2005) filing Amazon describes the environment for its products and services as ‘intensely competitive’. It views its main current and potential competitors as: (1) physical-world retailers, catalogue retailers, publishers, vendors, distributors and manufacturers

beyond the finance. Marcus (2004) describes an occasion at a corporate ‘boot-camp’ in January 1997 when Amazon CEO Jeff Bezos ‘saw the light’:

‘At Amazon, we will have a Culture of Metrics,’ he said while addressing his senior staff. He went on to explain how web-based business gave Amazon an ‘amazing window into human behavior.’ Marcus says: ‘Gone were the fuzzy approximations of focus groups, the anecdotal fudging and smoke blowing from the marketing department. A company like Amazon could (and did) record every move a visitor made, every last click and twitch of the mouse. As the data piled up into virtual heaps, hummocks and mountain ranges, you could draw all sorts of conclusions about their chimerical nature, the consumer. In this sense, Amazon was not merely a store, but an immense repository of facts. All we needed were the right equations to plug into them.’

James Marcus then goes on to give a fascinating insight into a breakout group discussion of how Amazon could better use measures to improve its performance. Marcus was in the Bezos group, brainstorming customer-centric metrics. Marcus (2004) summarises the dialogue, led by Bezos:

‘First, we figure out which things we’d like to measure on the site,’ he said. ‘For example, let’s say we want a metric for customer enjoyment. How could we calculate that?

There was silence. Then somebody ventured: ‘How much time each customer spends on the site?’

Not specific enough,’ Jeff said.

‘How about the average number of minutes each customer spends on the site per session,’ someone else suggested. ‘If that goes up, they’re having a blast.’

But how do we factor in purchase?’ I [Marcus] said feeling proud of myself. ‘Is that a measure of enjoyment?’

‘I think we need to consider frequency of visits, too,’ said a dark- haired woman I didn’t recognize. ‘Lot of folks are still accessing the web with those creepy-crawly modems. Four short visits from them might be just as good as one visit from a guy with a T-1. Maybe better.’

‘Good point,’ Jeff said. ‘And anyway, enjoyment is just the start. In the end, we should be measuring customer ecstasy.’

It is interesting that Amazon was having this debate about the elements of RFM analysis (described in Chapter  9) in 1997, after already having achieved $16 million of revenue in the previous year. Of course,

these questions since actual consumer behaviour is the best way to decide upon tactics.

Marcus (2004) also notes that Amazon has a culture of experiments, of which A/B tests are key components. Examples where A/B tests are used include new home page design, moving features around the page, different algorithms for recommendations, changing search relevance rankings. These involve testing a new treatment against a previous control for a limited time of a few days or a week. The system will randomly show one or more treatments to visitors and measure a range of parameters such as units sold and revenue by category (and total), session time, session length, etc. The new features will usually be launched if the desired metrics are statistically significantly better. Statistical tests are a challenge though, as distributions are not normal (they have a large mass at zero, for example, of no purchase). There are other challenges since multiple A/B tests are running every day and A/B tests may overlap and so conflict. There are also longer-term effects where some features are ‘cool’ for the first two weeks and the opposite effect where changing navigation may degrade performance temporarily. Amazon also finds that as its users evolve in their online experience the way they act online has changed. This means that Amazon has to constantly test and evolve its features.