Saturday, January 22, 2011

On the dangers of observation - Nigel Hollis Milward Brown


Dan Cobley, marketing director, Google, believes there are similarities between physics and marketing (see his Ted video below which Martin Bishop has summarized on his blog, “Brand Mix”).

I love Dan’s use of Newton’s Law to explain why it is difficult to shift perceptions of big brands, but I think his conclusion based on Heisenberg's Uncertainty Principle is wrong. Relying on observation alone can be just as misleading as relying on questions.
According to Dan, Heisenberg's Uncertainty Principle states that it is impossible to measure the exact state of a particle because the act of measuring changes it.
Dan suggests the marketing equivalent is that the act of observing consumers changes the way they behave. He recommends that we try to measure what consumers actually do, rather than what they say they'll do.
I am reasonably sure that the Heisenberg lack of precision arises when trying to measure two related properties. The more accurately you measure one, the less accurately you can measure the other. Now that is true in marketing. Think of all the hoops statisticians put themselves through trying to deal with collinearity in a time series model.
I would argue there is little proof that observation, instead of asking questions, provides better understanding of why people do what they do. Why? Because our interpretation of people’s behavior is just as subjective and context-specific as our interpretation of their answers to questions.
What is more, it is a lot easier to take into account other people’s biases than our own. Behavioral data is great for measuring what happens, but it does not explain why.
Let me give you an example from the domain of astronomy of just how misleading observation can be.
For centuries, man believed that the sun revolved around the earth. It had to be true. The movement was visible to the naked eye. The sun rose in the East, and set in the West. The Ptolemaic astronomical system, which assumed the sun and planets revolved around the earth, was used to calculate the position of the planets with a fair degree of accuracy. But observation and the Ptolemaic system proved incorrect.
Nicolaus Copernicus’ De Revolutionibus Orbium Coelestium (On the Revolutions of the Celestial Spheres), detailed an astronomical system that placed the sun at the center of the universe. Published in 1543, the Copernican Model proved more predictive than its predecessor in a number of respects, although by no means perfect.
Later work by other scientists improved on the original, until it too was displaced.
Now there is a big difference between believing that the sun revolves around the earth, rather than the opposite. And yet the observational evidence supported the belief. The evidence is so compelling we still talk about the sun rising, even though science now confirms we are spinning around the sun while the solar system hurtles through space at the same time.
So rather than just dismissing one form of data collection in favor of another, perhaps we should instead embrace the need for multiple measurement systems. If multiple sets of attitudinal and observational data lead you to the same conclusion, then there is a far better chance that it’s right, than if only one of them does. But remember, even then your own understanding and biases will shape the conclusion you draw from the data.
Can you think of any other occasions on which observation alone might lead you to the wrong conclusion?

No comments: