Quantitative Research - Advantages and Disadvantages

Size Doesn't Always Matter. What Can You Do About Number Fetishism?

Bullseye on dart board
Courtesy Dan Tink, Photographer. © September 5, 2005 Stock.xchng

Market researchers often employ quantitative approaches in our work. It is important to understand the strength and limitations of any research approach. This is particularly true with regard to quantitative research techniques for two reasons: (1) Scientific and lay cultures are quite enamored by quantitative research techniques and tend not to look deeply at the design and mechanics of such procedures, and (2) It is inordinately easy to design a quantitative research effort badly.

A False Focus on Numbers

Its "cool" to be able to say that you are a quantitative researcher. "Quants", the affectionate term by which quantitative analysts are known, have been put on the great numbers in the sky pedestal. I believe this is more because complex mathematics and statistics are venerated simply because most of us don't understand much of the field. If something appears to have a social benefit and is at once difficult and mysterious it tends to take on a cultural "glow." The market is preoccupied with computer modeling and simulation. It so revered the black box of derivatives that it was slow to react when the models failed to predict the inevitable volatility.

Quite on the other hand, say aloud that you are a qualitative researcher and people are likely to give you a puzzled look. Most people know that ​quants are somehow engaged in stock selection and portfolio evaluation.

But what does a qualitative researcher do? Beyond being Margaret Mead, that is, what role is left to a qualitative researcher? Or so the conventional thinking might go.

A very old tenent of computer science is. Computer models are only as good as the content upon which they are built. The issue of reflexivity is never very far away.

George Soros has used the word reflexivity in conjunction with economics in general, and financial markets in particular. Heisenberg's uncertainty principle, the peer of reflexivity in the field of physics, is also relevant in this context. Heisenberg -- in a nutshell that does not do the principle justice -- argued that we cannot measure two attributes of a thing at once because, in our measuring, we impact the attributes or the thing and therefore bring about change or distortion of the original.

Consider George Soros's comment to the MIT Department of Economics World Economy in 1994.

"The generally accepted theory is that financial markets tend towards equilibrium, and on the whole, discount the future correctly. I operate using a different theory, according to which financial markets cannot possibly discount the future correctly because they do not merely discount the future; they help to shape it. In certain circumstances, financial markets can affect the so called fundamentals which they are supposed to reflect. When that happens, markets enter into a state of dynamic disequilibrium and behave quite differently from what would be considered normal by the theory of efficient markets."

Another more contemporary look at essentially the same phenomenon is described in the book The Black Swan by Nassim Nicholas Taleb. A black swan is not common in nature -- few people have seen a black swan. According to Taleb, a black swan is a positive or negative event that is considered to be highly improbable. But when a black swan does occur, it causes massive consequences. Some people believe that black swan events explain a great deal about the world. But most people -- particularly experts —- are blind to black swans.

A skeptical approach is essential to evidence-based science. There are some things to consider when exploring the concepts related to the number fetishism that blinds people to the pitfalls of accepting quantitative research at face value and being overly reliant on the normal distribution.

It is a mistake to believe that quantitative research based on inferential statistics is more credible or scientific than insight-based observational research. A truly important point in the comparison between quantitative research and qualitative research is that the subjective participation of the researcher -- that is one of the most resilient objections regarding qualitative research -- takes place in quantitative approaches. In fact, it occurs earlier in the empirical sequence of the research stream in quantitative research than it does in qualitative research.

The researcher generates a hypothesis in quantitative research that will be "tested" by the statistical processes. The generation of a hypothesis can be a very subjective activity. And the very narrow focus of hypothesis testing can be misleading. Many forms of qualitative research allow emerging patterns in the data to point to themes to which can attribute relationships (this is the equivalent of hypothesis testing in quantitative research). Qualitative research is more likely to be open to the "black swans" that occur, for which there is no hypothesis to be proved or disproved.