There are many steps in the process of generating research findings through to delivering those findings to management.  Along the way there is ample scope for disconnects, but perhaps none as great as the step from quantitative analysis to findings.  In many instances this is not so much of a step as a leap of faith.  One reason for the 'leap' is an absence of an understanding of the underlying assumptions used in the analysis.

Those of us on the quantitative side sometimes enjoy highlighting the soft edges of qualitative research however, in quantitative analysis; assumptions are often just as many and varied.  Broadly classifying them, there are those that have a statistical basis and those that are conceptual.  In how many instances does the person presenting the quantitative findings understand the assumptions of the techniques that were used?  For example, are the assumptions conceptual as for conjoint analysis or statistical as with say, multiple regression analysis?

Highlighting assumptions to management is not always necessary.  At one level, we want the people presenting the quantitative findings to understand if the assumptions have been materially breached and yet, at the same time, we do not want management to be unnecessarily bogged down in hearing about assumptions that have not been breached.

I am all for encouraging managers to have confidence in the research findings and being aware that too much unnecessary emphasis on the assumptions may result in ambiguity in their understanding of the findings.  However, equally I am not in favour of seeing research consultants presenting findings to management in the absence of a reasonable appreciation of the assumptions underlying the data analysis.

The separation in roles between those who undertake the analysis and those who present the findings to the management, has the potential to be the source of this ambiguity.  This is exacerbated when this separation falls between those who originally designed the software and those who simply ‘check the boxes’.  In other words, I suspect there is a good bit of analysis being undertaken in the research world by people who do not have an adequate appreciation of the quantitative assumptions associated with the tool being used.  Good practice demands that the person undertaking the analysis communicates any material breach of the assumptions to the person presenting the findings.

In my experience, the greatest scope for difficulty arises in techniques such as conjoint analysis where we are attempting to optimise a design or determine price elasticity.   In a rush to provide management with ‘the answer’ the assumptions are often not well communicated.

Crudely, conjoint analysis enables management to understand, based on the assumptions, how consumers make their choice.  But getting to that point is littered with conceptual assumptions.  Not knowing those assumptions is risky for both the user and the reporter.

Some typical conceptual assumptions include ceteris paribus, that is, the market will remain unchanged and therefore the simulation continues to approximate market conditions.  Another is that the consumer has the same perfect information that the respondent had when undertaking the conjoint task.  Also, that the attributes tested are actually the ones that the consumers’ decision is based on.  Another is that the level of one attribute does not change the utility of another.  For example, this assumption is breached in instances where the high-involvement product is more sophisticated than the buyer and therefore price attributes are used by the respondent to infer higher quality with higher prices.

If you are a consultant presenting the findings insist that the analysts list the assumptions and if they consider any material breaches have occurred.  If you are an analyst, insist in participating in proposal writing when it is most appropriate to raise the various assumptions.