As James and I were chatting recently, it became clear that whilst we both call ourselves analysts, there are some important differences in what we do and how we spend our time. James therefore asked me to put a few thoughts together on the role of a quantitative research analyst, (or “numbers analyst” as he calls it) and how to become one.
So, to begin with, what do us numbers guys do?
At a simplistic level, we design and execute surveys, then analyse and interpret the results, which may then be used as input into business planning and development activity or as the basis for market analysis reports and advisory services. It sounds relatively straightforward and at a high level it is, but the devil is in the detail, so it’s worth talking through the process to get a feel for the kinds of things a numbers analyst needs to deal with.
Given a particular requirement or brief, for example, the first challenge is designing the study. Considerations here are sampling of target respondents, qualification of respondents for inclusion in the study, positioning of the interview, and method of execution (face to face versus telephone versus online).
All of these have an impact on the result. To use a an extreme case to illustrate the point, if the objective is to investigate a particular aspect of Linux and you use an online survey enticing people to participate by saying “Give us your views on Linux in the context of …”, Linux advocates will be much more likely to respond than sceptics or agnostics. This gives you a skewed sample which clearly means it is invalid to subsequently say things like “X% of companies think Linux is great for ….”. However, in a study of this nature, if you have asked the right questions, you would be able to make statements confidently and legitimately about those with experience of relevant use or differences in perceptions between users and non-users.
And this brings us on to the important area of questionnaire design. This is a massive topic and anyone on the receiving end of a poorly designed questionnaire executed by a pure market research company will understand this. In IT related research, questionnaires really need to be designed by industry analysts with a knowledge of the area being investigated, a feel for how the data will ultimately be analysed, and experience of “choreographing” the interview flow in a way that builds rapport and elicits unambiguous answers.
Execution is a topic I won’t go into here (i.e. the project management and operational side of actually gathering the data), but suffice it to say that this is pretty involved and unless it is handled well, the end result is likely to be compromised.
Finally, a numbers analyst will work with the data, analysing the obvious, but also looking for those trends and correlations that provide the real insights into what’s going on. Again, this can only be conducted effectively by an industry analyst with the necessary industry knowledge, otherwise important findings are missed and/or observations are misinterpreted – a common occurrence, for example, when PR companies engage generic market research firms and come out with ludicrous headlines for press releases.