Avoiding Analysis Paralysis

It is a moment met with many emotions. Excitement. Nervousness. Fear even. You have just received a final data set, now you will learn if the hours you spent discussing, crafting, programming and checking the survey instrument were worth it. For many researchers this can be a moment of analysis paralysis.

There is just so much to do, where do you even start? There is no one answer to this question, but there is a very good option, an annotated questionnaire. An annotated questionnaire is simply a questionnaire that has the survey responses written (or typed) in next to the question. An annotated questionnaire is a very quick, easy and clean way to look at the question along side the answers. Looking at this combined information can be very helpful as you begin to determine what story the data is trying to tell and what the report might look like.

There are some guidelines to keep in mind when creating an annotated questionnaire. First, keep the data as basic as possible. Total respondents is a good place to start. When you are posting the data to the questionnaire just include the answers to the questions. Resist the urge to combine or net any data, as this is not the place for that.

Second, keep note of changing base sizes. In the world of online research, there can be all sorts of programming logic in place such as skips, rotations, randomized exposes to stimuli etc. all of which can cause fluctuations in base sizes. If only a small percentage of your survey respondents answered a question then most likely the findings from that question should be given a similar weight in the story.

Third, try to complete the entire annotated questionnaire in one sitting. The simple exercise of reading the question and then posting the answers to that question naturally creates a good deal of insight that can be lost if you change focus.  Remember the whole reason for an annotated questionnaire is to start the analytical and reporting process so any early insight is important.

An annotated questionnaire will not work for every researcher. However, since the opportunity cost for trying one is relatively low, there is little reason not to try it for yourself. At the very least it is a chance to work on your penmanship ;-)

Happy annotating!