Doing Qualitative Research: The Book

Chapter 2 of David Silverman’s Doing Qualitative Research:  A Practical Handbook (2010, p.16) asks students to consider why they believe a qualitative approach is appropriate for their possible research topics.  In fact, I had not initially considered a qualitative approach at all.  With my background in artificial intelligence, software engineering, and information retrieval, I was tending towards quantitative methodologies.  Information retrieval is very much about calculations and measurement, so that was a natural fit. Wikipedia (2010) describes the qualitative method as one that “investigates the why and how of decision making, not just what, where, when.”

Much of my survey data, like population demographics, is very amenable to quantitative methods to usefully describe the types of people and characters who participated in the first survey.  However, the core questions I was interested in were more what some people would call “touchy-feely” or how and why questions:

1. How do people describe the guilds they belong to.
2. What motivated people to play World of Warcraft initially.

While the first of those questions could be approached in a quantitative way by coding each 140-character response into one of a number of categories, I found that approach unsatisfying.  Even in such short responses, there was more nuance than I could easily accommodate in a simple, quantitative coding scheme.  For the second question, which I had not yet even attempted to analyze, I knew the number of game players saying the same thing was not the important part; the variety was important because I was interested in the underlying themes being expressed and, because I gave survey participants the space to write an essay, one or two categories was definitely not going to capture the detail.  Traditional quantitative analysis tools would not easily allow me to explore and group themes dynamically either, which is why I started investigating NVivo, a qualitative analysis tool.

So for this study, I am looking at mixed methods research.  I will be using quantitative analysis for the demographic details and qualitative analysis for analyzing the content of free-form responses.  The moral of the story, and one which David Silverman tries to get across right at the start, is that you need to choose your methods based on your data and what you want to discover.  Don’t be wed to a methodology just because it is familiar to you or even necessarily just because it has always been done that way.

Silverman, D. (2010) Doing Qualitative Research: A Practical Handbook, 3rd edition, London, United Kingdom, Sage Publications Ltd.

Wikipedia. (2010) Qualitative Research, [online] web page, Wikipedia. Available from: http://en.wikipedia.org/wiki/Qualitative_research (Accessed September 14, 2010).

Share/Save

Image: Elsheindra and Team Pink tackle the Dragonhawk Boss in Zul’Aman back in 2008. As a healer, Elsheindra has to make difficult decisions about who will live and who will die, in her role as main healer.  Being a researcher and maintaining anonymity is, I’ve discovered, a lot easier.

Back in April, I posted my first preliminary study to look at motivation, community formation, and learning in World of Warcraft.  When I was crafting my ethics approval for that study and future studies, I was very concerned with maintaining the privacy of the individuals participating.  The first survey was designed specifically to not require any personally identifiable information, although participants did have the option of giving an e-mail address if they wanted to participate in future studies or if they did not mind being contacted for any follow-up questions.

A problem arises, however, in following participants across multiple studies.  This is somewhat related to longitudinal studies where repeated observations are collected over long periods of time from the same participants.  The purpose of such studies is to help distinguish actual effects from short-term causes.  However, longitudinal studies aren’t the only time researchers may want to track participants across time and across multiple studies.  That would also be useful to help me build a more complex, detailed picture of participants, even though I intend to be asking different questions in different surveys.

While looking at other projects investigating World of Warcraft and motivation, I came across Nick Yee’s Daedalus Project, his old research project, and PlayOn, his new research project investigating social dimensions of virtual worlds.  I was quite surprised that, in at least one of his previous studies, he invited people to identify themselves by their e-mail addresses so that they could be tracked across his multiple studies.  Although I like Nick Yee’s work, I thought this approach was ethically incorrect.  The question is: how do you do it in a way that does not compromise the participants’ anonymity or their rights to privacy?

I got an answer to this recently from an unexpected place: the virtual common room of associate lecturers at The Open University where the topic was anonymous feedback from students being used potentially as a performance measurement mechanism.  Many of the lecturers felt that anonymous data collection wasn’t reliable.  Fellow IDEAs Lab alumna  Diane Brewster chipped in to say that a large quantity of research data is collected anonymously.  I got in touch with her via Twitter and she gave me the following tip: ask participants to identify themselves using a combination of specific letters from their month of birth and digits from their mobile telephone number.

Depending on the size of your participant pool, there might be some duplication.  However, if you choose your identifier tokens well, you can minimize that and still retain the desired anonymity.  Great tip, Diane.  Thanks a lot.  I will be putting this idea to use in my future survey work.

In the actual experiment, I gave Basil the following three essays:

Essay 1:
At first it was a way of keeping in touch with friends after I’d moved away. But I made more new friends throughthe online gaming community that occurs around the game. I’ve met a good number of my fellow guild members, including my guild leader and most of the other officers. To me, game has always been about exploring, storylines and the exotic locales presented therein. That’s all secondary to killing bosses, and trudging through raids really.

Essay 2:
I play WoW and other MMORPGS for the simple reason that I’m intrigued by the online community and game play aspects. WoW is my particular favourite that I return to again and again. I believe the reasoning behind this is the friendly community that has matured to quite a size over the number of years I’ve been playing. In addition to the community I find the story lines within the game interesting, challenging and sometimes, dare I say it, exciting. By exciting I mean, that like a good book, you want to see what is going to happen next!

Originally I started playing WoW for the simply reason it was an MMORPG. I was intrigued by the genre and WoW was really one of the first to be highlighted through the media, etc. As I progressed in the game, I discovered that it was a great way to relax after a busy day. As a form of escapism, it helped with relieving stress.

Now I rarely get to play WoW or any other MMORPG for that matter, however, for the same reasons of relaxation, online community, exciting stories, I still try to play as regularly as I can.

Essay 3:
Originally I moved to WoW simply because the majority of my guild had moved from DAoC, when WoW was released it was the next game that the existing guild members were collecting in. Ironically even though I followed my guild to the game I am actually motivated by the personal achievement.

I am the kind of player that likes to explore every location, complete every quest before moving on to the next zone and maximise trade skills. With each expansion, I spent most of the time solo’ing to the level cap, then exploring group content with my guild or raiding alliance.

With access to the raiding alliance I get to try challenging content which often require a level of skill and co-ordination. Currently I am motivated with the challenges of raiding with the aim to have completed as much as possible before the next content patch.

I know there is a sigma [sic] attached with gamers, but when you consider some people will return from work and just sit passively in front of a TV for 5hours. Similarly you see people sit all night on online chat channels. Given how some spend their time, how can spending your time problem solving and socialising with others with similar interests be so wrong.

Basil was asked to summarize the main ideas that occurred in each essay. Unfortunately, he was somewhat influenced by the question and noted down what people said their initial impetus for playing World of Warcraft was and then why they continue to play. I had to send him off to do it again. Table 1 illustrates our responses.

When I looked at essay 1, there was a question about things being “secondary to killing bosses , and trudging through raids…” Secondary implies that the other things were of lesser importance, but the negative tone implicit with words like “trudging” would seem to bely that, so I didn’t include the raiding. In talking to Basil, I know he had the same problem, because he asked me about it and I told him I would not give him an answer. As a result, he included raiding, whereas I did not.

On the whole, we don’t seem that different. If we had gone through the essays in advance together and agreed on some themes, I suspect the coding would have been similar. What do you think?

The One and Friends at the start of Razorgore encounter in Blackwing Lair in March of 2010

I’m now in week two of my first survey into learning, communities of practice, and World of Warcraft in support of my doctorate. The first part of the survey is collecting some in-game demographic details, such as how long people have played, what their first character was, favourite professions, etc. The meat of the survey, however, is a short answer question about why people play World of Warcraft.

I am still looking for additional responses and the survey is scheduled to close at the end of this weekend (April 18th). If you play World of Warcraft or know other players, I would be grateful if you could encourage them to visit the survey information page and participate. On completion, participants will be given a code to enter a draw for three Blizzard store pets as prizes.

Tweet, ask in guild, ask at university, or poke your workmates to pass the word along. Just remember that it’s only open to those 18 years or older as I’m interested in examining adults and learning and there are restrictions on participation in things by those younger than 18.

Thanks!

• Survey 1: Why Do You Play World of Warcraft