Image: Marbles, many marbles. I think I have lost mine in a sample of many marbles.

I’ve been re-running analyses today on my population of survey responses. I decided to remove some more responses to eliminate some the scatteredness in the population. The majority of responses were from European PvE (player versus the environment) realm players, so I removed the four American realm players and then the five non-PvE players, leaving me with a sample of 30.

The more I read about sampling, the more confused I am.

When we read the Oliver and Carr (2009) WoW communities of practice and learning paper the other day, we were somewhat dismissive because it only had five couples. However, the paper also mentioned that it used “theoretical sampling”, which we had not heard of. Someone looked it up quickly on Wikipedia and it sounded like you chose your sample based on it generating the features you wanted to look at. Now the description in the Oliver and Carr paper sounds more like “purposive sampling”, because they described their sampling in a way that seemed to fit with Cohen et al’s description of “…qualitative researchers handpick the cases to be included in the sample on the basis of their judgement of their typicality or possession of the particular characteristics being sought” (2007, p. 114-115):

Players were recruited through online guilds and real-world social networks. The first two sets of participants were sampled for convenience (two heterosexual couples); the rest were invited to participate in order to broaden this sample (one couple was chosen because they shared a single account, one where a partner had chosen to stop playing and one mother–son pairing).
Oliver and Carr (2009, p. 446).

I was browsing through Research Methods in Education today and it specifically mentions theoretical sampling as a feature of grounded theory and the sample size is immaterial. The important part is that you have enough data to saturate the categories in your theory. You collect more and more data until the acquisition of more data does not advance or modify the theory developed. It suggests that the size of the data set may be fixed by the number of people to whom one has access but you have to consider that it may be necessary to seek further data (Cohen et al. 2007, 116-117). A sample of five couples would then possibly be acceptable. Although I am taking a grounded theory approach, this does not feel quite like what I am doing, although I do have the intention of generating the theory from the data I have and then pursuing a larger-scale study later.

Another possibility is volunteer sampling. This is apparently different than convenience sampling. I suppose in a convenience sample, you have more control over how many people respond, e.g. a class of students, and you are directly asking them. In volunteer sampling, you rely on volunteers, like personal friends or friends of friends, although it can also be via, for example, a newspaper advertisement (Cohen et al. 2007, p. 116). This sounds similar to the approach that I took. I already knew I had to be careful about making generalizations and certainly the representativeness of the sample is lacking. This is probably acceptable, provided the lack of typicality is made clear.

Real World Research describes a convenience sample as one of the most widely used (Robson 2002, p. 265). Sensible uses of convenience samples, Robson suggests, are for piloting a proper sample survey or getting a feeling for the issues involved. This too feels like what I was doing, since I designed the study originally to be the starting point for a future, larger study. Providing a springboard for future research is also described as being acceptable by Bryman (2008, p.183) in Social Research Methods.

My section describing the survey distribution currently reads as follows:

A blog site was created for the overall project and readers invited to participate (Hoyle 2010) through an initial posting. Readers were given a brief explanation of the survey’s purpose, contact details for the author, and an explanation of the rules and time and effort expected. The page explained that there would be an opportunity to enter an optional draw to win a virtual in-game pet as a reward. This page also contained a link to the survey, hosted on SurveyMonkey, a third-party commercial web survey site.

At a minimum, 25 to 30 participants fully completing the survey were required and more than 50 to 75 would be burdensome. Advertising was therefore not  ambitious or comprehensive. Short messages were broadcast periodically on a European (player versus environment) game realm to a text communication channel shared by members of five allied guilds. A month before the survey, allied guild leaders were questioned about their current membership numbers. This information is available in the game and reflects the number of individual accounts that belong to a given guild. Total number of player accounts was 437. That count includes inactive players and players belonging to more than one allied guild. It is also possible for players to have more than one account, if they are willing to pay for it, resulting in the same person being counted more than once. However, after discussion with the guild leaders, the number of people with multiple accounts or multi-guild membership was believed to be small; the number of people reported is therefore probably fairly close. However, it is difficult to estimate what proportion would be active players or would have seen the periodic messages.

In addition to the in-game messages, the study was also advertised numerous times via the author’s main Twitter account and an account dedicated to news for the allied guilds. This resulted in a number of rebroadcasts as other researchers and followers tried to assist by passing along the message. Twitter messages, by their nature limited to 140 characters, were very brief, basically a tease along with the survey blog posting URL containing more information and the actual survey link. Finally, there was some promotion and requests for participation on guild forums belonging to the allied guild members, but not on the official Blizzard World of Warcraft forums, Elitist Jerks, Joystiq, or other large WoW community forums. Most participants would therefore be recruited from a community of people who knew of the author. This was intentional to benefit from social capital gained already by being a guild leader and co-leader of the allied guild group, especially as participants were expected to engage in a non-trivial task.

The study was designed as the first of a series investigating factors contributing to players persisting in learning and working in massively multiple online games, like World of Warcraft. Solicitation for participation was deliberately low-key to make the analysis of discursive responses manageable. Themes derived from the discursive responses could then be used to design a larger scale survey in the future. In this study, I particularly wanted to start collecting data on the following six research questions from a combination of qualitative and quantitative questions:

1. What motivates people to play World of Warcraft?
2. What motivates people to persist in playing?
3. Is there a relationship between gender and stated motivations?
4. Is there a relationship between age and stated motivations?
5. Is there a relationship between nationality and stated motivations?
6. Is there a relationship between character roles and classes and motivation?

In keeping with the overarching theme of learning, I hoped to see some evidence of learning behaviour or practices, prompting the most important research question:

7. What, if anything, are people learning in World of Warcraft?

The question therefore remains: convenience sample, volunteer sample, theoretical sample, or a mixture? I originally thought it was a convenience sample, but now I do not feel confident in that at all. Oliver and Carr describe two of the couples in their theoretical sample as being convenience samples. Are mixtures “acceptable”? I am leaning now strongly towards labelling it a volunteer sample. What have I done? Help!

Sincerely,
Confused in London

References

Bryman, A. (2008) Social Research Methods. 3rd edition. Oxford, United Kingdom, Oxford University Press.

Cohen, L., Manion, L. & Morrison, K. (2007) ‘Chapter 4: Sampling’, in Research Methods in Education, 6th edition. Milton Park, United Kingdom, Routledge UK.

Hoyle, M.A. (2010) WoW Learning: A Study of Learning in World of Warcraft by Michelle A. Hoyle, [online]. (Accessed June 24, 2010).

Oliver, M. & Carr, D. (2009) ‘Learning in Virtual Worlds: Using Communities of Practice to Explain How People Learn From Play’, British Journal of Educational Technology, 40 (3), pp:444-457. Also available from: http://doi.wiley.com/10.1111/j.1467-8535.2009.00948.x (Accessed June 14, 2011).

Robson, C. (2002) Real World Research: A Resource for Social Scientists and Practitioners-Researchers. 2nd edition. Oxford, United Kingdom, Blackwell Publishing.

Image: I can just imagine this Blood Furnace boss exhorting people “L2P!” as he kills them over and over.

It is day two of the writing regime. Today’s plan is writing 750 words, writing CMA feedback, and working on the paper. I was musing last night about the approach to the paper, thinking that having an abstract or an introduction actually makes it easier to write because it provides a focus for the paper’s direction. I have heard other people say that it makes sense to leave the introduction to the last because then you know what you’ve written. I think the former approach might be more sensible for me. I can always go back and revise the introduction if it does not reflect what I end up doing. Focus, however, is priceless.

In addition to an introduction or an abstract, a title might also help. I was experimenting with variants of “L2P! Learn To Play Or…”. I thought that was clever, as it’s something you often see more experienced, impatient players saying to players who they think are not living up to their expectations in terms of expertise or speed. In the context of my work, however, it probably makes more sense to say “P2L! Play To Learn”, but I’m not sure how many people will get that. Nevertheless, a title is a starting point. I had both before I started my keynote writing and that turned out well. Perhaps I can incorporate the factoid into the abstract.

Abstract:

“L2P! L2P!” This is the exhortation you might encounter in massively multiplayer online games (MMOGs) when other players around you believe your skill or speed in playing is inadequate. It means “learn to play”. In this paper, we demonstrate how L2P has been turned on its axis to yoke the trials of play to the game of learning. 39 World of Warcraft players primarily playing in Europe submitted essays answering the question “Why do you play World of Warcraft?” in a 2010 study.

Using a grounded theory approach and discourse analysis, the essays were analyzed to ascertain the contributors’ motivations for playing and their reasons for persisting in playing. Yee’s player motivational framework subcomponents (Yee 2005; Yee 2006) were applied to each essay and contrasted with Bartle’s original player typology (Bartle 1996; Bartle 2003) in aggregate to determine overall, general motives these players had. While participants were not asked to write explicitly about learning and many did not provide any examples, several contributions are examined here as case studies of mundane and unusual examples, illustrating what these adults are playing to learn–a learning that goes beyond dungeons, dragons, and dwarves.

That does not seem too bad as a first go. I need to check on the discourse analysis; it might not be completely true. I also have no idea how I am going to write up the grounded theory bit appropriately, but at least that is accurate. I definitely followed that kind of approach in tagging the essays. I need to find some time to pore through the James Paul Gee’s book on discourse analysis. I just saw someone else in #phdchat mention it again yesterday. It keeps cropping up and I keep not reading it, even after I went to buy it and then realized I already had. That is trying to tell me something, if I would only listen. I also need to check on what to call Yee’s framework.

References:

Bartle, R. (1996) ‘Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs’, Journal of MUD Research, 1 (1). Also available from: http://www.mud.co.uk/richard/hcds.htm (Accessed April 22, 2011).

Bartle, R. (2003) Designing Virtual Worlds. New Riders Publishing.

Yee, N. (2005) A Model of Player Motivations, [online] Daedalus Project. Available from: http://www.nickyee.com/daedalus/archives/001298.php?page=1 (Accessed March 31, 2011).

Yee, N. (2006) ‘Motivations for Play in Online Games’, CyberPsychology & Behavior, 9 (6), pp:772-775. Also available from: http://www.liebertonline.com/doi/abs/10.1089/cpb.2006.9.772 (Accessed March 31, 2011).

Doing Qualitative Research: The Book

I was reading Chapter 3 of David Silverberg’s Doing Qualitative Research: A Practical Handbook (Silverman, 2010 p.17-42) in September.  In it he gives three research diaries of Ph.D. students he had, detailing how they went from the start of their research projects, through methodology choice, and then through to data analysis.  While it was quite striking how coherent and “painless” the stories were, the more relevant realization I took away from it was the importance of having a framework around which to direct your research and to make sense of your data.  In two of the cases, the students used conversational analysis, a ethnomethodological approach.

Conversation analysis (commonly abbreviated as CA) is the study of talk in interaction (both verbal and non-verbal in situations of everyday life). CA generally attempts to describe the orderliness, structure and sequential patterns of interaction, whether institutional (in school, a doctor’s surgery, court or elsewhere) or in casual conversation.
Wikipedia (2010a)

In my case, conversational analysis wouldn’t be appropriate, because it requires audio or video data of conversations and then a detailed transcription is made.  The inductive data-driven analysis part to explain discovered patterns is a process I want to emulate, so I need a similar approach.  That reminded me that some of James Paul Gee’s work used discourse analysis (or what I think is discourse analysis) and that might be an appropriate framework for me to use to structure my own research.

Unfortunately, I think standard discourse analysis isn’t quite correct.  According to Wikipedia (2010b):

Discourse analysis (DA), or discourse studies, is a general term for a number of approaches to analyzing written, spoken, signed language use or any significant semiotic event.

The objects of discourse analysis—discourse, writing, talk, conversation, communicative event, etc.—are variously defined in terms of coherent sequences of sentences, propositions, speech acts or turns-at-talk. Contrary to much of traditional linguistics, discourse analysts not only study language use ‘beyond the sentence boundary’, but also prefer to analyze ‘naturally occurring’ language use, and not invented examples. This is known as corpus linguistics; text linguistics is related.

James Paul Gee’s use of discourse in social linguistics differentiates between “Discourse” (big Discourse) and “discourse” (little discourse).  Discourse (big) is a combination of the language and its contextual environment, such as behaviour, values, perspectives, etc., so it’s a form of situated language analysis.  He also looks at Discourse communities as part of that context.  A seminal work in discourse analysis is Gee’s An Introduction to Discourse Analysis: Theory and Method (1999).  I wonder if I should acquire this?  I note a second edition was released in 2005.  I’ve put in a request to the OU library to have their copy recalled and mailed to me.

References

Silverman, D. (2010) Doing Qualitative Research: A Practical Handbook, 3rd edition, London, United Kingdom, Sage Publications Ltd.

Wikipedia. (2010a) Conversation Analysis, [online] web page, Wikipedia. Available from: http://en.wikipedia.org/wiki/Conversation_analysis (Accessed September 15, 2010).

Wikipedia. (2010b) Discourse Analysis, [online] web page, Wikipedia. Available from: http://en.wikipedia.org/wiki/Discourse (Accessed September 15, 2010).

Share/Save

Hermeneutics (English pronunciation: /hɜrməˈnjuːtɨks/) is the study of interpretation theory, and can be either the art of interpretation, or the theory and practice of interpretation. Traditional hermeneutics — which includes Biblical hermeneutics — refers to the study of the interpretation of written texts, especially texts in the areas of literature, religion and law. Contemporary, or modern, hermeneutics encompasses not only issues involving the written text, but everything in the interpretative process. This includes verbal and nonverbal forms of communication as well as prior aspects that affect communication, such as presuppositions, preunderstandings, the meaning and philosophy of language, and semiotics.[1] Philosophical hermeneutics refers primarily to Hans-Georg Gadamer’s theory of knowledge as developed in Truth and Method, and sometimes to Paul Ricoeur.[2] Hermeneutic consistency refers to analysis of texts for coherent explanation. A hermeneutic (singular) refers to one particular method or strand of interpretation.
Wikipedia (2010)

It’s apparently related to computational semiotics or used in computational semiotics.  That reminds me of James Paul Gee again because he talks about the semiotics of things in his What Video Games Have To Teach Us about Learning and Literacy (2007).  Is it another sign that I need to be looking at Gee’s book on discourse analysis (Gee 2011)?

References

Gee, J.P. (2007) What Video Games Have To Teach Us About Learning and Literacy, 2nd edition, New York, NY, United States, Palgrave Macmillan.

Gee, J.P. (2011) An Introduction to Discourse Analysis Theory and Method, 3rd edition, Abingdon, United Kingdom, Routledge.

Silverman, D. (2010) Doing Qualitative Research: A Practical Handbook, 3rd edition, London, United Kingdom, Sage Publications Ltd.

Wikipedia. (2010) Hermeneutics, [online] web page, Wikipedia. Available from: http://en.wikipedia.org/wiki/Hermeneutics (Accessed September 21, 2010).

Tony Hirst (@psychemedia) built a Google custom search engine that scraped the profiles of Twitter users employing the #altc2010 hashtag for website addresses.  For a laugh, I typed in “World of Warcraft”, not expecting much to show up other than myself.  Well, I was there, but so was mention of a poster and a talk entitled “Cultural Capital and Community Development in the Pursuit of Dragon Slaying (Massively Multiplayer Guild Culture as a Model for e-L:earning)” at the 2007 Alt-C conference by David White.  That pointed me to an Alt-C talk and a GLS one in 2007.  So, not long before I started my Ph.D., David was already out there talking about this.  Ouch!  The “ouch” part is that I met him earlier this year at a gaming-related discussion panel.  He was chairing my table, but  we were discussing  digital residents and visitors.  David follows me on Twitter too!  World of Warcraft has never come up.

The abstract mentions guilds, World of Warcraft, social capital, and communities of practice.  His description is eerily similar to my current focus.  Unfortunately, I couldn’t find a matching paper for the talk.  There’s just the GLS 2007 26-minute talk embedded in the blog pos from Tall Blog.  I’d best add this to my list of things to investigate soon.  It sounds very, very relevant.  Perhaps he has something I can build on or I will obtain some ideas on how to differentiate my work.  I am also interested in seeing his ethnographic approach and what he discovered.  This is a challenge, not a disaster.  There is always something different you can do.  You just need to find it.

The requirements to complete the “Insane in the Membrane” achievement.
Image from WoWWiki (2009)

I want to say something about the kinds of motivations people have for playing World of Warcraft. Specifically, I want to enumerate factors that motivate players to persist in the game even when it involves tasks that are repetitive, boring, or seemingly impossibly long.

For example, there’s an achievement in World of Warcraft called “Insane in the Membrane” that gives the completer a reward of an in-game title of “The Insane.” This achievement requires you to raise your reputation points with different game factions to exalted, the highest level. Generally, you need about 21,000 points to reach exalted. Points are gained by completing quests, collecting and turning in items, or sometimes killing certain types of things. If you only had to gain exalted reputation with one or two factions, this would not be difficult. However, you need to do this with eight different factions, most of which are not factions you would be accruing large amounts of reputation with during the normal course of play.

To increase the difficulty, several of the factions involved have rival factions. With those factions, as you gain reputation with one, you lose reputation points with the rival faction, making the process of completing this achievement complex in addition to time-consuming. The WoWWiki (2009) page describes some strategies for completing this achievement and the complexities of the faction-rival relationships.

Most tasks players undertake are not going to be as complex, time-consuming, or mind-numbing to complete as the aptly-named “Insane in the Membrane”. There are, however, many smaller day-to-day activities necessary for successful raiding or to get some particular piece of gear, such as doing daily quests to earn gold, or harvesting materials for potions or enchantments, or completing instance and after instance to get badge rewards or reputation rewards. I’m making it sound like getting achievements or gear is the be-all, end-all, but I think the situation is more complex than that. It’s that hypothesis I want to verify.

Other things I would like to be able to comment on include the relationships between gender and motivation, or motivation and age, or possibly even motivation and nationality. I do not necessarily believe there will be a relationship between motivation and nationality necessarily, but how can you definitively say if you do not look for the correlation? That gives me the following questions I want answered:

1. What motivates people to play World of Warcraft?
2. What motivates people to persist in very boring or difficult tasks?
3. Is there a relationship between gender and stated motivations? If so, what is it?
4. Is there a relationship between age and stated motivations? If so, what is it?
5. Is there a relationship between nationality and stated motivations? If so, what is it?
6. Is there a relationship between character roles and classes and motivation?

With those six questions in mind and the original study idea of determining motivation via analysis of free-form essays about motivation, I can now go ahead and develop the specific survey questions that will help elicit data to answer those questions.

Going back to considering my approach-whether I should start with planning versus start with survey question-it was not as clearcut as I expected. By starting with some potential survey questions and then thinking about the answers I would get from them, I gained a better idea about what answers I wanted, a kind of iterative development process. Sometimes putting the horse first helps you know where and how to put the cart!

References:

WoWWiki. (2009) Insane in the Membrane, [online] WoWWiki. Available from http://www.wowwiki.com/Insane_in_The_Membrane (Accessed November 6, 2009).