Posted by 
1 comment
Credit: Photo by Marsha Brockman (whodeenee) under an Attribution-NonCommercial-NoDerivs 2.0 Generic license
Image: Marbles, many marbles. I think I have lost mine in a sample of many marbles.
I’ve been re-running analyses today on my population of survey responses. I decided to remove some more responses to eliminate some the scatteredness in the population. The majority of responses were from European PvE (player versus the environment) realm players, so I removed the four American realm players and then the five non-PvE players, leaving me with a sample of 30.
The more I read about sampling, the more confused I am.
When we read the Oliver and Carr (2009) WoW communities of practice and learning paper the other day, we were somewhat dismissive because it only had five couples. However, the paper also mentioned that it used “theoretical sampling”, which we had not heard of. Someone looked it up quickly on Wikipedia and it sounded like you chose your sample based on it generating the features you wanted to look at. Now the description in the Oliver and Carr paper sounds more like “purposive sampling”, because they described their sampling in a way that seemed to fit with Cohen et al’s description of “…qualitative researchers handpick the cases to be included in the sample on the basis of their judgement of their typicality or possession of the particular characteristics being sought” (2007, p. 114-115):
Players were recruited through online guilds and real-world social networks. The first two sets of participants were sampled for convenience (two heterosexual couples); the rest were invited to participate in order to broaden this sample (one couple was chosen because they shared a single account, one where a partner had chosen to stop playing and one mother–son pairing).
Oliver and Carr (2009, p. 446).
I was browsing through Research Methods in Education today and it specifically mentions theoretical sampling as a feature of grounded theory and the sample size is immaterial. The important part is that you have enough data to saturate the categories in your theory. You collect more and more data until the acquisition of more data does not advance or modify the theory developed. It suggests that the size of the data set may be fixed by the number of people to whom one has access but you have to consider that it may be necessary to seek further data (Cohen et al. 2007, 116-117). A sample of five couples would then possibly be acceptable. Although I am taking a grounded theory approach, this does not feel quite like what I am doing, although I do have the intention of generating the theory from the data I have and then pursuing a larger-scale study later.
Another possibility is volunteer sampling. This is apparently different than convenience sampling. I suppose in a convenience sample, you have more control over how many people respond, e.g. a class of students, and you are directly asking them. In volunteer sampling, you rely on volunteers, like personal friends or friends of friends, although it can also be via, for example, a newspaper advertisement (Cohen et al. 2007, p. 116). This sounds similar to the approach that I took. I already knew I had to be careful about making generalizations and certainly the representativeness of the sample is lacking. This is probably acceptable, provided the lack of typicality is made clear.
Real World Research describes a convenience sample as one of the most widely used (Robson 2002, p. 265). Sensible uses of convenience samples, Robson suggests, are for piloting a proper sample survey or getting a feeling for the issues involved. This too feels like what I was doing, since I designed the study originally to be the starting point for a future, larger study. Providing a springboard for future research is also described as being acceptable by Bryman (2008, p.183) in Social Research Methods.
My section describing the survey distribution currently reads as follows:
A blog site was created for the overall project and readers invited to participate (Hoyle 2010) through an initial posting. Readers were given a brief explanation of the survey’s purpose, contact details for the author, and an explanation of the rules and time and effort expected. The page explained that there would be an opportunity to enter an optional draw to win a virtual in-game pet as a reward. This page also contained a link to the survey, hosted on SurveyMonkey, a third-party commercial web survey site.
At a minimum, 25 to 30 participants fully completing the survey were required and more than 50 to 75 would be burdensome. Advertising was therefore not ambitious or comprehensive. Short messages were broadcast periodically on a European (player versus environment) game realm to a text communication channel shared by members of five allied guilds. A month before the survey, allied guild leaders were questioned about their current membership numbers. This information is available in the game and reflects the number of individual accounts that belong to a given guild. Total number of player accounts was 437. That count includes inactive players and players belonging to more than one allied guild. It is also possible for players to have more than one account, if they are willing to pay for it, resulting in the same person being counted more than once. However, after discussion with the guild leaders, the number of people with multiple accounts or multi-guild membership was believed to be small; the number of people reported is therefore probably fairly close. However, it is difficult to estimate what proportion would be active players or would have seen the periodic messages.
In addition to the in-game messages, the study was also advertised numerous times via the author’s main Twitter account and an account dedicated to news for the allied guilds. This resulted in a number of rebroadcasts as other researchers and followers tried to assist by passing along the message. Twitter messages, by their nature limited to 140 characters, were very brief, basically a tease along with the survey blog posting URL containing more information and the actual survey link. Finally, there was some promotion and requests for participation on guild forums belonging to the allied guild members, but not on the official Blizzard World of Warcraft forums, Elitist Jerks, Joystiq, or other large WoW community forums. Most participants would therefore be recruited from a community of people who knew of the author. This was intentional to benefit from social capital gained already by being a guild leader and co-leader of the allied guild group, especially as participants were expected to engage in a non-trivial task.
The study was designed as the first of a series investigating factors contributing to players persisting in learning and working in massively multiple online games, like World of Warcraft. Solicitation for participation was deliberately low-key to make the analysis of discursive responses manageable. Themes derived from the discursive responses could then be used to design a larger scale survey in the future. In this study, I particularly wanted to start collecting data on the following six research questions from a combination of qualitative and quantitative questions:
- What motivates people to play World of Warcraft?
- What motivates people to persist in playing?
- Is there a relationship between gender and stated motivations?
- Is there a relationship between age and stated motivations?
- Is there a relationship between nationality and stated motivations?
- Is there a relationship between character roles and classes and motivation?
In keeping with the overarching theme of learning, I hoped to see some evidence of learning behaviour or practices, prompting the most important research question:
- What, if anything, are people learning in World of Warcraft?
The question therefore remains: convenience sample, volunteer sample, theoretical sample, or a mixture? I originally thought it was a convenience sample, but now I do not feel confident in that at all. Oliver and Carr describe two of the couples in their theoretical sample as being convenience samples. Are mixtures “acceptable”? I am leaning now strongly towards labelling it a volunteer sample. What have I done? Help!
Sincerely,
Confused in London
References
Bryman, A. (2008) Social Research Methods. 3rd edition. Oxford, United Kingdom, Oxford University Press.
Cohen, L., Manion, L. & Morrison, K. (2007) ‘Chapter 4: Sampling’, in Research Methods in Education, 6th edition. Milton Park, United Kingdom, Routledge UK.
Hoyle, M.A. (2010) WoW Learning: A Study of Learning in World of Warcraft by Michelle A. Hoyle, [online]. (Accessed June 24, 2010).
Oliver, M. & Carr, D. (2009) ‘Learning in Virtual Worlds: Using Communities of Practice to Explain How People Learn From Play’, British Journal of Educational Technology, 40 (3), pp:444-457. Also available from: http://doi.wiley.com/10.1111/j.1467-8535.2009.00948.x (Accessed June 14, 2011).
Robson, C. (2002) Real World Research: A Resource for Social Scientists and Practitioners-Researchers. 2nd edition. Oxford, United Kingdom, Blackwell Publishing.
lizit 
Posts Feed
