• The Ecstasy and Agony of Primitive Learning Analytics

    I’m awake and trying to be productive (for me) early in the day. I’m technically on a medical leave of absence but I’m not very good at doing nothing. I therefore promised to coordinate and edit the efforts of four moderators to produce a cohesive TT284 moderators’ report and I have some work ahead contributing my share to one for T320 too. This led to some musing about the primitive learning analytics I like to collect based on forum participation and the difficulties in obtaining them.

    Forum Statistics for OU Courses

    One thing I like to do is track forum usage statistics, a primitive form of learning analytics. Since we changed to Moodle from FirstClass, I don’t find this very easy. In FirstClass, not only could you do standard types of search on message data, but the read history of each message was also searchable. Combine that with a built-in way to restrict the search to specific conferences, sort the output by conference, user, or date, and group by conference or user, and you could determine all kinds of things. Some of my favourites were:

    • Total # of messages posted.
    • Total # of unique posters.
    • Total # of unique readers contrasted with enrolled students.
    • Percentage of posts that were moderators/course news versus students.
    • Top ten student posters and % of overall posts they contributed.
    • A breakdown of posting activity by logical parts and subparts, e.g. “Block 1″ overall but also “Block 1: Software Support” and “Block 1: Discussion”.

    The last one was useful to examine between different presentations when combined with knowledge of total number of students enrolled. It permitted me to see where students had the most problems and collect evidence if, when changes had been made for the following presentation, changes were having a positive effect. You could also see the trends in posting behaviour across cohorts.

    Getting at the Data

    In theory, some of this information is available in the Moodle logs. I just downloaded the log for one of my past courses I chaired and was surprised to note I could see “add reply” buried amongst the many “view forumng” entries. It’s downloadable as a CSV, so you’d have to roll your own data analysis tools to pull out the relevant bits. There are built-in statistics analysis facilities but they always seemed to be disabled on my courses, making download logs the only real option.

    The problem is access to those logs isn’t always available. As a course chair on Moodle 1.x, if the course was “editable”, then the admin tools were visible and the logs could be accessed. My last presentation (2012B, ending May 2012) somehow got into LTS’s update loop and the status/workflow changed back to needing to request access, so the admin links aren’t visible. I was able to hack the URL based on access to another course and get at it but that’s a bit of a pain.

    On my Moodle 2.x version course, I can see “Reports” but not a link to logs anywhere. I could edit the course site and back up the content, but perhaps I don’t have the permissions to access the logs. Certainly a typical moderator likely wouldn’t.

    What I Do in Moodle

    My approach generally in Moodle, regardless of the version, has therefore been very simplistic. I discovered that if I used Safari (but not Firefox) and copied the table listing the threads in a given forum and then pasted that into a spreadsheet, the HTML table’s columns were preserved. I could then have it sum the total number of messages per forum as one of the columns was number of thread posts. This isn’t very automated. I have to do it per forum and copy the totals into an appropriate place and most forums have multiple pages, each of which has to be handled separately.

    To Automate Or Not

    This is ripe for automation because certain actions are predictable, repeatable, and tedious. It’s the classic story though: do I spend the time trying to write something to automate it or just do it? Which will take less time? In the long run, if you do this yearly and across many courses, then automating it will save you time but there’s that up-front cost.

    A tool would also need to have a settings file, probably listing the module’s base URL and containing a list of the forum ID numbers/URLs and names. These are required because every presentation has a different ID and every forum has its own unique ID used to access it. Most modules don’t maintain a page that solely lists only the forums and the number/structure of those forums would vary between different modules. I suggested including names—or at least names I’d like to use to refer to them in reports—because otherwise you have to scrape that off the forum pages too and I’d find shorter ones more useful than the full, formal names.

    Another issue to contend with is authentication. I don’t already have code that can sign into the OU and maintain authentication for the session, although I know some people must. Before we had the “Dashboard”, one T320 AL wrote a tool to scrape metadata from the VLE and stored it in a local MySQL database. He then had an interface producing a dashboard for him that was something more than just a list of forums per course with an unread message indicator. I’ve recently heard, however, he gave up on his tool because VLE changes kept breaking it.


    Here I am writing about what I should be doing rather than doing it, but the process of thinking about it is always useful. Perhaps someone’s already done some of or all of this? My bet would be on Tony Hirst, but LTS colleagues may have some tools and I just don’t know about them.

  • OER and a Pedagogy of Abundance

    Martin Weller gave a 30-minute presentation last week for George Siemens’s CCK09 course on an idea he called “the pedagogy of abundance.” The key idea was that teaching in the past had been based on a scarcity model. I interpreted this as meaning knowledge was scarce (or closely guarded) and educators (the “talent”) were the scarce high priests on high–classic sage on the stage. He likened it to the music industry, which doesn’t strike me as too far off-base.

    Read the rest of this entry »

  • World of Warcraft and Me: A True Confession

    Elsheindra is Michelle's night elf druidAs part of a course under development at The Open University, I was approached as a known World of Warcraft player and asked to write a short paragraph or two on why I play World of Warcraft. I freely admit to failing to only write a short paragraph or two, but that’s probably because I’m passionate about World of Warcraft and my activities in it, especially given the prominence it plays in my life in so many areas. Read on to find out why I play World of Warcraft.

    Read the rest of this entry »

  • What Do I Know? A Reflection on Influences

    For the first time in years, I’m taking a postgraduate course myself: H812: Postgraduate Certificate in Academic Practice, which I’m doing both for personal development and to provide theoretical groundwork in educational pedagogy for my Ph.D. work in educational technology.

    A recent activity asked us to reflect on influences on our teaching practices, considering: practices arising from personal experiences as a student; practices from our departments; and practices we can attribute to other sources. In addition, we were asked to consider aspects of our workplace that favoured or hindered good practice. I starting making notes on the 14th of October. I did not post them to my group because I felt this was a really important activity. If you don’t know where you’ve come from, it can be difficult to move ahead in a purposeful fashion. I wanted this activity to serve as a good baseline, so I invested a substantial amount of effort into thinking about it and writing it up in a coherent, cohesive fashion.

    Read the rest of this entry »

  • More Marking Madness

    If I don’t finish my marking very soon, they’re going to kill me. The phone calls are increasing. With the ADD, though, I just have this overwhelming sense of guilt, failure, and frustration. More about that later. Something must be done.
    Anyway, I have a plan. My Sweetie is helping me do some of the grunt administrative work that’s required (filling out the forms, uploading some of the files to each project directory to help check the functionality, checking for missing/incorrect project directories, etc.). While Sweetie’s doing that, I’ll put in a big push to finish assessing the remaining reports (22). Then, to take a break, I’ll fill in the scanning sheets for all the report-related marks for each student, and then have a go for a bit at assessing the coding parts of the projects.
    The coding part is cognitively easier to assess for me as it’s definitely more black and white (it works/it doesn’t work; it’s written well/it’s written poorly). It still takes time to do, though, because you have to check through all the functionality for various points and write up the notes.
    With luck, proper use of my ADD medication, and SweetieSupport, I hope to get it all in tomorrow evening’s post. I’ll let you know how I make out.

  • Marking Madness and Motivation

    Project marking has to be one of the banes of my existence. Right now I’m working on grading an end of course project consisting of a coding component and a report. The coding component is fairly straightforward to do. Other than perpetual shock at the things people believe is good design/coding, it’s something I can do in a reasonable amount of time. The part I have trouble with is the report.
    Actually, any kind of marking where you need to subjectively weigh how close an answer is to what you want is difficult for me. Perhaps it’s because Ein’s have two states in many things in life: Ein/Auf, Happy/Sad, Tired/Bouncy. There’s not much room for shades of grey in the EinWorld.
    Anyway, that leaves me with a 53 projects to finish and I’m already two weeks late and having trouble mustering any enthusiasm for it. To be fair, I already finished doing 60 for another course which also had a coding component and a project, so I am feeling a little burnt out and I do have attention deficit disorder. Nevertheless, I promised I’d be done.
    As things stand, I finished 5 completely before deciding to switch to doing all the reports first. As the reports are independent of the coding component, that’s feasible. I picked the reports to do first because I like them the least and I’ll feel the most relieved when they’re done and the rest will be easy.
    Of the 48 reports to grade, I’ve done thirteen. Any motivation, inspiration, or encouragement welcome!

  • Conceptual Change

    David Jonassen visited the IDEAs lab on May 11th from the University of Missouri to present a talk on “Model-Building for Conceptual Change (Cognitive Tools in Action)”. While this isn’t (or so I thought) related to my own research or interests in any way, we were all encouraged to attend if possible and I’m always interested in talks about learning in general. Here, belatedly, is a synopsis of my understanding of his presentation.
    The key underlying principle seemed to emphasize having people fail in their problem solving attempt at some issue because then conceptual change has a change to be engaged and then students will learn. This failure need not be catastrophic; in fact, it probably should not be, I would say, or the failure would foster a strong sense of discouragement, which is not going to get a student into the “learning zone.” So, how do you put students into a non-threatening environment where they can safely experiment and fail? David Jonassen’s idea was to encourage them to engage in model building which demonstrates their conceptual understanding of the problem/issue at hand. When learners build models,their understanding of the problem domain is deepened because you cannot model what you do not understand. Model building also allows you, as the instructor, to view the learner’s level of conceptual change as their models evolve. It is therefore possible to assess their underlying understanding without resorting to formal assessment tests. Finally, David Jonassen suggested that model building also improves critical reasoning and thinking because model building forces the model builder to examine the process and problem solving methodology.
    David Jonassen researches (among other things) the use of technology in educational settings to improve understanding. More information on his approaches to problem solving are available from on the following web site page: http://tiger.coe.missouri.edu/~jonassen/PB.htm.

    Read the rest of this entry »