This is the final post of four capturing my paper presented at the ALT Conference, 3-5 September 2019, Edinburgh (abstract, annotated slides, video recording). This section looks at the contradictions prevalent in designing MOOCs (massive open online courses) and expands upon the presentation with exploration of personalised learning.
Previous posts explored: professional learning perspectives, contradictions in open online learning, and over-reliance on data to infer learning. In this post I explore the idea of learning rhythms and how courses are designed (or not) around patterns of behaviour, and how those designs influence learners.
Learning cannot exist solely online
Learning takes place across spaces, as such online learning is always ‘blended’. An online course will involve an individual sitting at a computer, or perhaps with their mobile phone in their hand. They may be at home, or at work, or travelling. The learning environment may be situated within their professional practice, or in their personal space. The key point though is that data provided through online learning platforms doesn’t capture the full ‘blend’ of learning taking place.
Where learning is taking place is important to consider, particularly in the scope of professional development, as the interplay between learning activities and the learner’s context, where new ideas may be implemented and reflected upon, is at the heart of many professional development learning objectives. Rarely are online courses created in the professional development sector simply for learners to ‘know’ something new. It is the action of putting something into practice that is significant.
“An over reliance on log file analyses and click stream data to understand learning leaves many learner activities and experiences invisible to researchers… participants reported that their participation in MOOCs varies according to the daily realities of their life and the context of the course. Learners’descriptions of how these courses fit into their lives are a powerful reminder of the agency of each individual.”
Veletsianos, et al., (2015:582-583)
The data cannot capture when and how practice is implemented, at least not through measuring clicks in the online space. Neither can it capture the offline conversations and how learners have worked with colleagues to further their understanding. Platform data must be complemented with, or even viewed through, a lens provided by the learning outcomes of a course.
Outcomes, not clicks
Whilst learning outcomes are integral to the learning design, personal learning outcomes may vary from what has been prescribed. The transference of professional development activities into professional contexts will give rise to learning outcomes that may be unanticipated, outcomes that have not been designed for, and which, perhaps designed learning activities do not support. The learning outcomes from any course may therefore be influenced not just by the activity, but the context of the learner as much as anything else. Data from online engagement with a learning platform may be representative of designed learning outcomes, but cannot reliably describe the individualised professional and personal achievements of learners.
Using data alone to infer learning outcomes and effectiveness of learning design can therefore be fraught with complexity. However, the data does show patterns of engagement with courses and these in turn may be used to infer learning behaviours, if not outcomes. Reviewing these patterns may in turn suggest ways the designed course meets, or does not, the professional development needs of individuals.
Linking learning and practice needs scaffolding
Being able to transfer learning from a designed activity into a practice context requires a level of translation on the part of the learner. It is already well established that learners on open online courses tend to have a high degree of learning skill (Conole, 2015). Yet, it can be argued that this learning skill is developed within traditional learning situations, rather than self-paced and self-directed as is the case with open online courses. Learners should be supported to learn and develop using open online courses.
The aspects that Conole (2015) explain underpin an effective learning design for open online courses, could form the foundation of a learner support. For example Conole proposes a course vision document containing:
The pedagogical approaches used.
The core principles of the MOOC.
The nature of the guidance and support provided.
The types of content and activities.
The forms of communication and collaboration that are encouraged.
The ways in which reflection is encouraged and how the participants can demonstrate achievement of the learning outcomes.
Conole (2015:248)
Whilst the purpose is in the first instance for the course designer and author, the same information could be of value to learners. This helps to set expectations of the purpose of the course, from the design perspective, and by doing so will enable the learner to compare their intended form of engagement against what has been designed.
In fact, this ‘learning how to learn’ does not need to be formally explained but could be integrated into the design of activities. This is crucial within short, open online courses, where the learner may be looking for immediacy over learning outcomes. Time may not be fully allocated, from the learner perspective, to learning how to learn. The data shows that perhaps, online learners in MOOCs and open courses simply follow previously known routines. This has big implications for allowing an individual learner to be able to contextualise the course content and achieve learning outcomes relevant to their practice.
What the data shows about MOOC learner behaviours
Most participants will take the course in a linear pattern.
The graphs below are indicative of patterns found in the professional development MOOCs I manage. The x-axis represents each course step (page of content). The scale on the left is the displacement of first access to a step. This is the mean (average) of the modulus (i.e. value, not negative) of deviation from a linear pattern.
For example, if a learner started by visiting step 1, then visited step 2, then visited step 3. This is a fully linear pattern. The value of each step’s deviation from the linear pattern is 0. If the learner started by visiting step 1, then visited step 5, then visited step 4. The deviation would be: 0, 3, 2. Step 5 is three more than the expected next linear, which is step 2. Step 4 is two less than the expected next linear step from 5, which is step 6.
This may come across as a little contrived, but it overcomes the typical pattern of attrition exhibited in open courses (steep drop-off, as explained by Ferguson and Clow, 2015). Instead of looking at the cohort as a whole, we are now looking at individual learner behaviours. The graphs below show that in the majority of cases, engagement with course content is indeed linear. The peaks show parts of the course that learners deliberately reach out for, outside the linear flow. Typically these are steps at the start of each week, but also key summary steps and features such as Q&A recordings.
Some courses had less pronounced peaks at the start of weeks. The course providing research and evidence to inform practice has more linear week-by-week engagement compared to a course which is more about specific practices. This might be inferred that learners are selecting new approaches that they wish to develop, or drawn to highlighted content (for example steps promoted in course emails). Also note-worthy is that very few participants will start elsewhere than at the beginning. Across the courses analysed, 10-13% of learners entered the course not on the first step.
However, interpretation of these graphs still holds the caveat that the learning platform itself will also influence behaviour. When a learner joins a course on FutureLearn, they immediately visit step 1.1. They then click through step-by-step through the course. Only if they actively visit the weekly layout will they see the complete course content, grouped steps by theme and truly be able to adopt non-linear approaches to course engagement.
Social learning is not mandated
Whilst the platform influences some behaviours, others require a greater sense of purpose from the learners themselves. 20%-30% of learners comment on the professional development courses analysed here. Yet, every step (with only a few exceptions) will have a learning activity that leads the learner to make some form of contribution. Each step on FutureLearn has a comments box, which learners often use in a variety of ways. This may be to post their own thoughts, to provide an active contribution, express an opinion on the content, or simply to acknowledge they’ve read the step.
Since amending the style of how activities are written to include a clear verb-based heading, (anecdotally) we have noticed fewer of the short, one-line remarks on the course, and more detailed, thoughtful contributions to the course comments. Thus, a richer experience for both contributor and non-contributors who may learn from the ideas and experiences shared from others.
Most comments are not replies to other learners. Again, this may be due to the social learning pedagogy requiring interpretation and understanding on the part of learners. The act of commenting upon someone else’s thoughts, ideas and experiences, may seem daunting and carries risk of misinterpretation. What is clear is that completion patterns differ (statistically significantly) based on: commenting, responding to others, or receiving a response to comments. However, again, there is a caveat that completing gives more opportunity for commenting (Swinnerton, et al., 2017). Behaviours can be suggested to learners, with reference to types of contributions that they could make. For example, in the Quick Guide to Effective Online CPD for each course, learners are encouraged to “offer your understanding; pose questions to the group; respond to others’ contributions; provide support; share your experiences.” These are loosely based on the suggestions by Laurillard (2012) to categorise approaches learners may take when contributing to discussions:
Question… Explanation… Conjecture… Comment… Critique…
Laurillard (2012:153)
Again, what the data fails to capture though is how the activity is placed in the learner’s context. A single learning activity could well be a reflection exercise for one learner, or a stimulus for offline discussion or experimentation for another. Though designed within a ‘social learning’ pedagogy, the learner is not tied down to this pedagogy as, particularly with open online courses, they have full control over their own engagement with that activity. How that activity is interpreted, and importantly acted upon, will be dependent on the prior experience, capacity to change practice and ability to reflect, which will be unique to each learner. Therefore, an online learning design doesn’t have to fit one pedagogy.
What is the impact on learning?
Although most learners still exhibit patterns of linear behaviour, whether through the ingrained traditions of education or implied patterns due to platform interfaces, those that do complete open online courses are capable of achieving learning outcomes that are valuable to them. There are many features of open online courses that allow this flexibility of both pedagogy and learning achievement: the mix of the cohort, the time-independence, the structure and form of content. Personal ownership of learning arguably also plays a crucial role. Here are just two quotes from learners:
“Love the structure – bite-sized pieces, and very useful discussion from participants. I’m new at teaching biology, feeling much more confident about planning.”
Learner feedback
“Seeing the practical aspects of this and exploring the comments and activities suggested by this learning community has been superb.”
Learner feedback
These quotes, perhaps subtly, indicate how different pedagogies have been adopted by the learners: both acknowledge the value of discussion; the first explicitly notes the ‘bite-size’ approach; the second notes the clear links to practice; the first also has a clear personal driver to boost their confidence.
Currently, the learner feedback is only representative of completing learners. Those who decide not to continue to the end of the course, for whatever reason, are unlikely to skip to the final page and offer their reflections on learning. However, 98% of learners surveyed indicated a high or medium impact on themselves after completing one of the professional development courses. With such a diversity of needs and learners, yet with pre-defined activities, for that level of impact we cannot assume all learners are learning the same way.
Open pedagogy
On a personal note, when I first started designing and managing a programme of open, online courses (MOOCs) for professional development, I said that I had to rethink everything I thought I knew about online learning. Social constructivism and communities of practice just didn’t seem to work the same way in open courses lasting at most 5 weeks and with many different levels of experience, knowledge and learning goals within the cohort. Yet, as the data and success measures repeatedly show, open online courses don’t have to abide by the rules of a particular, specific and defined pedagogy. The pedagogy can be open.
To conclude then, I am still very much steered by a particular learning approach at the design stage. As any professional does, I make a choice about the type of learning technologist, course designer, educationalist I want to be. This is influenced by my own prejudices about what I believe to be ‘good learning’ and the research I select to justify my position. In course design, I have a responsibility to be open about this, to explain to learners how the course has been designed a particular way. However, open online courses have the potential for us as learning designers to also acknowledge the value of learning in a different way to how the course has been designed. Rather than seek data to support our designs, we need to find ways to use data to best support learners in achieving their development goals. Alongside challenging learners with the course content, I also need to be challenging learners in their approach to professional development through online courses, and this includes empowering learners to make choices about what they will and won’t engage with. In doing that, they will both meet their learning needs and be open to unintended learning outcomes too.
References
- Ferguson, R. and Clow, D. (2015). ‘Consistent Commitment: Patterns of Engagement across Time in Massive Open Online Courses (MOOCs)’, Journal of Learning Analytics, 2(3), 55–80.
- Laurillard, D. (2012). Teaching as a Design Science. Abingdon: Routledge.
- Swinnerton, B., Hotchkiss, S. and Morris, N. P. (2017). ‘Comments in MOOCs: who is doing the talking and does it help?’, Journal of Computer Assisted Learning, 33, 51-64.
- Veletsianos, G., Collier, A. and Schneider, E. (2015). Digging deeper into learners’ experiences in MOOCs: Participation in social networks outside of MOOCs, notetaking and contexts surrounding content consumption. British Journal of Educational Technology. 46(3), 570-587.
Leave a Reply