LWMN018: A Competency-Based Analytics Mastercast | Week of November 27th, 2017

0
1541
The last week in moodlenews 27 NOV 17

LWMN on iTunesLWMN on Android


Hey there – welcome to the Last Week in MoodleNews, I’m Stephen Ladek from MoodleNews.com.

In this episode I’ll be talking about the most important stories from the Moodleverse for the week of November 27th, 2017. Also, I dive deep with a Competency-Based Analytics Mastercast.

Before we get started, a quick reminder share this show with a fellow Moodler or a colleague or friend who is interested in learning, EdTech or innovation. You can also find these episodes on our twitter feed @moodlenews or on facebook.com/moodlenews.

THE WEEK THAT WAS

In this section, I summarize the three most popular posts from the last 7 days on moodlenews.com

  • Our top story last week was our review of the Cron Lock Statistics plugin for Moodle.
    • So… what is “Cron”? Is it the green-eyed cousin of Tron? While that would be cool, the real answer is that “Cron” is essentially the heart beat of an open source system based on Linux like Moolde.
    • Cron makes sure the system is always up-to-date. In Moodle, Cron is responsible for things like making sure scheduled activities run automatically and instant messaging.
    • The Cron Lock Statistics plugin helps with identifying problems where Cron is involved, like activities that are running suspiciously long, taking up too many resources.
  • Next, we have another plugin set, part of a gamification approach by Adrian Greeve from Moodle HQ.
    • Adrian talks about the Stash plugin family – which is notable as one of the few awarded the “Reviewer’s Choice” badge from the Moodle Directory.
    • With the Stash family, students take part in a “treasure quest” across course activities. The idea behind Stash is to reward attentiveness, which in turn requires teachers to “stash” coins or magical items in unusual places within Moodle.
    • But on a deeper level, Stash could be the first in a trend where curiosity is fully embraced in learning, particularly as a problem-solving strategy in corporate settings. Whether it will catch on depends on the answer to this question: Is it possible to teach adults how to be curious?
  • And, finally, we bring you the latest updates about MoodleMoot India, which will take place next December 18th to 19th, in Mumbai.
    • The Moot could not have a better opener: Martin Dougiamas, the Founder of Moodle. Joining him from Moodle HQ, Solange Lalonde and Tom Murdock will talk about their recent but prolific work in Moodle’s Education Team.
    • I cannot overstate the importance of a MoodleMoot in Mumbai. India is already the largest market for Higher Education in the world, with one of the highest population growth. Which explains the estimated $2-billion-dollar level of investment in online education by 2021.
    • Registration are still open, and include everything an Indian Moot has to offer: Keynotes, Masterclasses, Community Panels and a Moodle Party to close the event.

Find these stories, and all the links mentioned in this episode at moodlenews.com, and if you’re looking for more information and resources, send us an email, a comment or a tweet.

MOODLE PRACTICE

In this section, I focus on a practical way to help you up your Moodle game.

This week we’re going to look into the future of Moodle practice through two trends that are here to stay: Analytics and Competencies.

  • Let’s start with Analytics. Or, the art and science of turning data about behavior into information that drives strategic decision-making.
    • The fact that a robust analytics engine is shipped as part of the core of the just-released Moodle 3.4 is supposed to be a game changer, especially in organizations that do not have the resources to purchase an analytics dashboard, or the technical talent to build one in-house.
    • One reason why analytics are being offered directly by Moodle is that we are in the age of “Open Data.” As we all learn the new playbook, more people are starting to realize how valuable, but above all, how critical it is that users remain both in control and also the main beneficiaries of analytics outcomes.
    • As such, as a global project, Moodle is emphasizing to every user that the data it captures is for them to take advantage of, regardless of background, learning goals or budgets.
  • Next, we have Competencies: the unit of measurement of every learning intervention, or at least we hope!
    • The subject of Competency-Based Education might not be as “hot” as analytics, but that doesn’t mean there is not interesting work being done on CBE by some of the brightest minds in the field.
    • As an example from the latest MoodleMoot Australia, the presentation from Courtney Bentley & Claire Machia from eThink contained loads of inspiration, with the caveat that the adoption of Competencies over the past year has been somewhat subpar.
    • There are many benefits associated with competencies, but the underlying focus should be accountability. Every action in your learning project should contribute to one or more competencies, otherwise it should not exist. It sounds simple, and it really is, but it’s also true that changing the way some organizations think and apply CBE is a real challenge.
  • Alright! Perhaps you’ll agree with me that thinking about Competency-Based Analytics, or CBA makes sense. Now, we’ll add the final ingredient: ROI. You might ask: why?
    • Well, I believe that adding a performance metric helps closing that sometimes elusive “feedback loop,” which is kind of the point of analytics.
    • However, I have to admit that I’ve chosen ROI because it’s a common performance metric and one that’s been extensively covered. Hopefully as more organizations share their ROI it would be possible to benchmark the efficiency of learning interventions using Moodle. But ROI is not without it’s problems, especially in education contexts.
    • Another issue here is that Moodle does not offer a tool to estimate ROI or any other organizational performance indicator. So while I’m not going to get into the details of setting up an ROI system in this episode, I encourage you to be conscious about the performance metrics in your learning project, especially if your project has limited resources.

Now, with all of that explanation out of the way – let’s dive into some nuts an bolts about brining CBA to life in your Moodle. In fact, you might want pause the show for a second to grab a pen an paper before I dive into this next section.

  • Ready? Ok. If you think about it, Competencies are to a large extent about bringing project management principles into education.
    • In the classic Logical Framework approach, for example, a project’s goal is the sum of its components, nothing more and nothing less. Now, with more agile methods, the basic structure is actually not that far off. Perhaps the main difference is that now you and your team should be willing to re-evaluate your goal in the middle of the game.
    • For a learning project, you can make the activities out of the competencies, and the project’s components would be the competency sets, arranged by subject. A competency set could be a subject, but remember that competencies don’t have to be exclusive to one subject.
    • So, your first CBA assignment: What is the one learning goal of your learning project? What is that one problem on which you want to focus for the time being?
    • When I was a project management consultant, it was always interesting to see how common it was for organizations of all sizes to get stuck with that question. Usually the person in charge was really hesitant to move forward until the perfect goal was found.
    • But if you’re lucky, you are part of an organization where everyone realizes that focusing on having a solid structure is more important than reaching a goal at all costs.
  • Now it’s time to bring analytics into the mix. It might feel time-consuming now, but once you’ve created a solid foundation, you will be able to use it going forward.
    • So, here’s assignment number two: Write down every competency. This will be your project’s Competency Framework.
    • Now, for each competency, add at least one way to measure and track it. If you are using Moodle 3.4 or the Inspire plugin, you can take advantage of the “risk of dropping out” estimate and use it as the risk of not achieving the competencies associated with the course from where they are at risk of dropping out.
  • Now, before we move this structure onto Moodle, I want you to apply the following, simple “stress tests.” Remember, it doesn’t have to be perfect, it just have to be “good enough”
    • Test #1: Check for redundancy. Make sure the full set of competencies you’ve listed does not contain items that are too similar. If you find any, just merge them! However, remember that this is different than allowing more than one course to be associated with the same competency.
    • Test #2: Check for bias. At the beginning, it’s inevitable that your project is filled with assumptions no one is aware of. Whenever possible, the achievement of a competency should rely on any one person’s opinion as little as possible. Ideally, anyone should be able to follow and replicate the process of building an analytic from the ground up. This is not only for the sake of transparency, it also fortifies the project’s foundations.
    • Test #3: Check for structural stability. Meet early and often with the project’s core team. It’s critical to ensure there are plenty of opportunities for people to share challenges and lessons learned. Make sure everyone gets to participate in the same level regardless of background, personality, etc. But above all, be ready to ask in every meeting: Do we still want to achieve the goal we originally set out to accomplish? Or perhaps it is time to pivot…

It’s finally time to put Moodle to work. The amount of options and tools available within Moodle is too long to cover here, but you can get a sense of what different levels of CBA implementation look like.

  • First, you can set up the basic structure of Competency Analytics to get things going.
    • The first step, naturally, is enabling Competencies. An Admin must visit Site Administration > Competencies to do so. This will open a panel where you can add your project’s Competency Framework.
    • Next, it is time to link competencies to Moodle courses. Teachers in charge of each course can find a “Competencies” option and add the competencies associated with the course. Remember that courses can share competencies.
    • Finally, set up the Analytics engine. In Moodle 3.4 an Analytics section is available from the Site administration. In its basic setup, it will generate a report showing students at risk of dropping out, which you can generate as often as you want.
  • Next you need to improve your CBA game.
    • You have already linked competencies to Moodle courses, but you can also link them to specific activities inside courses. In so doing, completion of the activity can award the competency automatically. Just be careful not to make it too easy for students to achieve a competency, as they are supposed to be long-term outcomes.
    • You can enable profiles with Competency breakdowns for each student, which lets you see their point in the path of accomplishment. These profiles can be made public, so each student can see them and compare, or just keep it to teachers only.
    • And of course, you can take advantage of the several plugins available that help you visualize, monitor and streamline your CBA workflow.
  • Finally, here are some topics that may or may not fit the needs of your organization, but could help you discuss deeper issues in CBA implementation.
    • You can adopt or export a Competency Framework, and in Moodle.net you can upload your own, or download the several that are available. But in my opinion, unless your project is designed to comply with existing standards from the start, it’s better to use other Frameworks as reference but focus on sharpening your own.
    • Your project might have one learning goal, but if there are many ways to achieve the competencies associated with that goal, you can allow for different Learning Plans, or specific paths.
    • And before I end, let me stress once again the importance of the ROI conversation, which might be better called the “organizational performance metrics” conversation. I hope that by setting a CBA structure, your organization can identify ways to be more efficient in the way resources flow to attend the students’ population needs.

So, that it! Of course, you can always get deeper with Competency Analytics in Moodle. The Moodle Documentation offers plenty of resources, starting with a Frequently Asked Questions page. And check out our previous coverage on both Analytics and Competencies at MoodleNews.

THIS WEEK IN MOODLENEWS

In this section, I discuss what we’re excited about publishing this week at moodlenews.com.

  • We might be witnessing the start of a revolution in LMS, thanks to the rise of content integration, mainly OER.
    • The designation OER, for “Open Educational Content,” has been a blessing and a curse for companies looking to secure partnerships and develop sustainable business models with open-licensed content.
    • To capture the market, companies like Lumen Learning or LinkedIn are securing technological integrations with LMS, offering a value-added bridge for OER delivery.
    • We’ll take a look at the implications of these technological partnerships, as well as what Moodle could (and should) do. Is OER-LMS integration here to stay?
  • Next, with the recent release of a report by Gallup on the “Student Advice market,” if you can call it that, we take a look at the ways technologies like Moodle can help career advisors, coaches, mentors, tutors and the like, give the best advice to students.
    • The Gallup study surveyed over 22 thousand adults in the US, and it found four levels of formalization of the role across organizations.
    • It is clear that more than half of the student population receives some informal advice at best, and most of the time it comes from their “informal networks,” not school.
    • In Moodle, the ability to create custom roles makes it possible that advisors stay on top of student performance, to give critical and personalized advice right when students need it most.
  • Finally, we report on the latest interview of Martin Dougiamas, this time with Brian Carlson from Moodle Partner eThink Education.
    • On the heels of Dougiamas’ first appearance at the Educause conference, they gave an interview to the EdTech portal Nibletz.
    • The interview includes a few revelations about the use of Moodle in corporate and diplomatic levels, like the fact that over 11 departments at the United Nations deploying Moodle “in a variety of ways.”
    • Brian also shares some details about the business side of the bargain, with eThink’s efforts to provide ever-improving customer service. (Full disclosure: eThink Education is a sponsor of MoodleNews.)

Alright! That’s it for this week. Thanks for listening to The Last Week in MoodleNews Podcast. If you like what you’re hearing please take just a few seconds to give us a review on iTunes or whatever podcast app you happen to be using… And, of course, join me next week for all the most important news about Moodle.

Subscribe to Last Week in MoodleNews Podcast via iTunes, RSS, Stitcher, Android, YouTube or listen to it at soundcloud.com/moodlenews.

LWMN is hosted and produced by Stephen Ladek, with writing, research, and editing by Cristian T. Duque and Joseph Thibault.