Posts Tagged ‘Evaluation’

Case Study: Practical Insights for Selecting a Learning Management System

October 11, 2012 Leave a comment

Social Accountability International (SAI) is an award-winning company that provides training,  policies, and procedures to help organizations protect the basic human rights of workers around the world. One of the company’s key services is named Social Fingerprint®, a program to help organizations understand and measure their social impact, and then learn how to improve it.

The Social Fingerprint® program requires that participants undergo a training program. Traditionally, this training consisted of two to five days of instructor-led, classroom-based instruction per year. This approach created a number of challenges.

  • For one, SAI’s customers are global, making face-to-face instruction difficult.
  • In addition, once individuals completed their training,  they would often return to work and would soon forget what they had learned. The result was that it was exceedingly difficult for customers to implement SAI’s recommendations.

In the hope of addressing these and other challenges, SAI decided to expand their learning strategy to include self-paced, online learning. Support for this initiative required the acquisition of a learning management system.

Brandon Hall Group has just published a case study describing SAI’s move from instructor-led training to a blended approach to learning. You can download a copy of this case study here.

What Do Learners Really Think of Their Learning Environments?

February 16, 2012 Leave a comment

Richard Nantel, Vice President, Enterprise Learning Solutions, Blatant Media | Absorb LMSLuana Vargas, Educational Development Manager at International Society of Arboriculture, recently posted a great question on a LinkedIn discussion group:

“Do you directly sell courses to your e-learning end-users? If you do, how are you evaluating (reaction) the courses or the platform in which they are offered?”

First of all, kudos to Luana to care about learner reaction and satisfaction. As a commercial content provider, it’s critical that her learners see the value in her courses. What I especially like about Luana’s question is that she hasn’t restricted feedback to the course content. She’s also looking to gather satisfaction data related to the platform.

Donald Kirkpatrick defined level one of his Four-Level Evaluation Model in Instructional Design as: “The participants’ reactions to a training event. It is basically a measure of (internal) customer satisfaction.” Here are typical level one evaluation statements that students are asked to rate, usually using a Likert scale:

  • I understood the learning objectives.
  • I was able to relate each of the learning objectives to the learning I achieved.
  • I was appropriately challenged by the material.
  • I found the course materials easy to navigate.
  • I felt that the course materials will be essential for my success.
  • Etc.

The statements above are valuable but focus on content. Occasionally, as in this example, the evaluation includes statements about the environment:

  • I found the room atmosphere to be comfortable.
  • I was pleased with the room set-up.
  • I experienced minimal distractions during the session.

In online learning, the environment is typically a learning management system. Consequently, evaluation of the relevant LMS features and characteristics should be a part of post-course evaluations. Here are a few statements that may be appropriate to your learning initiatives:

  • The purchase process for this course was straightforward.
  • Logging into the system was easy and intuitive.
  • The interface was intuitive.
  • The interface was attractive.
  • The learning environment reacted quickly to my commands.
  • The learning environment behaved as it should, free of glitches, or crashes.
  • The learning environment allowed me to assess my progress.

Dissatisfaction with the learning environment will impact learner satisfaction with the content in the same way a good meal in a dirty, brightly lit restaurant may result in a poor restaurant review. Ask your learners how they feel about your LMS. Address quickly any concerns that may be uncovered through this evaluation.

Doing Things: The Ignored Learning Content Type

December 6, 2011 2 comments

Richard Nantel, Vice President, Enterprise Learning Solutions, Blatant Media | Absorb LMS

You can feel pretty smug and confident in your knowledge of a subject after having completed a course. Attempting to apply your newly-acquired knowledge, however, can be very humbling. Why is that we think we know something and only discover we don’t when we try to apply that knowledge? There’s a significant disconnect between our perception of our abilities and our actual performance.

The famous learning theorist Donald Kirkpatrick was well aware of this gap. That’s why his four-tiered model for learning evaluation placed learning and behavior on two separate levels:

Level 1: Reaction—To what degree participants react favorably to the learning event or content

Level 2: Learning—To what degree participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in a learning event

Level 3: Behavior—To what degree participants apply what they learned during training when they are back on the job

Level 4: Results—To what degree targeted outcomes occur as a result of the learning event and subsequent reinforcement

The word “course” is used very broadly within the Absorb LMS environment. An online course can consist of lessons, quizzes, exams, tasks, and objects, in any quantity and order. Here’s the definition of these content types:

Here’s the outline of a hypothetical sales course:

Chapter 1: Getting Started

  • Overview of our Customer Relationship Management (CRM) application (Video)
  • Quiz
  • Task:  Log into the CRM and create your user profile

Chapter 2: Managing contacts

  • Etc.

The third item in Chapter 1 is a “Task” content type. The learner is required to log into the CRM and create his or her user profile. How will Absorb LMS know if the learner has completed this task? It won’t, of course, but an administrator will be able to check and mark off the task as complete within the LMS. The administrator will also be able to grade the learner on how well the task was completed.

There are a lot of things you could do with the “Task” content type. You can even use tasks for non-learning activities. For example, when a new employee is hired, he or she invariably has documents to complete and return to the HR department. To streamline the onboarding of new employees, you could create a course that looks like this:

Section 1: Welcome!

  • Overview or our company
  • Download the documents attached to this course, complete, and e-mail to
  • Etc.

In the example above, the task to complete and return documents to HR has nothing to do with learning. Afterall, the learner will only complete this paperwork once,. The task is simply inserted to standardize the onboarding process and ensure nothing falls through the cracks. Once the employee’s documents are received by HR, a representative of the HR department can log into the LMS and mark off the paperwork as complete. If information is missing, the HR representative can e-mail the employee directly from within the LMS and inform them of what’s required.

Used as illustrated above, the LMS becomes more than a learning environment. The system becomes an integral part of an organization’s workflow.