Navigating the Labyrinth of Measuring Learning Objective Achievement

Mar 07, 13 | 12:06 am

By Jim Hartigan

Training evaluations…there are many different approaches and methodologies: Kirkpatrick’s four levels, Phillips additional ROI calculation, Kaufman’s five levels, Tyler’s pre-test/post-test, Stake’s responsive evaluation, and on and on. A Google search of the term “training evaluation” returns over 223 MILLION results!  With so many different choices, it’s all-too-easy for a Learning and Development Professional to get lost in the dreaded Labyrinth of Perpetual Learning Objective Confusion (beware the deadly Instructional Designer Minotaur!)  Well fear not! We’re here to equip you with a map and enchanted sack full of magical tips to help you navigate your way to safety – and successful measurement of learning objectives!

The first question you will want to answer is “what do I want to know about my training course?”  Specifically, is it important to you:

• How the learner felt about the course?  If so, consider utilizing “smile sheets” via online or paper and pencil surveys.
• What knowledge was transferred to the learner? (And, frankly, who isn’t concerned about this one?)  To determine achievement of learning objectives, capture metrics based on integrated knowledge checks, quizzes, tests, or performance demonstration.
• How efficient your development and delivery process is?  Measure the cost effectiveness of the course design, development, delivery by comparing it to some metric of student time (per seat hour, per student, per completed course, etc.).
• What impact your training course had on your organization?  Measure the change in key benchmark data such as customer satisfaction, revenue, or a tracked behavior such as call handle time, conversion rates, etc.

All of these measures have importance in their own right, to be sure.  Happy students make for engaged students and referrals to others to take the course (“positive word of mouth” as we say in the marketing realm).  Efficiency in training design, development, and delivery is critical to the bottom line of every organization – and gaining funding for development of future courses.  At Orgwide we believe when it comes to revamping your training evaluations or beginning a training evaluation program, getting back to the basics is in order.  To that end, we believe the building of understanding, or successful knowledge transfer, enables changed behavior which drives organizational impact.

Therefore, the foundation of successful training is the creation and achievement of well written learning objectives.  As such, we incorporate the creation of our tests, quizzes, and knowledge checks early in the course development process.  So if you’re looking to improve your training evaluation process, consider starting with better measurement of learning objective achievement.

If you’ve been in the Learning and Development arena for any length of time, you’re likely familiar with the ADDIE method of training course design (Analysis, Design, Develop, Implement, and Evaluate).  Our method involves creating the tests, quizzes, and knowledge checks at the conclusion of the Design phase, after the learning objectives have been determined but before course development begins.  We find this process provides a number of benefits including:

• Validation of the “goodness” of the learning objectives – If you have difficulty measuring the achievement of the learning objective, you may need to revisit the learning objective phase of the project to ensure that your objectives were crafted in an inherently measurable way, using demonstrable and quantifiable language.
• Focusing the development efforts – With clearly defined learning objectives and defined methods of measuring student achievement, content development can stay focused with the “end in mind” thereby eliminating non-essential materials which could confuse the student and lengthen the learning experience.

Of course, the best way to measure actual achievement of learning objectives is to conduct pre-testing of the learner to measure both before and after the training event.  If it is not possible to test learners before training, or when the learning objectives require demonstration of a newly acquired skill, you may need to test their performance after training (demonstrating the new skill) and ask the learner whether or not their understanding or skill came from the training session.  This “self-reported” method of measuring learning objective achievement is less than ideal, but better than forgoing the analysis completely.

Whether you are looking to improve your established training evaluation metrics or are incorporating new training metrics and evaluations with a full refurbishment of your training program, starting with the measurement of your course learning objectives is a “back to the basics” approach that will always pay dividends.  Until next time, remember to take care of the customer, take care of each other, and take care of yourself.

Reprinted with permission of

About Jim Hartigan

Jim Hartigan, Partner joined Orgwide Services in April 2010.  A 30 year hospitality industry veteran, he most recently served as SVP, Global Brand Services at Hilton Worldwide where his team was responsible for ensuring excellence in quality assurance, customer satisfaction, market research, brand training, media planning, and sustainability across 10 brands and more than 3,400 hotels in 80 countries around the world. In 2004, Hartigan was Orgwide's first client launching online learning for Hilton in North America.