Trainers and training providers know that their success depends on demonstrating the value of training investment. The simplest way to prove training’s value to a client is to document that the training has achieved its desired outcome, through evaluation.
Evaluation methods help determine whether training achieves its objectives. This point is particularly important for Registered Training Organisations (RTOs) to ensure training delivered meets the Australian Quality Training Framework (AQTF) and/or the VET Quality Framework (VQF) standards. Australian Qualification Framework (AQF) programs are structured and designed around a set of outcomes to be achieved (competencies) during a particular timeframe.
Herein lies the fundamental secret of evaluating training: The evaluation process and procedure must be incorporated at the start, and it must be an integral part of any program development process. For example, baseline data must be collected before training begins; otherwise, there’s no basis for any meaningful comparison with training program results. Historically, evaluation of training has been done after training rather than before and after. If program development follows the classic steps of assessing needs and generating objectives, the evaluation criteria that follow are then based on measuring how well the program components: learners, trainers, and training materials, have met these objectives. For AQF programs, training packages (an accredited courses) are a great help to course developers as provide the training objectives expressed as a set of outcomes (competencies) recognised by the industry as the standard competencies required to perform a specific job. The evaluation will measure how well the learners have developed the required competencies, whether those competencies can be transferred to the workplace, and the impact of the new competent employee in the workplace performance.
A sound system of evaluating training provides valuable information for the learner, client, training management, and senior corporate management. The information elicited from training evaluations should be the final instrument on which training decisions, such as program additions, changes, or deletions, should be made. Good evaluations document the results of training programs, which can be used subsequently to prioritize training needs at the organizational level. Then financial and other resources can be shifted from training that has less impact on corporate goals to those objectives that have the most favorable cost-benefit ratio.
These are some benefits of evaluation:
- Determining business impact, the cost-benefit ratio, and the ROI for the program: What was the shift in the identified business metric? What part of the shift was attributable to the learning experience? Was the benefit to the organisation worth the cost of providing the learning experience? What is the bottom-line value of the course’s impact on the organisation?
- Improving the design of the learning experience: Evaluation can help verify the needs assessment, learning objectives, instructional strategies, target audience, delivery method, and quality of delivery and course content.
- Determining whether the objectives of the learning experience were met and to what extent: The objectives are stated in measurable and specific terms. Evaluation determines whether each stated objective was met. Nevertheless, knowing only whether objectives were met isn’t enough; a trainer must know the extent to which were met. This knowledge help focus future efforts for the content reinforcement and improvement.
- Assessing the effectiveness and appropriateness of instructional strategies: case studies, tests, exercises, and other instructional strategies must be relevant to the job and reinforce course content. Does the instructional strategy link to a course objective and the course content? Is it the right instructional strategy to drive the desired learning or practice? Was there enough instruction and feedback? Does the strategy fit with the organisation’s culture? Instructional strategies, when used as part of evaluation, measure the knowledge, skills and attitudes the learning experience offers.
- Reinforcing learning: Some evaluation methods can reinforce learning. For example, a test or similar performance assessment can focus on content to measure and evaluate content retention. The measurement process itself causes learners to reflect on the content, select the appropriate content area, and use it in the evaluation process.
- Providing feedback to the trainer: Did the trainer know the content? Did the trainer stay on topic? Did the trainer provide added depth and value based on personal experience? Was the trainer credible? Will the evaluation information be used to improve the trainer’s skills?
- Determining the appropriate pace and sequence: Does the trainer need to schedule more or less time for the total learning experience or certain parts of the learning? Were some parts of the learning experience covered too fast or too slowly? Does the flow of the content make sense? Does the sequence follow a building-block approach?
- Providing feedback to participants about their learning? Are participants learning the course content? Which parts are they not learning? Was there a shift in knowledge and skills? To what extent can participants demonstrate the desired skills or behavior?
- Identifying which participants are experiencing success in the learning program: Evaluation can identify which participants are grasping the new knowledge and skills and which are struggling. Likewise, evaluation can identify participants who are excelling at understanding the content and using on the job.
- Identifying the learning being used on the job: What parts of the learning experience are being used on the job? To what extent are they being used?
- Assessing the on-the-job environment to support learning: What environmental factors support or inhibit the use of the new knowledge, skills, attitudes, and behaviors on the job? These factors could be management support, tools and equipment, recognition and reward, and so on.
Although this list is not exhaustive, it does highlight some key purposes of evaluation. Insources have developed a half-day workshop to help trainers and practitioners to design and implement an effective evaluation system.
