Evaluating Training Across Organizations and Supply Chains

Evaluating Training Across Organizations and Supply Chains

By Dave Johnson, Phylmar newsletter editor

“More than ever organizations need to achieve the very best training and performance improvement possible,” states learning effectiveness experts Robert O. Brinkerhoff and Dennis Dressler.

Why? Today’s competitive environment demands a workforce that learns quickly, and consistently turns new learning into organizational, team, and individual performance improvements using metrics such as these:

• Increased compliance with supply chain codes of conduct, standards, performance expectations, and brand protection among all tiers of suppliers
• Fewer internal and external complaints (i.e. about safety) from organizational leadership, employees and stakeholders
• Better communication, collaboration and alignment (i.e. consistent safety performance) within organizations and throughout supply chains
• Enabling a safe workplace and advancing safety knowledge, skills, incident reporting, and performance rates (fewer injuries, illnesses, fatalities, property damage, and enhanced leading indicators)
• Reduced operating downtime and increased productivity
• Ability to quickly adopt new technology (including safety technology)

This year, with the many COVID-19 disruptions, many companies have diversified their supplier base. Training is critical for companies in assessing their Tier 1, Tier 2 and Tier 3 suppliers – including their level of knowledge, skills, and ability to comply with company standards.

Training- a term that has been largely replaced by “learning and development,” is an enabler and capacity builder. It is essential for organizational and supply chain efficiency, predictability, agility, resiliency and identifying vulnerabilities.

Brinkerhoff writes in the article, “Getting Real about Evaluation” that the training alone is never the sole cause of its success or failure. About 80 percent of training failures are not caused by flawed training interventions, it’s more so inadequate supervisory support or preparation for learning can impede the training’s impact, he writes.

“Training in how to take specific actions by using a tool is less likely to fail because the trainee and the supervisor know the trainee will be using the tool, and correct or incorrect use can usually be detected immediately,” says Cass Ben-Levi, Director of Phylmar Academy. “Training in preventing musculoskeletal injuries, work organization, sexual harassment and workplace violence may be more subject to non-training influences. The training may not be put to use right away, and the injury may take time to develop,” she adds. 

Evaluating training effectiveness takes many forms. Post-training quizzes, one-on-one discussions, focus groups, employee surveys/polling, participant case studies and certification exams are a few of these methods. The more data you collect on measurable outcomes, the easier it is to quantify both your training effectiveness and return on investment. According to Training Industry magazine, employee training and education expenditures in the United States are growing incrementally at 14 percent every year.

“The post-training quiz is the least important element in gauging failure,” says Ben-Levi. “Most people can remember what they learned for a short period of time. More importantly: Did the trainee put the new knowledge or skill to work? (Just because they know how to do something doesn’t mean they will.) Did the trainee continue to use the training over time? (Are they still using it after 6 months, a year?).

“One of the best ways to assess effectiveness is to ask workers through a survey. They need to know their responses are confidential and there will be no negative repercussions for honesty. Once those ground rules are accepted, you can often find the most useful information from those workers who did not understand the training, did not like the training, do not use the training, or have found that it does not work for them. 

“Without the full support of management, it is nearly impossible to assess training effectiveness. With that support, you can learn what is working, what is not working, and pinpoint what improvements need to be made,” Ben-Levi says.

 Macro and micro evaluations

Training evaluation is a diagnostic tool with micro (the impact on individual learners) and macro (impact on organizations, enterprise-wide systems, and far-flung supply chain) applications. 

A training scenario was recently implemented by a multinational corporation and the Phylmar Training Academy with follow-up evaluation conducted by Robert Brinkerhoff. The goal is to evaluate the impact of the corporation’s supply chain training on its suppliers’ level of sophistication regarding occupational health and safety cultures. 

After training, supply chain partner cultures are evaluated at all levels of their organization starting with leadership. Do leaders prioritize safety and health through visible behaviors daily on the floor? Is it clear that safety and health are part of organizational strategic plans and communications?

In this training, the workforce is evaluated for health, safety skills and capacity including awareness, hazard recognition, safety technical skills, and safety problem-solving as a core competency.

Supplier cultures are evaluated and scored afterwards for evidence of health and safety fundamentals such as job hazard analysis (JHAs). JHAs deconstruct job tasks into step-by-step functions to uncover hazards, ensure work processes are standardized, and safety rules are developed based on the results.

Risk assessment is another fundamental process that is evaluated after training. Assessments include external risks as well as internal system, machine, operational and individual behavioral risks. Risk assessments are conducted beyond health and safety to measure the likelihood and potential severity of geographic (weather and natural) risks, political infrastructure risks, as well as the need for business continuity and contingency planning.

Two additional fundamentals are also evaluated after mandatory training: the existence and effectiveness of health and safety management systems, and leading and lagging performance indicators. 

Post-training evaluation scores result in a maturity curve to assess health and safety cultures in the supply chain. 

The lowest scores indicate a reactive culture that waits for accidents to happen before any action is taken while higher scores reflect cultures that have moved beyond the reactive mode to emphasize compliance with supply chain standards and a desire to improve performance. Higher scores show that management systems exist and health and safety resources are consistently maintained. 

The highest scores identify cultures where safety is an embedded value, and leadership and the frontline staff are consistently engaged. Cultures that perform successfully are recognized as world class and are benchmarked by other companies because the most effective cultures realize safety results lead to business success- they are widely shared and proactive health and safety systems drive continuous improvement.

For more information on the Phylmar Academy’s training programs, click here.