Training Evaluation Series Part 3: Measuring Intangible Benefits
When considering the word “intangible” terms like ethereal, ghostly, spectral, unearthly and supernatural come to mind. Not exactly how we think of the efficient and grounded process of calculating the return on investment (ROI) of training.
In the Phillips ROI Methodology (see diagram below), intangible benefits are those that not readily converted to a monetary value; outcomes that are difficult to value. Examples of intangible benefits include:
- Job satisfaction
- Self-efficacy or confidence
- Leadership
- Lower Stress
- Commitment
- Citizenship
Although the ROI calculation usually limits the inclusion of intangible benefits, going for only “hard” numbers like profit, turnover, and errors, they represent many the most difficult challenges faced by organizations today. Can we accurately calculate the value of training without considering the critical contributions of intangibles like improved satisfaction, commitment, and communication?
The key to making intangibles more tangible lies in how we measure them. Since most trainers have neither the time or the skill-set to embark on an extensive validation process for each “construct” they measure, the most timely and accurate way to measure intangibles is through the use of validated tests. When developing tests, researchers carefully analyze each question ensuring that it adds unique value and that it measures what it is supposed to measure (validity) accurately over time (reliability).
Locating valid and reliable tests requires more than Googling terms. The simplest way to begin your search is to use an online database that compiles published tests. Here are instructions that will get you started:
First, identify the construct you want to measure; what intangible benefit would you expect your training or intervention to reap?
Once you have a construct or outcome in mind, begin your search by visiting the “Mental Measurements Yearbook” (instructions on how to access this database at the end of the article) available free and fully online through the RU Library.
Enter search terms and locate instruments that both measure the outcomes you have identified and do so in the correct population group (adults, workplace).
Evaluate the instruments by considering information about reliability and validity.
For example, f you were to conduct a training intervention around stress management and wanted to measure results, you might enter the terms stress and burnout and locate the Maslach Burnout Inventory (MBI) which you may purchase through Mind Garden. Or if you were delivering training or an intervention focused on teams, you might use the OTCI to measure orientation toward team culture. While the MBI is one of the most researched measures of stress and burnout the OTCI, on the other hand, demonstrates reliability in adult work-based populations but is lacking in proven validity, so use this instrument with caution.
In this brief article, you’ve learned about intangible benefits and how to locate and select them. Stay tuned for the next article in this series to find out more about measuring intangibles with test resources in research publications.
Can you name additional intangible benefits that might arise from training and should be measured?
Accessing the Mental Measurements Yearbook:
Log in to the RU Library. See instructions on how to do this from off campus.
Go to the RU Library Home Page and click on the link to Databases.
Enter the search term “mental measurements yearbook” and click on the link to the database
Enter appropriate search terms and locate information about tests. Note the appropriate age and setting (adult, workplace versus children, educational setting) and information about reliability and validity.
RU Training @ Roosevelt University in Chicago 
Great article! In one of my current courses we are reviewing the ROI model and there is no question that the focus in on monetary measures. I often questioned some of the same intangible benefits listed in this article to truly measure validity. Additional intangible benefits that come to mind for me are work/life balance, chance for growth or mastery, and company culture. I’m looking forward to seeing the intangible benefit examples that others post.
Thank you for adding to our list of intangible benefits.
This is very helpful! I have helped people find such tests, but I never thought to use them for my own training work. The benefit there would be that skilled researchers have created those tests, and by the time they get published in Mental Measurements they’ve also probably been used multiple times so hopefully any issues with the tests would have been resolved by then. Of course, the test itself is no substitute for a solid understanding of how to administer tests like this and conduct rigorous studies, but I can see how they’d be a great use. Thanks for the idea!
Here you illustrate important concepts about turning soft data into hard numbers. When designing performance or workplace interventions, practitioners are certainly hopeful to produce a number of intangible benefits. Things like, “increased job satisfaction” or “improved teamwork” are commonplace. When it comes to an ROI evaluation, monetizing these intangibles significantly improves your chances for positive ROI. Thank you for the new resources!
Donna R. Guy
This is a very good article! I’ve often wondered about how to measure the soft or intangible skills in the workplace. Especially, work/life balance, job satisfaction, improving productivity and morale within the workplace. Mastering this will allow companies to reduce staff turnover and increase productivity.
I am a firm believer that happy employees produce more, take less days off and are more eager to go the extra mile to see a product or project completed. I believe a good way to start the measuring process of intangible skills you should get the opinions of those in question perhaps with the use of questionnaire’s or a suggestion box. Either way, the opinions and suggestions provided is a good start to measuring and gaining the necessary data needed.
A few additional intangibles which are equally important to recognize and measure that can have an indirect or direct impact and influence within workplace performance are internal or external cultural differences such as language barriers, socio or economic (positive or negative) influences, learning challenges or disabilities. I believe these additional intangibles link to direct behaviors, morale, turnover, and corporate culture often associated when measuring intangible benefits in the workplace. They often go unnoticed even by the most seasoned professionals because it’s out of sight/out of mind from what’s typically evaluated in the workplace. There may also be a direct correlation of intangible benefits such as job satisfaction, self-efficacy or confidence, leadership, lower stress, commitment, and citizenship. Additionally, can these intangible benefits have an indirect association linked to short or long term results of tangibles often evaluated in the workplace such as training, productivity, and work redundancies?
Excellent additions to the list! Thank you for adding these. I agree, training can have effects on intangibles, and if the outcomes are important, they may be worth measuring.
Very interesting article. Including intangible benefits seems like a win-win situation in my mind. If something has the ability to affect, positively or negatively, a program it would seem that it makes logical sense to include it in the evaluation process. The fact that there are resources already out there, which I knew there were tests but not that they could be utilized for this is a plus. Not a lot of additional resources, time and energy would need to be used in order to incorporate it into existing evaluation methods.
Good information! I love the resources shared in this article. The information provided in this article helps developers measure the intangible benefits that one may question. You’re right, we have to go beyond simply googling results for targeted subjects. That is why I find great pleasure in the resources here and obtaining tangible benefits.
Very helpful and insightful article! When I graduated from Roosevelt University in 2008 the best advice from our then president, Charles Middleton, was to “read and don’t stop reading.” An intangible asset to training in a corporate setting is to gain feedback from the front line employees who are the faces and voices of the organization. Some feedback from these employees can lead to tangible and measurable results and may not be easily discovered otherwise.
An intangible benefit that comes to mind is an increase in confidence. While participants leave a training equipped with a better understanding or new skill, we can’t really measure the confidence gained. And not really the confidence to perform but rather knowing a resource is there should you have issues in that particular area.
Great article I love the resources about the intangible benefits and how to apply them especially when they give the great details of training that may take place. It also shows you how to measure intangible ways of accessing skills and how to apply the measuring outcome of it as well.
Oftentimes, many organizations equip their employees with the skills and tools to help them master their roles and responsibilities in a more practical way. What is sometimes lost or not measured in training spaces is the level of motivation associated with participants after learning. How do we measure of people actually want to apply the knowledge and feel excited about the new behavior. I think motivation is the area where practitioners need to hone in on how it impacts application of learned skills or content. It goes back to ensuring that participants in training programs are fully bought in to the process. By measuring motivation, we are sure to find the gaps in results and desire when it comes to evaluation.
I found this article very helpful as I was struggling to conceptualize how we quantify these intangible benefits. It seems a rather difficult and illusive task especially considering there doesn’t seem to be any standardized or universal accepted values to associate with these things. I guess then it is up to the company or organization to assign their own weighted value to what they consider to be the most important of intangible benefits. Reflecting on previous experience, when I think about intangible benefits I think of work-life balance. I learned all too well in my last position how valuable work-life balance is. It is in my mind what gives you strength to tolerate your job. I think this also addresses another point people often misinterpret or overlook. Sometimes their expectations about a job are a little out of touch with reality. Everyone craves and wishes for that perfect dream job that realizes all their hopes/dreams and life satisfaction. Unfortunately, this is a myth. The reality is you can find happiness and fulfillment in life even with a job you don’t love. What’s most important is whether or not you can tolerate it. You don’t have to love it, but can you tolerate it? Having a healthy work-life balance can greatly assist in one’s productivity and tolerance of their employer.
Excellent points! Yes organizations must identify the value of various intangible benefits, and I agree, work-live balance can be very important. It’s linked to job satisfaction and reduced turnover, two very desirable outcomes. Thanks for sharing your thoughts.
Kathy
I do agree that the ROI surrounds the intangible benefits when it comes to employment. For example, job satisfaction is key because this is how organizations retain their talent. Unfortunately, trainers are not equipped with the time needed or the skill set to consider the contributions of the intangible benefits.
I do agree that the ROI surrounds the intangible benefits when it comes to employment. For example, job satisfaction is key because this is how organizations retain their talent. Unfortunately, trainers are not equipped with the time needed or the skill set to consider the contributions of the intangible benefits.
Employees need to be confident in their roles and leadership so that the ROI works for both sides (employer and employee).
One intangible benefit that may come from training and can be measured are advancement pursuits. Depending on a programs’ content, often times employees feel motivated and empowered about skills/concepts/ideas they have recently learned and have a desire to continue learning or move to a more senior role to seek more opportunities. Training can definitely be motivational and inspire others in this capacity. I think this is an area that is worthy of exploration.
M
I never knew there were such resources to test and measure the return on intangibles that result from training. This article was very helpful. I am currently taking a class on training evaluation and the Philips ROI Method is one that is mentioned. I do think that training evaluation has evolved and continues to evolve. The benefits are not always just increased profit margins
This is an extremely informative article with a wealth of information. I think some additional tangible benefits that could be measured are enhanced teamwork, reduction of conflicts within an organization, and improvement of client service and satisfaction.
This was a great article! I never realized there were actual tests that we could use to help measure intangible benefits because the very definition of intangible is unable to be touched or grasped. Well there is now a way to make the intangible tangible. This is exciting to know especially in a field where you are fighting for/promoting how training even for none sales people is very beneficial. One intangible benefit that we could add is overall company morale.
Great article.!!
In my opinion, few intangible benefits could be to have an to have an appropriate work-life balance, appropriate teamwork, understanding each other to avoid conflicts, job satisfaction, leadership. All these aspects are very common but play a major role in the organization. Also, I recommend that both the sides should be seen: the employer and the employee to avoid conflicts.
Thanks for sharing it.!
This is a great article for introducing resources and frameworks for measuring intangible benefits of training. The Mental Measurements Yearbook provides an array of validated tests that would make it possible for trainers to capture some of this data. As the post underscores, “The key to making intangibles more tangible lies in how we measure them.” The database can help us to maintain validity and reliability in the measurements. The list of intangible benefits provided is comprehensive, and some other comments have highlighted other useful categories (such as overall morale and work/life balance). I also considered recent studies on the role of neuroscience in training and development and how learning is impacted by stressors, changes, and stimuli in the workplace. It seems that some of the intangible benefits measurements might overlap with some of the recently published neuroscience analyses. An overview is published here: https://www.td.org/insights/why-learning-neuroscience-matters
This has been an extremely helpful read to learn about the Mental Measurement Yearbook to have a tool that I can access to gain access to ways to evaluate intangible constructs. In change management, it is helpful to be able to reference intangible and tangible measures, because change management, up until this point has been one of those things that you know the benefit of it, when it’s not there, but it can be hard to measure the benefits of change management being applied. Two additional intangible constructs that I can think of are morale and employee engagement. With morale, it can refer to how employees feel about the work that they do, the environment that they do their work in and the people that they do the work with. OD programs that are focused on connecting employees day to day work to higher objectives that align to personal value sets can benefit morale. Also, OD programs that focus on creating effective working relationships amongst team members can benefit the employee perception of relationships and the overall work environment. Employee engagement can refer to how employees proactively contribute to the goals of the organization through the intentionality of their work – if they consistently and proactively produce high quality work and ideas and innovation to contribute to the goals of the organization. OD programs like recognizing high performance and innovation through incentives can contribute to a lift in employee engagement.
Pingback: Group Sales Training - everything you need to know - Klozers
Wow! The Mental Measuring Yearbook is fascinating and an awesome tool. I did not know such a resource existed to measure intangible ROI. I started thinking how useful this tool would be when measuring employee morale during change transitions in OD. We know that change transitions incite fear of the unknown and uncertainty for employees and has the potential to decrease morale which in turn, decreases productivity and organizational commitment. An OD practitioner/change management team can administer tests before, during and after the change initiative to measure employee morale and make suggestions on change interventions to keep morale up. This technique would have been useful when I worked for a non-profit who brought in a new CEO. The organizational culture with the former CEO was positive and folks enjoyed their roles. OD practitioner/change management team was not initiated nor any change interventions implemented to reduce the decrease in morale. The transitional time was chaotic, produced harmful rumors and the organizational culture/morale suffered because of it. I am going to utilize this resource in the future when measuring intangible benefits. What a great resource!
My biggest takeaway from the article was that through good validation test, that are valid and reliable, you can measure intangibles. The mental measurements yearbook was an interesting introduction to some of the tools available for validation tests. I think some intangible benefits that should be measured in trainings is motivation, looking at the different components of the ARCS model for example. Attention, Relevance, Confidence, and Satisfaction. These are some of the basic things to look at when assessing issues in learning transfer.