Training Evaluation Series Part 3: Measuring Intangible Benefits

When considering the word “intangible” terms like ethereal, ghostly, spectral, unearthly and supernatural come to mind. Not exactly how we think of the efficient and grounded process of calculating the return on investment (ROI) of training.

In the Phillips ROI Methodology (see diagram below), intangible benefits are those that not readily converted to a monetary value; outcomes that are difficult to value. Examples of intangible benefits include:

  • Job satisfaction
  • Self-efficacy or confidence
  • Leadership
  • Lower Stress
  • Commitment
  • Citizenship

Although the ROI calculation usually limits the inclusion of intangible benefits, going for only “hard” numbers like profit, turnover, and errors, they represent many the most difficult challenges faced by organizations today. Can we accurately calculate the value of training without considering the critical contributions of intangibles like improved satisfaction, commitment, and communication?

The key to making intangibles more tangible lies in how we measure them. Since most trainers have neither the time or the skill-set to embark on an extensive validation process for each “construct” they measure, the most timely and accurate way to measure intangibles is through the use of validated tests. When developing tests, researchers carefully analyze each question ensuring that it adds unique value and that it measures what it is supposed to measure (validity) accurately over time (reliability).

Locating valid and reliable tests requires more than Googling terms. The simplest way to begin your search is to use an online database that compiles published tests. Here are instructions that will get you started:

First, identify the construct you want to measure; what intangible benefit would you expect your training or intervention to reap?

Once you have a construct or outcome in mind, begin your search by visiting the “Mental Measurements Yearbook” (instructions on how to access this database at the end of the article) available free and fully online through the RU Library.

Enter search terms and locate instruments that both measure the outcomes you have identified and do so in the correct population group (adults, workplace).

Evaluate the instruments by considering information about reliability and validity.

For example, f you were to conduct a training intervention around stress management and wanted to measure results, you might enter the terms stress and burnout and locate the Maslach Burnout Inventory (MBI) which you may purchase through Mind Garden. Or if you were delivering training or an intervention focused on teams, you might use the OTCI to measure orientation toward team culture. While the MBI is on of the most researched measures of stress and burnout the OTCI, on the other hand, demonstrates reliability in adult work-based populations but is lacking in proven validity, so use this instrument with caution.

In this brief article, you’ve learned about intangible benefits and how to locate and select them. Stay tuned for the next article in this series to find out more about measuring intangibles with test resources in research publications.

Can you name additional intangible benefits that might arise from training and should be measured?

Accessing the Mental Measurements Yearbook:

Log in to the RU Library. See instructions on how to do this from off campus

Go to the RU Library Home Page and click on the link to  Databases 

Enter the search term “mental measurements yearbook” and click on the link to the database

Enter appropriate search terms and locate information about tests. Note the appropriate age and setting (adult, workplace versus children, educational setting) and information about reliability and validity.

Advertisements

18 comments

  • One intangible benefit that may come from training and can be measured are advancement pursuits. Depending on a programs’ content, often times employees feel motivated and empowered about skills/concepts/ideas they have recently learned and have a desire to continue learning or move to a more senior role to seek more opportunities. Training can definitely be motivational and inspire others in this capacity. I think this is an area that is worthy of exploration.
    M

  • I do agree that the ROI surrounds the intangible benefits when it comes to employment. For example, job satisfaction is key because this is how organizations retain their talent. Unfortunately, trainers are not equipped with the time needed or the skill set to consider the contributions of the intangible benefits.
    Employees need to be confident in their roles and leadership so that the ROI works for both sides (employer and employee).

  • I do agree that the ROI surrounds the intangible benefits when it comes to employment. For example, job satisfaction is key because this is how organizations retain their talent. Unfortunately, trainers are not equipped with the time needed or the skill set to consider the contributions of the intangible benefits.

  • I found this article very helpful as I was struggling to conceptualize how we quantify these intangible benefits. It seems a rather difficult and illusive task especially considering there doesn’t seem to be any standardized or universal accepted values to associate with these things. I guess then it is up to the company or organization to assign their own weighted value to what they consider to be the most important of intangible benefits. Reflecting on previous experience, when I think about intangible benefits I think of work-life balance. I learned all too well in my last position how valuable work-life balance is. It is in my mind what gives you strength to tolerate your job. I think this also addresses another point people often misinterpret or overlook. Sometimes their expectations about a job are a little out of touch with reality. Everyone craves and wishes for that perfect dream job that realizes all their hopes/dreams and life satisfaction. Unfortunately, this is a myth. The reality is you can find happiness and fulfillment in life even with a job you don’t love. What’s most important is whether or not you can tolerate it. You don’t have to love it, but can you tolerate it? Having a healthy work-life balance can greatly assist in one’s productivity and tolerance of their employer.

    • Excellent points! Yes organizations must identify the value of various intangible benefits, and I agree, work-live balance can be very important. It’s linked to job satisfaction and reduced turnover, two very desirable outcomes. Thanks for sharing your thoughts.

      Kathy

  • Oftentimes, many organizations equip their employees with the skills and tools to help them master their roles and responsibilities in a more practical way. What is sometimes lost or not measured in training spaces is the level of motivation associated with participants after learning. How do we measure of people actually want to apply the knowledge and feel excited about the new behavior. I think motivation is the area where practitioners need to hone in on how it impacts application of learned skills or content. It goes back to ensuring that participants in training programs are fully bought in to the process. By measuring motivation, we are sure to find the gaps in results and desire when it comes to evaluation.

  • Great article I love the resources about the intangible benefits and how to apply them especially when they give the great details of training that may take place. It also shows you how to measure intangible ways of accessing skills and how to apply the measuring outcome of it as well.

  • An intangible benefit that comes to mind is an increase in confidence. While participants leave a training equipped with a better understanding or new skill, we can’t really measure the confidence gained. And not really the confidence to perform but rather knowing a resource is there should you have issues in that particular area.

  • Very helpful and insightful article! When I graduated from Roosevelt University in 2008 the best advice from our then president, Charles Middleton, was to “read and don’t stop reading.” An intangible asset to training in a corporate setting is to gain feedback from the front line employees who are the faces and voices of the organization. Some feedback from these employees can lead to tangible and measurable results and may not be easily discovered otherwise.

  • Good information! I love the resources shared in this article. The information provided in this article helps developers measure the intangible benefits that one may question. You’re right, we have to go beyond simply googling results for targeted subjects. That is why I find great pleasure in the resources here and obtaining tangible benefits.

  • Very interesting article. Including intangible benefits seems like a win-win situation in my mind. If something has the ability to affect, positively or negatively, a program it would seem that it makes logical sense to include it in the evaluation process. The fact that there are resources already out there, which I knew there were tests but not that they could be utilized for this is a plus. Not a lot of additional resources, time and energy would need to be used in order to incorporate it into existing evaluation methods.

  • A few additional intangibles which are equally important to recognize and measure that can have an indirect or direct impact and influence within workplace performance are internal or external cultural differences such as language barriers, socio or economic (positive or negative) influences, learning challenges or disabilities. I believe these additional intangibles link to direct behaviors, morale, turnover, and corporate culture often associated when measuring intangible benefits in the workplace. They often go unnoticed even by the most seasoned professionals because it’s out of sight/out of mind from what’s typically evaluated in the workplace. There may also be a direct correlation of intangible benefits such as job satisfaction, self-efficacy or confidence, leadership, lower stress, commitment, and citizenship. Additionally, can these intangible benefits have an indirect association linked to short or long term results of tangibles often evaluated in the workplace such as training, productivity, and work redundancies?

  • Donna R. Guy

    This is a very good article! I’ve often wondered about how to measure the soft or intangible skills in the workplace. Especially, work/life balance, job satisfaction, improving productivity and morale within the workplace. Mastering this will allow companies to reduce staff turnover and increase productivity.
    I am a firm believer that happy employees produce more, take less days off and are more eager to go the extra mile to see a product or project completed. I believe a good way to start the measuring process of intangible skills you should get the opinions of those in question perhaps with the use of questionnaire’s or a suggestion box. Either way, the opinions and suggestions provided is a good start to measuring and gaining the necessary data needed.

  • Here you illustrate important concepts about turning soft data into hard numbers. When designing performance or workplace interventions, practitioners are certainly hopeful to produce a number of intangible benefits. Things like, “increased job satisfaction” or “improved teamwork” are commonplace. When it comes to an ROI evaluation, monetizing these intangibles significantly improves your chances for positive ROI. Thank you for the new resources!

  • This is very helpful! I have helped people find such tests, but I never thought to use them for my own training work. The benefit there would be that skilled researchers have created those tests, and by the time they get published in Mental Measurements they’ve also probably been used multiple times so hopefully any issues with the tests would have been resolved by then. Of course, the test itself is no substitute for a solid understanding of how to administer tests like this and conduct rigorous studies, but I can see how they’d be a great use. Thanks for the idea!

  • Great article! In one of my current courses we are reviewing the ROI model and there is no question that the focus in on monetary measures. I often questioned some of the same intangible benefits listed in this article to truly measure validity. Additional intangible benefits that come to mind for me are work/life balance, chance for growth or mastery, and company culture. I’m looking forward to seeing the intangible benefit examples that others post.

Please post a comment

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s