Training Evaluation Series Part 4: Measurement Tools from Scholarly Publications
Last week, in the post, “Measuring Intangible Benefits,” you were introduced to strategies to locate tests that allow you to quantify the intangible outcomes of training by using the Mental Measurements Yearbook.
As you will find, the “Yearbook” reviews commercially available tests, but there are many other tests in published research studies.
The best database available to locate tests is PsychTests, which is relatively new and unfortunately not yet available in the RU Library system. But there is a workaround that allows you to locate tests without this resource. It’s a bit like prospecting for gold–you have to sift through a great deal of rubble before hitting pay dirt–but it’s worth the time and trouble to locate valid and reliable instruments to measure the intangible benefits of training.
Follow the instructions in last week’s article to log into the RU library system. Then, as before, go to the Database section. This time, choose Google Scholar, By first logging into and going through the RU library system you will have access to full-text articles in Google Scholar beyond those available to the general public.
In the search box, enter the construct you want to measure and add additional search terms like “instrument,” or “measure,” or “validation,” or “scale” and hit search. For example, say you are delivering a program or intervention designed to increase service to internal customers (or coworkers) within an organization, and you want to measure changes in the attitude of your trainees. One “intangible” benefit or outcome might be increased organizational citizenship, which relates to willingness to go above and beyond job expectations. When I enter the terms organizational citizenship and instrument, I see a lengthy list of articles related to this topic. In the list, I select the article titled “Organizational citizenship behavior: Construct redefinition, measurement, and validation” because it has the terms measurement and validation, which should yield actual test questions. I click on the PDF link to the right and see the entire article. As I scroll through the article, I hit pay-dirt when I come to Table 3 (Page 781) and find a series of questions measuring various aspects of citizenship. I probably wouldn’t use all the questions to measure citizenship but will choose those most appropriate to my audience and purpose. Rarely am I so lucky to find a rich resource like this one so quickly, but often have to enter a variety of search terms and sift through the results.
As you probably realize, a critical step in this entire process is the identification of the intangible benefits that you want to measure that are logical outcomes of your training program. Here are some possibilities:
Leader-Member Exchange: a brief measure to assess relationships between workers and their supervisors after leadership or supervisory skills training.
Job Satisfaction: a measure to assess overall satisfaction with the workplace after a change initiative, reorganization, or merger.
Workplace Thriving: an attitude of feeling energized and alive (vitality) and a sense of continually improving and getting better at what one does (learning) that might be used following a large-scale training initiative or a lengthy new hire training program.
Take the challenge. Identify a training program that you might deliver in your current or past organization, come up with one or more possible intangible benefits, and use Google Scholar (by way of the RU Library) to locate potential instruments that you might use to measure your intangible. Most importantly, share your findings with us in the comment box.