Jane and I talked about proving the ROI of a mythical (I think) training course called “Ethics for Firefighters”. She said to prove the ROI of such a course that first the dollar value of the public perception of ethical behavior needed to be determined. I disagreed with this, asking if the real measurement was public perception, a decrease in legal actions (both the action and the cost of defending/prosecuting the action)?
Then I think Jane thought that meant I was turning into a cold corporate numbers person because she asked “So ROI for ethical behavior of public servants is based entirely on legal costs?” And that isn’t where I was going at all.
This exchange made me think that maybe there are translation problems between the terms used in Performance Improvement and the terms used by MBAs.
Lost in Translation
I’m not an MBA. But most of the executives I know are. They use MBA terms to manage their people and environments. In one more semester I’ll have an MS in Instructional Systems. I have been trained to look at organizations as systems, and to troubleshoot what is causing performance problems in these systems. I have been taught a completely different set of terms than the terms the MBAs I work for use and understand.
This could be a problem.
MBAs love to talk about ROI. ROI literally means “return on investment”. Now, just like there are good IDs and bad IDs, there are good MBAs and bad MBAs. The good MBAs also see the organization as a system. The bad MBAs only care about quarter-to-quarter hard numbers. Unfortunately, we need to translate performance terms for good and bad MBAs.
Let’s define the term. If an MBA wants to correct me, please feel free! I got this definition from the business dictionary.com:
Earning power of assets measured as the ratio of the net income (profit less depreciation) to the average capital employed (or equity capital) in a firm or project. Expressed usually as a percentage, it is a measure of the profitability which (while not taking the time value of money into account) indicates whether or not a firm is using its resources in an efficient manner. For example, if the ROI of a firm (in the long run) is lower than its cost-of-capital then the firm will be better off by liquidating its assets and depositing the proceeds in a bank. Also called rate of return, or yield.
Sigh. That is why I didn’t get my MBA, that definition just seems so void of human concern. My gut feeling is this is where the translation issues start to happen. So what would be comparable from a systems (or human performance technology) point of view?
The first letter of ADDIE stands for Analysis. In Pershing’s Handbook of Human Performance Technology, there is a chapter written by Dr. Allison Rossett titled Analysis and More. Rosset says “analysis is the process used to figure out what to do”. She provides a this view of what analysis is:
Organizations provide the results of their learning analytics for many reasons. The most popular reason is to showcase the training’s value to the organization. Another common reason is to indicate the quality of the training services provided. Additional reasons are because stakeholders request it or to justify large expenditures.
The way I see it the analysis step, the “A” that is the key repeatable step in every instructional design theory, is central to the MBA concept of ROI. We’re all trying to get at the same thing. MBAs want to prove their organization is using their assets in the most efficient way possible. The problem is that more and more in this age of information, those “assets” are human.
This is where we as performance and learning specialists can come to the aid of our MBA friends. Proving that your human assets, the information workers that hold the means of production inside their heads, are being utilized the best way possible is very different than proving your datacenter or factory equipment is optimally configured.
A potential blogging ROI example
Here’s a real example of how hard this is. Our guys out in the field who install and configure EMC products for customers are very very busy. They usually have much more to do than there are hours in a day. And since they are not machines, there is just no way to configure them to miss sleep and work 24 hours a day so that they can catch up.
I’m going to concentrate specifically on blogging in this example. Our field guys are super smart, and are very quick learners. They have to be, or they would not be able to do the jobs put in front of them.
Now I’ve blogged before about blogging being an excellent way to reflect on new concepts. If I think what our field guys do, I would imagine that if they were able to blog about their activities they could reflect on lessons learned, what would have been helpful to know going into an engagement, advice they would give to colleagues in case they faced the same situations, etc. These sort of blog posts would be useful for other field guys, support guys, engineers, training folks, etc. The blogs may also give management a better idea about how much time certain types of engagements take.
The ROI question is which is better for the company: book these guys out for 100% of their time in customer sites to plow through the workload, or give them some time to blog about what they have been through in an engagement, providing them reflection time and providing insight into the experience for other information workers in the company?
Lets say I proposed this, and everyone bought into the plan. How would I prove hard dollar numbers that the time given to these field folks to blog benefits the company? I think this would take planning and analysis throughout the project lifecycle, here are some guesses on what could be measured:
- Measure time it takes to blog
- Measure traffic to particular blog posts
- Measure time it takes for individual blogging to perform similar tasks after the blog was written (after time for reflection)
- Observe case deflection in the call center (number of calls on topic before and after post is written)
- Measure engineering enhancements made because of post
- Measure training enhancements made because of post
- Measure impact on planning (actual time it takes to do the things talked about in the post vs what was originally planned, utilization rate improvements based on adjustments)
I would think creating a community of support larger than field support colleagues would also have some sort of impact, not sure on how to begin to measure that though.
I think we have a translation problem between MBAs and ID/performance support folks. ROI is just the end result of analysis. The wrong measurement will always give an incorrect ROI, which is not good for business (is there an MBA term for that?). As education folks, we understand what interventions motivate and support performance for “human assets”. We need to figure out how to communicate this to our MBA colleagues.
I think the problem for the MBAs is getting numbers to prove the ROI of the work information workers are now expected to perform. Those numbers don’t exist in many cases because the analysis is not typically done on the human impact of information work. Its an infinite loop: we don’t do analysis on that sort of thing because we don’t do it, we don’t do it because we don’t have hard numbers, and since we don’t have hard numberswe can’t prove ROI.
What do you think? Am I totally off-base here? Is it wrong to want to collaborate with MBAs to prove ROI?