Spending Money on Learning and Development - Measuring ROI

Desikamani Gopaladesikan had posted the following question on LinkedIn (http://www.linkedin.com/groups?home=&gid=27003&trk=anet_ug_hm&goback=%2Eana_27003_1247990854435_3_1).

Does spending money on L+D really add tangible value to an organization? Or is it like prayer, spend and hope for some benefits to accrue...? Over the last fifteen years in the learning services business, selling and delivering training services I observe that it is very difficult to make CEOs perceive L+D spends as critical. They look at it charitably. I wonder if this will ever achieve a mission critical item status in organizational spending.

The note mentioned below were the various response. The names of the responders have been mentioned therein as well. I have put only the relevant ones... those which take the discussion ahead fruitfully.

________________________________________________

John Bower

Investing on the TLD of employees is no different than spending money on an IT system. If training is expensive try employee ignorance and see what that can cost the company. The key here is to ensure that all employee TLD is timely and effective with some measurable results that can be communicated back to the CEOs. I have been a TLD expert for the past 25 years and found it necessary to articulate to top management, the benefits of training and how this can impact the business success.

Mike Andrew

CEOs do look at training "charitably" as you described. I can't blame them especially with all the off-the-shelf programs and consulting firms who speak too much the HR language while not possessing the necessary insight and making an effort to understand the business. Training is often viewed as a nice thing to do, and it does not hurt the business. It is simply not core to the business. Now, on the other hand, if the learning and developing group puts in the rigor to understand the business, the challenges and opportunities facing the business, understand the strategic imperatives, etc. they are then in a position to put in the same rigor to create learning that is relevant to the business and practical to those inside the company to begin applying. Finally, it has to be more than hoping. Working with the same rigor to determine ways to measure impact or affect on the business is critical. The company may not ask for it, but the learning group would be wise to make an effort to determine the impact.

Rebecca Everett

Read my article, published in TandD magazine in February, for some specifics into what it takes only to get ROI and to thoughtfully plan an initiative so you can actually measure it. https://www.box.net/shared/f5mmi7xee0

Jennifer Good

Too many people think of training as a one day training course aimed at changing behavior. Training courses are only a small part of the training and development function. If top management does not believe in the usefulness of the training, improvement is unlikely. Coaching, performance goals and measures, on-line follow up, on the job training activities, stretch assignments etc. must all be incorporated into the training plans if goals are to really be achieved.

Manju M N

We can think back to the pre civilization era human race had the raw material to set a fire. With changes, innovations and constant learning now the same human race can use all the modern gadgets which can ignite in seconds. All this is the result of consistent and constant learning and development. Hence it is needless to think that any business can grow without spending on learning and development.It is an integral part of growth .With growth comes profit and motivation to go further.Kudos ! to CEOs who consider learning and development as the core support for any organization aiming growth and expansion.

Rhonda Askeland

The points reinforce the need for L+D to create/sustain a business alliance with the organization's leaders in order to understand the needs of the business and position training as an aspect of getting business results. L+D needs to stop seeing training as an event and start seeing it as part of a complete experience that is tied to how an organization achieves its goals. If L+D wants the "seat at the table" then they have to be a partner to the business and that means talking the language of the business and not "HR/OD speak" as Mike said previously and demonstrating results that are tied to the business.

Gopal Bhutani

After two decades in sales / marketing, I have chosen to switch to the formal TLD domain as a service provider. The point raised by Desikamani and responses by learned fellow members give an insight to the as-is state of the domain and also fuel my thought process. Thanks. With the above intro, and excuse me if am being simplistic, I place my views...

> There is insufficient meaningful accounting of "human capital" in most of the enterprises - on a YoY basis as a "critical metric" - it is not a balance sheet item

> Most of the L+D initiatives, if executed in right earnest, are long-gestation investments, going beyond the annual perspective - so it takes a leadership which has the necessary conviction, perspective and self-tenure security to commit for meaningful L+D initiatives.

> I re-call a half-serious joke by one of the trainer - "an expert is a person who knows 10% more than you". May be the L+D professional pool is not really as great... and that inhibits the leadership in enterprises to commit on "suspect" L+D activities.

> The risk of investment in L+D activities leaking - the employees moving out, leaving the enterprise high and dry

> The L+D service providers not articulating the RoI as effectively - yes we swear by coaching on RoI concepts

> One possible solution is that the service providers ( consultants, trainers, coaches ) adopt partnership model/s - linking the gains of service provider and the enterprise.... a kind of a shareholding.

ME

The question is darned good! While I believe in what Jennifer says, the most important is John Bower's response. Let's look at it this way... (that's the way how I look at it... its a bit long!!!)

Basic Premise...

All humans have common-sense, and most of the common-sensed executives are mostly bothered about two things, a) their growth, and b) the perception that people hold of them. That goes for you and me as well, whether we are in the L+D business or business of sales or wherever. But, more importantly, we in L+D should understand that.

We are enablers, and that goes for both businesses and people, which includes our customer's personal expectations. If we focus on that, we would be able to logically create solutions that impact the above, we would in turn influence the aboe mentioned points a and b for ourselves. Most of the time most of L+D executives forget that, and focus on just a and b for themselves (which includes the Paco in '02-'03). To top it up most of L+D executives are not fully geared to handle L+D, esp., in India. This assumption holds good for the entire world.

And if I were to write on the mistakes which most L+D executives make, I could write a 50K-word dissertation. Let's not focus there.

Let me share something that we have done.

The Environment...
Our organisation is an exceptional organisation, and we decided to use simple methods to 'make the most important person in our company successful' - the Relationship Manager (RM) on the field, and when that happens, we knew would have impacted the topline, and as we move forward, the bottomline as well.

The Problem...
We are in such a business where sales is accrued from our distributors advising/persuading investors to invest in our Mutual Fund schemes. Our RMs faced a huge challenge whilst addressing the 'capital-market fallout' related queries that our investors had for our distributors (I'm putting it mildly... very mildly). To top it up there was competition from other Mutual Fund schemes, and many of our distributors not being able to manage their investors expectations in terms of advisory services... and many more. We realised that the important aspect is that our distributor needs to be engaged and supported to be able to in turn support their customers, and while we do that we need to focus on sales. So, we devised a plan

The Solution...
We decided that we would have a 'focus-of-the-month', and then we would go on a formal blitz of formal/informal (including huddles) training sessions with our distributors across the country on the focused product, and these training sessions would be conducted by our RMs; but our RMs were not trained facilitators, but we tackled that by smart 'ID'ing. This impacted their engagement levels, build conviction in themselves, and also made them percieved as demi-specialists in front of the distributors. The distributor also gets to interact on a formalised framework. But this doesn't end here; we are piloting a mobile-based learning engagement model, which would keep the distributors engaged (and be reminded that 'we are always there') when our RM is not in front of the distributor. In parallel, we have launched a drive to enable our distributors with a suite of skills such advisory skills, influencing skills, assertiveness, negotiation etc. So, whilst the earlier helps build knowledge, the latter helps build the skills to apply the knowledge, and these in turn impact the behavior that the distributors demonstrate in front of their customers which help them build a positive perception about themselves in the eyes of their customers and hence impact their growth. When the distributor grows, it has a cascading impact on the perception that our RM carries about a) oneself, b) scope of business development activities, and c) the capabilities of the L+D function. This has already yielded results

The Inference
But, does that mean we have stopped doing some (as I say) hygiene activities, which many of my industry colleagues seem to be doing most of the times; well, no! I am OK with the miniscule proportion; as we focus on what makes the difference, we don't have prove the worthiness of the function. WE do come across challenges, but they are either personality-based challenges or procedural. The former can be influenced, but one needs patience, and the success is still not guaranteed. The latter can be handled by working on it together
By maintaining this balance, I know we have been a) a tool for success, and b) have been able to add value to people

What's In It For Me
This enables me to face the man in the mirror without being labelled a moron by the man on the other side

As Rhonda mentions about the 'seat on the table' can only be gotten by adding value to business, and speak the only language that makes sense; that of making the enterprise successful. There is no point being a fountain of knowledge, if I am disconnected from reality... that's the enterprise's existence and growth and the perception that the shareholders/stakeholders hold.

Germaine Broadbent

Money spent on L+D should be focused, managed and measured and should relate to the bottom line and the brand of the company. Some measures are easier than others - are those concerned now able to do/produce something different that will make money? Sometimes what is contributed is difficult to measure and has to be discussed, valued and agreed to add to the integrity and long term growth of a business. For example, many Cactus clients are learning a language for work-related reasons. Some may have to be able to actually communicate messages regarding invoicing and / or delivery via telephone or email. However, some will be learning or improving their language skills in order to create better business dynamics or stronger business relationships. They already make the deals / have the business / speak the language but they view this form of L+D to have a value.

Peggy Jones

To Mike's point that some of the training program proposals make no effort in "understanding the business", I feel that the source of making a training program add tangible value to a business is to build it on a needs assesment of the organization. And to continually make a training program better an organization as a whole, a complete evaluation system will have to be put into place. Just as in the typical ADDIE model, analyzing is the first step and evaluating is the last, which makes the process come full circle.

When you create a Needs Assesment report of the organization and word it correctly, I feel that Chief Officers and senior management have much more of an understanding of how a particular training program will contribute to their company. This is especially true if the assesment includes a cost analysis. When you break down the costs of a training program and estimate a ROI, the senior management has much more of an inclination to give the okay

As so many people have said before, training is generally the first to go in a downward spiraling economy. Now the new buzzword is ROI in order to enable training professionals to add some worth behind their product. I am in complete support of this. If a training program does not have a postive and significant ROI then the program will not be carried out. In the past too many training programs have been based on industry fads and what the organization THINKS they should do rather than cold-hard numbers. Now is the time to reevaluate L+D's zeitgeist and direct it toward true tangible added value specific to each organization in order for training to survive.

Rhonda Askeland

Peggy, you are absolutely right that a needs assessment that looks at the organizational needs and ties the learning to the org. needs is essential. Too many times, L+D feels pressured to just "do something" and they don't build the value that a needs assessment provides into the process. It's a "we needed this yesterday, so just give us what we tell you we need" process and the patient doesn't know the root causes to sickness or problem they're having

If L+D is reactive, then senior leaders don't see the value to the business and the credibility of L+D is diminished. ROI is the new buzzword (with a lot of people at all levels in an organization who don't know what it means or how to calculate it) and that's part of talking the language of the business I mentioned in an earlier post. L+D needs to know what it means, how to calculate it, and how to present it so that organizations take L+D more seriously and so that organizations have sustainable results that L+D helps them achieve.

Peggy Jones

Rhonda, I completely agree that L+D professionals need to know what ROI is, how to calculate it, and how to present it... I also think it is very much easier said than done. My first project at my current organization was to conduct a cost analysis on our orientation program and present a ROI report. The process was so out of my league that we ended the project with a cost analysis and recommendations on how to cut costs on the program. No where near a full ROI report which would take so much more time and effort than we had

On that note, do you have any suggestions for figuring the ROI of training programs? After the initial cost analysis of the program how would you figure the ROI?

Charles Jennings

Whether ROI or ROE (Return on Expectation) - which is probably a better approach to demonstrate business value - the REAL value a TLD professional can bring is knowing when training won't add value. i.e. knowing when NOT to train. There's a blog posting of mine about this here http://www.linkedin.com/redirect?url=http%3A%2F%2Fis%2Egd%2F1smm7&urlhash=njDF&_t=tracking_disc or links on my website. http://www.duntroon.com/

Robert Bilotti

Hard to believe we are still talking about this... I would make one suggestion.... stop giving your people lousy, irrelevant, ineffective, overpriced, too long, poorly produced training.

Charles Jennings

Every year $ billions continue to be wasted on training that has absolutely no impact on individuals or organizations whatsoever. Yet managers continue to demand training when the business problems they want to address can't possibly be addressed by training, and internal training and development departments (and external training vendors, no less) continue to provide it, because 'that's what they do for a living'

If only we could get rid of the ubiquitous Training Needs Analysis we'd be part way there. TNAs assume that training is a solution or, at least, part of a solution even before the root cause of the problem is identified.

TNAs often back Training and Development departments or managers who don't know any better into a corner and force them to use the wrong medicine to fix the patient.

I refer to this situation as the 'Conspiracy of Convenience' because that what is often is

THE CONSPIRACY OF CONVENIENCE
> Business managers demand 'training' because they have a 'gut feeling' it can help fix a performance/business problem
> No-one does any real root-cause performance analysis to understand what's actually causing the problem (incidentally, most problems caused by inadequate performance are not due to lack of knowledge or skills but to external 'environmental' factors such as process that don't work, lack of motivation - read poor management -, poor infrastructure or a myriad of other factors)
> Training managers or training vendors design, develop and deliver training in response to the business managers request/demand
> No-one measures to determine if the training has any impact on individual or organizational performance
> Nothing happens. Everyone's happy

and, worse still, everyone feels that they've done their jobs. Managers have provided their employees with the opportunity for training, and Training Managers have delivered training. After all, isn't that what training managers should be doing?

Rhonda Askeland

Well said, Charles. Training is one solution and not the only solution. Until L+D people are adept at HPI and look at all of the root causes, solutions will continue to comply to what the business leaders demand and not meet the true needs.

How many times have you done a performance analysis to find that process, structure, and/or motivation/incentives and consequences (the environmental factors that Charles called out) are the culprits to poor performance and the business problem? I have to say this is most of the time. Not to say that there aren't training needs - they just aren't first (or second or third) on the list

Less focus on training as the only solution and more focus on the process/systems that are getting in the way.

John Bower

I agree with you Rhonda and Charles, I have been leading organizational TLD for 20 years in five different countries for medium to very large organizations

The biggest problem here in north america is that people who know little about TLD are managing the internal trainers and negatively influencing them. Or let me put another way they are creating barriers to their own direct reports performance

Hence a while ago I said trainers and vendors have become order takers rather than consultants for fear of being panel beaten by their superiors. Too often the easy way out is to please the boss or training requester

Another issue is not to use the terminology of TNA but change to Needs Analysis, and yes in my experience many of the requests for training were solved through implementing a simple employee procedure, or suggesting the supervisor or department head smarten up and do his/her job of having regular performance discussions where applicable

Too often especially in north america the training department has become the sacrifical lamb and is blamed for many problems while in reality the directors or managers leading the training department are the weak link and need to learn to trust and stretch the TLD expert to do his/her job according to the need and research. After all why did the organization employ a TLD expert if everyone else is trying to perform this role?

Rhonda Askeland

I have some empathy, but overall, I think that TLD has brought on their problems for a number of reasons. When I facilitate sessions on learning development, my challenge to the groups is always to think like a business person and provide business value by creating partnerships, asking a lot of questions and getting to the root causes so that they can garner more respect and credibility.

Sangeeth Ibrahim

I think we can change the way they think by measuring customer perception and internal indicators regularly. I head the TandD initiatives of a bank in the middle east. Three times a year we measure customer perception, speed of service, errors, attendance, 360 degree feedback. Once a dashboard is created from this data, we conduct customised Trainings for branches and one to one coaching for associates performing below targets. We also reward branches and associates going above targets.Three times in a year, we measure and three times we conduct trainings. In 1.5 years we have seen dramatic reduction in errors, increase in accuracy and increase in customer service. If we focus on making a real difference and if we measure this difference, then we can make a difference in the way people see our service. Now our CEO doesnt see it as a cost, he sees it as an Investment.

Rebecca Everett

Measure what's important and address those areas. Measurable increases in performance are all the C SUITE people are willing to entertain as relevant.

ME

Measurement is the reason why most training professionals stay away from measurement... mostly because they don't know; and all those who know a little (most of the others), start talking about ROI without even understanding what it is. Customer perception is a reflection of whether the customer-group liked or not, which could be a reflection of whether one's work impacted performances or was it that the trainer was a good public-relations guy. Having said that its good for getting the money invested.

As Rebecca mentioned it is right that the C-Suite undoubtedly recognises measureables, but the problem goes back to the fact that most don't know what to measure, when to measure, and how to measure. It basically stems from a deficiency of understanding. Because of this deficiency, most trainers jump to finding out what it means, and as they stumble upon Kirkpatrick, they forget about the initial stages and jump to ROI. It is interesting that most of us don't know the difference between Business Impact (Level4) and RoI (Level 5).

Once a young apprentice from HR came over to get feedback on HR processes. Ignorance was profound in her, which is OK; interestingly, she had taken some guidance from supposedly seasoned HR professionals. The questionnaire had questions which gave an idea that after training if one does regular reviews one can find out about the effectiveness. The apprentice had to go through a harrowing time with me!!! But, what about the seasoned-professional... oblivious to the truth and contented with ignorance. That's the case with most.

So, according to this HR colleague of mine, let's say, I have been trained on sales, and after my training is over regular reviews with my supervisor/me should give me an idea about effectiveness. My question is training the only impactor of performance, or are there others? So, if there are other things such as supervisory support or changed market conditions or improved product design and performance, which have helped me improve, then shouldn't YOU, my sales-trainer, share the spoils. Similarly, in a downturn of performance, one needs to examine the impact of the aforementioned factors (and many more).

To figure out the ROI, the best way to do is to conduct a controlled group exercise wherein one group is trained, and the other group is not. All the other variables are kept constant. The difference (PD) shows up in the performance, and that can be co-related to the cost of training (CT) for each of the control-group participants (CG). The coefficient [(PD/CT)/CG] thus arrived at could be used to invite investment.

ROI per Participant = [(PD/CT)/CG]
Investment = RoI per participant X Total number of sales people to be impacted

The methodology of calculation will be different for other forms of intervention, but the basis mentioned above holds good with most cases that I have come accross. I am yet to get exposed to all scenarios, and hence can't call this hypothesis a theory.

But, I have one theory to support this... Take credit when you know you are the contributing factor, and not whenever it comes. Take only that much what is rightfully yours. If you take more credit than what you deserve, then when the downturn happens, as Rhonda said, one would become the sacrificial lamb.

Rebecca Everett

RE: Pratapaditya's comment, CEOs..."don't know what to measure, when to measure, and how to measure
It is incumbent upon L+D professionals to decide these things and design learning interventions that get desired results. L+D professionals have to let go of the expectation that a C-level person is going to tell them what to measure. Take the initiative to choose interventions that will meet business needs as directly as possible and design them so they can both get those results and prove it by the metrics.

ME

That points out to my basic outcry... L+D professionals need to not only understand the science and art of L+D, but also be able to communicate the same in respect to organisational requirements.

Volney Spalding

I agree with the conversation regarding measuring performance and comparing against a norm or control group. In sales productivity there are many metrics to track: sales activity, pipeline generation, revenue, win rates, close rates, lead conversion rates, etc. As it turns out, at Salesforce.com of course we can measure all of these metrics since they are tracked in our app. We measure all of them and report on progress against a new hire baseline.

But how do we measure non-sales related activity and measures? If you are rolling out a Management development or Leadership development course, what are the day to day measures and activities you can track? Leadership development would appear to involve some highly abstract measures: Ability to convey vision, ability to correctly coach and guide employees relative to their experience, ability to motivate managers during down economic cycles, ability reduce complex variables to a simple plan, etc. What are the measures and where are they stored?

I accept Rebecca's challenge that we need to not wait for C-level to tell us what to measure, but am interested in best practices from others.

I would be very interested to know how others have achieved meaningful Level 4 and 5 evaluation for non sales related activities.

Srinivas Prakash

There are measurable and non measurable qualities. The representation of the subjectiveness involved objectively is of course one of the most difficult tasks. In my experience, especially in the service industry, I have dealt with level 4(business impact) which could be measured objectively and precisely. For example the improvement in the customer satisfaction scores (can be measured) indicates the overall improvement in quality (is subjective). But return on investment with regards to quality is often subjective though it can increase the customer base. I think training evaluation and measurement makes sense till level 4 in business. Level 4 has a direct impact on level 5 which CEO’s understand, however the strategic linking of these 2 levels needs to be objective and precise. In this case an improvement in quality can or cannot increase sales. However it is for the CEO to see the result at level 4 and develop an effective stratagy at level 5. If L+D tries to show ROI then it would be subjective and would not produce accurate and predictable results. Measuring physiological and behavioural aspects and their impact at level 5 is something I have never pitched for. Please provide inputs.

ME

Measuring at Levels 4 and 5 are a rather difficult proposition, but not impossible. The difficulty comes from the ability to point out the parameters of evaluation and the cycle time.

Let's take the example of leadership development. We may go ahead and conduct a training program on Situational Leadership, but its difficult to identify and measure its contribution in the entire 'leadership' trait-set, as there would be many a affecting factors.

So in case of measuring the development of complicated / higher levels of skills, one should look at the entire learning plan rather than just one singular program.

Secondly, the time taken for any of the attributing traits to bloom and be demonstrated so as to be measured at either 4 or 5 is very long, hence usually not taken in context.

However, the 3rd level needs to be done... changes in behaviour. I hope I am able to respond to your query. Incase you wish to discuss my experience on this, do respond back.

In actual scenarios, training managers agree to measuring levels 4 and/or 5 without understanding this... and as Rebecca mentioned, the onus lies on the L+D professional to articulate the measurement and its components to the stakeholders beforehand, rather than being reactive and defensive later on... and finally commit something completely not-do-able in haste.