Bernard_Sample_ROI_study

of 22

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
PDF
22 pages
0 downs
13 views
Share
Description
1. ROI and Coaching: Applying Metrics to Measure the Effectiveness of Coaching Programs A study by Paul Bernard (© Paul Bernard & Associations 2006) 2. Page 2 of 22…
Transcript
  • 1. ROI and Coaching: Applying Metrics to Measure the Effectiveness of Coaching Programs A study by Paul Bernard (© Paul Bernard & Associations 2006)
  • 2. Page 2 of 22 Scenario XYZ Financial Services Company pays for 400 coaching assignments per year with 60 different coaches at an average cost of $30,000 per assignment. Coaching is restricted to vice presidents and above, and it is offered to managers with a variety of developmental needs. Generally, these coachees' needs fall into four basic categories: • High-performers who suddenly experience performance challenges with new bosses and/or new assignments. • Star expatriates who are having difficulties managing the repatriation process. • Star lateral hires who are having some challenges managing the transition into the new corporate culture. • Senior technical or professional employees who are managing other professionals or technologists for the first time. A total of $12 million has been spent on coaching this year. The CFO and Purchasing Department have requested a Return On Investment (ROI) measurement for the $12 million that is spent on coaching. How should XYZ go about measuring ROI? Overview There’s been much discussion in the literature about the need to evaluate the effectiveness of executive coaching programs. Why is it important to develop effective metrics to evaluate executive coaching? There are a number of reasons: • Fiscal responsibility: a lot of money is being spent on coaching but there are no hard proofs that it works • Increasing scrutiny: tough economy, tougher decisions on spending • Some say coaching is just a trend: the need to validate coaching’s contribution to the bottom line • Need a measure to separate effective coaching from ineffective coaching The “Holy Grail” of an evaluation system for executive coaching programs is an ROI that enables senior management to effectively measure the efficacy of the coaching process and compare coaching to other corporate investments. Developing an ROI system for Coaching--the Challenges As Jack Phillips writes, developing training evaluation can be a complex process and utilizing best practices is important. In Phillip’s view, there are two key fundamentals in developing effective evaluation systems for all training programs: • Isolating the effect of the training • Quantifying both the cost and impact of the training There are special challenges in evaluating executive coaching programs because no two coaching assignments are the same and no two coaches will approach a given assignment with the same techniques. The bottom line is that, in establishing an effective evaluation system for coaching, there may be more variables and more non-hard skills that need to be evaluated than in evaluating other training programs.
  • 3. Page 3 of 22 Despite these issues, it is important to plan a coaching program that evaluates coaching by isolating its effect and quantifying its impact. The good news is that if care is taken in terms of both the assumptions and the process, the ROI measurement from coaching will be at least as good as any ROI measurement presented to the CFO for other corporate investments. The Foundation of an ROI Program Many experts believe that any evaluation of coaching, like the evaluation of all training programs, should begin with Kirkpatrick’s four level evaluation system. In Phillips’ view, ROI should be considered a fifth level of evaluation. Built into this evaluation process is the assumption that as you move up the evaluation levels, a smaller percentage of coaching programs will be evaluated. Most experts recommend that no more than 5% of all coaching programs should be formally evaluated for ROI, but a much higher percentage should be evaluated on levels one through four. Here is a grid based on the Kirkpatrick model with the recommended percentage and number of XYZ Financial Services Company’s coaching assignments to be evaluated at each level: Level Questions to be Asked How to be Measured Percentage/Number Evaluated Level One: Reactions and planned action What did the coachee think of the engagement? Ask coachee and coach. 100%/400 Level Two: Learning What did the coachee learn during the engagement? Ask coachee and coach. 100%/400 Level Three: On-the- job application What learning, skills, etc., did the coachee apply on the job Ask coachee, coach, and significant stakeholders through 360-degree surveys. 50%/200 Level Four: Business results What changes in results and productivity have been observed on the job? Ask coachee’s manager and the manager’s manager. 20%/80 Level Five: Return on investment (ROI) Did the monetary value of the results exceed the cost of training? See below. 5%/20
  • 4. Page 4 of 22 How can we isolate the effect of coaching on the person being coached? Some ideas: • Pick a coaching assignment that is in one of the major coaching categories (listed in the Scenario section on page one of this document). • Pick an assignment where the coaching length is similar to the length of most other coaching assignments, usually one year. • Pick a coaching assignment where the coaching request is a voluntary one. • Pick an assignment where the coachee is considered highly coachable (see attached Appendix A for a description of coachability). • Pick a coaching assignment where a detailed Personal Development Plan (PDP) was created. • Pick a coaching assignment where the coachee’s supervisor believes very strongly in coaching and is available to speak with the human resources department and the coach about progress and challenges in the coaching process. • Pick a coaching assignment which is at the same management level as most other coaching assignments. How do the experts suggest that you calculate the economic value of behavioral changes? What are some important markers? • Improvements in productivity • Reductions in absenteeism and employee turnover • Reductions in cycle time • Improvements in quality/reductions in waste • Increased customer satisfaction • Increased value of opportunity pipeline • Improved corporate reputation (measured in the amount of positive press coverage for the company or division versus the amount negative press coverage for the company or division) Case Studies So given the above, how can XYZ evaluate its coaching programs for ROI? Here are two cases that demonstrate how an effective ROI process might be set up. Case One Mr. Sylvester M. Managing Director, XYZ financial services firm Sylvester M. is currently Global Managing Director of Equities Trading Sales at the Institutional Investor Group (IIG) of XYZ financial services firm. After four years in London where he successfully ran Equities Sales Operations for another division of XYZ, Sylvester was repatriated to New York to run a new and important revenue area in the prime brokerage business of the firm’s New York office. Sylvester became Director of Global Sales for IIG, a profit center at XYZ with $250 million in revenue and great potential, but with a large and unwieldy sales force. After accepting the new job, Sylvester asked for a coach to help him optimize his performance in his new position.
  • 5. Page 5 of 22 One of the major challenges facing Sylvester was the fact that after four years in London, he had lost most of his political/networking connections. In his new position, Sylvester was going to be responsible for managing a group with great revenue potential but with a very unwieldy sales structure, since the sales force consisted of both institutional and retail sales people who had not worked together before. This meant that because of corporate politics, he needed a large network of XYZ colleagues to help him get things done. An additional challenge was that Sylvester’s new boss had a very different management style from his previous boss. Even more complicated was the fact that Sylvester’s new boss reported to Sylvester’s previous boss in London. Personal Development Plan (PDP) for Sylvester M. 1) Hire/manage new sales team that will increase revenue. 2) Build relationship with new boss. 3) Reduce sales costs. 4) Create and implement a five-year plan for the organization. 5) Build a personal network in New York and the rest of North America. 6) Build sales metrics to evaluate sales team. 7) Improve reputation of the division.
  • 6. Page 6 of 22 Level One: Reactions from Coachee and Coach Sample Evaluation Form for Coachees--Filled out by Sylvester and his coach Filled out by Sylvester Poor Average Excellent Effect of coaching in improving commercial effectiveness 1 2 3 4 5 Effect of coaching in improving leadership skills 1 2 3 4 5 Effect of coaching in improving reputation management skills 1 2 3 4 5 Effect of coaching in improving communication skills 1 2 3 4 5 Effect of coaching in improving strategic planning/strategic thinking skills 1 2 3 4 5 Overall Rating 4.60 Filled out by coach Poor Average Excellent Effect of coaching in improving commercial effectiveness 1 2 3 4 5 Effect of coaching in improving leadership skills 1 2 3 4 5 Effect of coaching in improving reputation management skills 1 2 3 4 5 Effect of coaching in improving communication skills 1 2 3 4 5 Effect of coaching in improving strategic planning/strategic thinking skills 1 2 3 4 5 Overall Rating 4.20
  • 7. Page 7 of 22 Level Two: Learning Achievement of Skills—Sample Forms Filled Out by Sylvester (coachee) and his coach Form filled out by Sylvester: How were your skills impacted by the coaching process? Skills Acquisition Connected to PDP Objectives—New Skills Acquired None Some Significant Hiring and sales management skills 1 2 3 4 5 Relationship-building skills 1 2 3 4 5 Cost-reduction skills 1 2 3 4 5 Strategic planning skills 1 2 3 4 5 Networking skills 1 2 3 4 5 Modeling and analytical skills 1 2 3 4 5 Reputation management skills 1 2 3 4 5 Overall Rating 4.57 Form filled out by Coach: How were coachee’s skills impacted by the coaching process? Skills Acquisition Connected to PDP Objectives—New Skills Acquired None Some Significant Hiring and sales management skills 1 2 3 4 5 Relationship-building skills 1 2 3 4 5 Cost-reduction skills 1 2 3 4 5 Strategic planning skills 1 2 3 4 5 Networking skills 1 2 3 4 5 Modeling and analytical skills 1 2 3 4 5 Reputation management skills 1 2 3 4 5 Overall Rating 4.00
  • 8. Page 8 of 22 Level Three: Behavior Measuring Perceived Behavioral Change—Sample Forms Filled Out by Sylvester’s Manager, Two 360-Degree Evaluators, and a Human Resources Officer How has your perception of Sylvester M. changed in the following areas over the last year? Filled out by Sylvester’s manager: Skills/Behaviors No Positive Change or Negative Change Some Positive Change Very Strong Positive Change As a revenue generator 1 2 3 4 5 As a leader 1 2 3 4 5 As an ambassador for XYZ company in improving its reputation with clients, customers, and the general public 1 2 3 4 5 Ability to get things done 1 2 3 4 5 Ability to build consensus 1 2 3 4 5 Ability to communicate and listen 1 2 3 4 5 Overall Rating 4.50 Filled out by a Human Resources officer: Skills/Behaviors No Positive Change or Negative Change Some Positive Change Very Strong Positive Change As a revenue generator 1 2 3 4 5 As a leader 1 2 3 4 5 As an ambassador for XYZ company in improving its reputation with clients, customers, and the general public 1 2 3 4 5 Ability to get things done 1 2 3 4 5 Ability to build consensus 1 2 3 4 5 Ability to communicate and listen 1 2 3 4 5 Overall Rating 5.00
  • 9. Page 9 of 22 Filled out by a 360-degree evaluator: Skills/Behaviors No Positive Change or Negative Change Some Positive Change Very Strong Positive Change As a revenue generator 1 2 3 4 5 As a leader 1 2 3 4 5 As an ambassador for XYZ company in improving its reputation with clients, customers, and the general public 1 2 3 4 5 Ability to get things done 1 2 3 4 5 Ability to build consensus 1 2 3 4 5 Ability to communicate and listen 1 2 3 4 5 Overall Rating 4.17 Filled out by a 360-degree evaluator: Skills/Behaviors No Positive Change or Negative Change Some Positive Change Very Strong Positive Change As a revenue generator 1 2 3 4 5 As a leader 1 2 3 4 5 As an ambassador for XYZ company in improving its reputation with clients, customers, and the general public 1 2 3 4 Ability to get things done 1 2 3 4 5 Ability to build consensus 1 2 3 4 5 Ability to communicate and listen 1 2 3 4 5 Overall Rating 4.00
  • 10. Page 10 of 22 Level Four: Results Sample Forms Describing Measurable Improvements in Performance—Filled Out by Sylvester’s Manager and by a Senior Manager PDP Objective Comments by Manager Hire/manage new sales team that will increase revenue. Excellent progress made. Six new hires. Tremendous increase in revenue. Build relationship with new boss. Sylvester and I have a good relationship now; however, he needs to understand that I like to get feedback in writing. Reduce sales costs. Sales costs reduced from 54% of revenue to 37.8% of revenue. Excellent progress made; however, more progress needs to be made next year. Create and implement a five-year plan for the organization. Plan written and approved. Implementation process has begun. Needs to be accelerated. Build a personal network in New York and the rest of North America. Seems to have done very good work in this area, and good progress seems to have been made. Needs to continue to work at this. Build sales metrics to evaluate sales team. Initial reaction is that the model developed is good. Effectiveness of model needs to be continuously monitored along with the development of other analytical tools. Improve reputation of the division. Three strong articles have been written. Excellent progress. PDP Objective Comments by Senior Manager Hire/manage new sales team that will increase revenue. Sylvester is a great sales manager. Would prefer if his boss took less day-to-day responsibility in sales. Build relationship with new boss. Relationship seems good. However, Sylvester’s boss continues to manage day-to-day operations more than I would like. Reduce sales costs. Excellent cost reduction by Sylvester. However, his boss is delaying some cost reduction efforts that involve a reduction-in-force. Create and implement a five-year plan for the organization. Great plan. Look forward to seeing implementation next year. Build a personal network in New York and the rest of North America. Seems very good. People seem to know Sylvester and appreciate his skills. Build sales metrics to evaluate sales team. Very good model. Look forward to the development of new tools. Improve reputation of the division. Great press. Significant new articles.
  • 11. Page 11 of 22 Level Five: ROI Sample ROI Calculations based on PDP PDP Objective Metric Estimated Impact by Sylvester Estimated Impact of Coaching on Sylvester Monetary Impact of Coaching ROI in One Year1 Hire/manage new sales team that will increase revenue. Revenue increased by $125 million from $250 million to $375 million 50% 25% $15.625 million 51,983% Build relationship with new boss. Cannot be quantified, but strong performance reviews N/A N/A N/A N/A Reduce sales costs2 Sales costs reduced by 16.1% or $60.4 million 40% 33% $7.97 million 26,467% Create and implement a five-year plan for the organization. Estimated opportunity pipeline revenue over the next year from launching three new products is $80 million 25% 25% $5 million 16,567% Build a personal network in New York and the rest of North America. Cannot be quantified N/A N/A N/A N/A Build sales metrics to evaluate sales team. Ability to understand profitability per account and to select profitable business had an estimated impact on bottom line of $50 million in the first year. 70% 30% $10.5 million 34,900% Improve reputation of the division3 10% increase in revenue or $25 million 10% 25% $625,000 1,983% 1 For the sake of simplicity, ROI here is simply evaluated over a one-year period. 2 Sales costs are computed as base + bonus + sales expenses. In 2002, sales costs were $135 million, or 54.0% of the total revenue of $250 million. In 2003, sales costs were $142 million, or 37.9% of the total revenue of $375 million. 3 There were three very strong stories about IIG, two in The Wall Street Journal and one in the Financial Times, and no negative stories. According to the Public Relations Society of America, strong public relations can add as much as 10% to annual sales.
  • 12. Page 12 of 22 All of these metrics are examples of how ROI can be described. One metric should be chosen to compare across cases. Which one should be chosen? Most CFOs and Directors of Purchasing would prefer the most conservative measurement.
  • 13. Page 13 of 22 Case Two Ms. Joanna G. Managing Director, Global Head of Transportation Investment Banking XYZ financial services firm In the Fall of 2002, Joanna was named North American Group Head of Transportation at XYZ financial services firm. She reported to Jeff M., the Global Head of Industrial Investment Banking at XYZ. In January 2003, Jeff M. moved to a competitor to become Vice Chairman. Joanna faced numerous challenges, including the fact that she had no previous group management experience, had to adjust to a new boss, Jeff M., in the Fall of 2002, and then, in January 2003, had to readjust to a different new boss, Tom O., with a very different management style from Jeff M. Jeff M.’s only concern with the Managing Directors reporting to him was that they generate short-term revenue; he did not care about long-term strategy or leadership development. Jeff’s successor, Tom O., on the other hand, insisted that the group heads reporting to him not only generate revenue, but that they manage their staffs well, develop new leaders, and that they think strategically. Within the Transportation Group, turnover during the Summer and Fall of 2002 was so severe that the problem ended up on the front page of The Wall Street Journal. In addition, Joanna also faced the challenge of managing other Managing Directors who were disappointed that they were not named Group Head. The group also faced short-term revenue challenges. Revenue had been flat from 2000 to 2002, averaging about $65 million a year. Finally, there were long-term strategic challenges: over 80% of the Transportation Group’s revenue came from aviation-related businesses, with little or no revenue from other transportation areas (including: shipping, rail, trucking, and logistics). Given the cyclicality of the aviation industry and that competitors were very strong in non-aviation areas, Joanna had to immediately create and implement a strategic plan to meet her challenges. Joanna asked for a coach, and a one-year coaching assignment began in January 2003. A personal development plan (PDP) was developed. Personal Development Plan (PDP) Joanna G. 1) Build relationship with new boss 2) Improve staff management skills/improve staff morale/reduce turnover 3) Increase revenue 4) Build internal network 5) Create a long-term strategy to handle competition and to develop non-aviation sources for revenue
  • 14. Page 14 of 22 Level One: Reactions from Coachee and Coach Sample Evaluation Form for Coachees--Filled out by Joanna and her coach Filled out by Joanna Poor Average Excellent Effect of coaching in improving commercial effectiveness 1 2 3 4 5 Effect of coaching in improving leadership skills 1 2 3 4 5 Effect of coaching in improving reputation management skills 1 2 3 4 5 Effect of coaching in improving communication skills 1 2 3 4 5 Effect of coaching in improving strategic planning/strategic thinking skills 1 2 3 4 5 Overall Rating 4.00 Filled out by coach Poor Average Excellent Effect of coaching in improving commercial effectiveness 1 2 3 4 5 Effect of coaching in improving leadership skills 1 2 3 4 5 Effect of coaching in improving reputation management skills 1 2 3 4 5 Effe
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks