AXIS Performance Advisors

Demystifying sustainability

Making the Most of Measures

Copyright 1999 AXIS Performance Advisors, Inc.

Making the Most of Measures

A report on how organizations measure teams


We've been talking a lot about team measures with our clients over thelast few years. The reactions we get to these conversations have been interesting.Sometimes when we introduce the subject of measures we see eyes glaze over.It's like talking about taxes - a tedious, detailed task of administriviawe know we have to do but really wish we could avoid. Intellectually, everyoneunderstands the importance of reliable and timely feedback data to continuousimprovement, but few seem able to muster the energy, time, buy-in and resourcesto develop useful measurement systems. We wondered how many organizationswere struggling with these issues and what we could learn from the successesand trials of others.


What we asked

Last fall we developed and distributed a survey that had five categoriesof questions regarding teams and team-based measurement systems. The firstfour categories of questions had to do specifically with how organizationscollected and used performance data. We asked about . . .

· What teams measure

· How they collected their data on performance

· What they did with the data once they had it

· What the result has been of keeping team measures

And because measurements and results so often form the basis for rewardand recognition systems, we also included a set of questions about whatorganizations were doing in this arena.

We identified several items within each category and asked respondentsto answer two questions about each of these items:

· Do you do this in your organization?

· If you do, how important or helpful has it been to team performance?(rated on a scale of 1-5)

Generally speaking, the responses we got confirmed out faith that measuresand performance feedback help teams. With few exceptions, our respondentsfound value in performing most of the example measurement tasks we includedin our survey. Inside are the highlights from the data.

Comments from our respondents tell us that establishing effective measuresis important, but not easy. Intimidation is no excuse for not starting somewhere,these experienced organizations told us. Get something in place as soonas possible and expect to tweak it as you go. They reassured us that itdoes get easier.


What teams measure

When deciding what to measure, start first with measures that track teamrather than individual performance. Our data indicates that individual performancedata are of less value (judging from the ratings and number of organizationstracking it) while team data reinforce collaboration and problem solving.

Be sure that you have a balanced set of measures. One of our clientsgot into trouble by having their teams track only sales. Not surprisinglysales were going up, but so were complaints from customers! To prevent theteams from sacrificing long term success for quick gains, the organizationadded customer satisfaction to team score cards.

Measures should also be meaningful to team members and flexible enoughto change over time to suit changing team needs and issues. One of our respondentstold us that they originally designed their measures to satisfy their BudgetCommittee. Since these had little meaning to the team members, there wasno buy-in to collecting or reviewing data.

The place to start, we were told repeatedly, is with the organization'sand the team's mission and vision. Use these to identify 4-5 key performanceindicators. If you pick too many, you will confuse people about priorities.(One organization reported making the mistake of starting with 20! No onecould remember them all.) Our survey respondents seemed to like some versionof the set we included on the survey: costs, customer satisfaction, quality,productivity and work processes.

We were also reminded to choose SMART measures: specific, measurable,attainable, relevant, and time bound.

How they collect their team data

Two points stand out on this issue: teams need feedback specific to theirperformance as opposed to the performance of the organization as a whole,and the feedback is more valuable if it is collected by the teams themselves.Also, using specific tools or instruments provided more value than casuallycollecting anecdotal data.

One of our respondents reminded us of how much easier the task of collectingand reporting data is when you have well designed information systems. Theless you have to manually record and compute, the better.

What they do with the measurement data

Performance data is only valuable to the extent that it is reviewed andused. Most respondents reported reviewing their data regularly (some atevery team meeting). Waiting until there is a problem or only looking atsome of the data doesn't have nearly the same pay-off.

Many organizations also reported satisfying results with using performancedata as the basis for goal setting as well as team feedback and performancereviews. AXIS has developed a process to help teams do this that we callthe Team Improvement ReviewTM process. It is built around a set of fourreview questions:

· How are we doing (an examination of the data)?

· What have we learned (what does the data indicate about ourstrengths and weaknesses)?

· What should we do next (setting improvement goals)?

· What resources will we need to achieve our new goals (what help,training, equipment, etc.)?

One of our respondents made the wise suggestion to make sure your teammembers know how to use their measures as a tool for implementation. Withoutthe skills to analyze data, problem solve, or build business cases for theirideas, all they'll have is a bunch of information.

What the effect of tracking performance has been

Measures build teams. Our respondents were quite clear on this point.Measures helped their teams focus, collaborate, and make progress on targetedimprovements.

Few of the people surveyed felt that tracking performance made theirteams feel as if they weren't trusted to do a good job. It's just as wesuspected. People want to know how they are doing.

What we found out about rewards and recognition

While technically a separate issue, we felt like asking about measureswithout asking also about rewards and recognition would have left everyonewaiting for the other shoe to drop. As we saw earlier, measures are usefulto a team's focus and goal setting. They also provide obvious fodder forrewards.

What our data indicated was interesting. Most organizations reportedmore satisfaction with simple team celebrations and "atta boys"than they did with tangible or financial rewards for either teams or individuals.This is consistent with Alfie Kohn's philosophy about rewards. (See Punishedby Rewards)

Most of the celebrations we have seen teams do center around food. Oneof our respondents, a manager at Con-Way Transportation Services, a subsidiaryof CNF transportation, Inc., in Portland, OR, described a time when histeam had to put in some late hours. He made a point of bringing a specialdinner in to them each night they worked­one night Chinese, anotherThai, another Mexican, and so on. The big hit was the night he brought homemadelasagna. At the end of the project the team celebrated at one of Portland'sbest Italian restaurants.

Where performance did impact compensation, we see that profit sharingis more popular than any of the other suggested reward systems. One respondentprovided this caution, however. They recommended not tying team pay to measuresuntil the metrics have been in place for one full cycle, and have been validatedand agreed upon by management and teams.

Who responded

Thanks to the Internet we had responses from team based organizationsaround the world! Below is a break down of respondents by industry.

Financial services 2

Government/education 30

Health care 5

Manufacturing 23

Non-financial services 5

Utilities/transportation 26

Other 8



The Results

What do your teams measure?

 Survey Item

Percent of respondents who do this in their organizations

Value of the practice to teams as measured on a 1-5 scale
 Customer satisfaction



 Quality of the end product or service



 Work processes









 Team effectiveness



 Individual performance




How do your teams collect their data?

 Survey Item



 We use formal, validated methods



 We collect the data ourselves



We have data specifically on our team



 We use our own methods/instruments



We have data for the whole organization



 Others collect and report data to us



 We use informal, anecdotal evidence



What do your teams do with the data?

 Survey Item



 Regularly review all our data as a team



 Use data to set team and improvement goals



Report the results to management



 Use team data in our official performance review process



Use data to give informal feedback



 Base rewards/compensation on team measures



 Regularly review some measures but not all



 Review data only when there's a problem



 We seldom look at our team data





What has been the effect of measuring teams?

 Survey Item



 Measures keep us focused



 Measures help us improve



The data shows what we contribute



 Having to track our performance makes us feel like we are not trusted





How do you recognize and reward teams?

 Survey Item



 Celebrations at team level



 Profit sharing



Informal pats on the back



 Tangible rewards to individuals



Tangible rewards for teams



 Celebrations for the whole org.



 Public recognition/reward ceremonies



 Gain sharing



 Financial rewards to individuals



 Financial rewards to teams





Case Example­ Using measures for redesigning
work processes

One of our clients is a healthcare non-profit providing services primarily
to women. Due to a number of factors, their patient revenues have been declining
while their expenses have remained level. Most of their facilities do not
pay for themselves and are subsidized by donations. Their management realized
that they needed to turn this picture around.

Step 1: Pre-Launch. We helped them form a task force to redesign their
work process. The first step was to sit down with the managers and develop
a project charter for this team, carefully clarifying boundaries, purpose,
expectations, roles, etc. This "pre-launch process" typically
takes four hours. In this we identified a set of measurable goals for the
team, including:

· Ability to handle 3-4 patient visits per clinician per hour

· Same day appointments for urgent needs; no more than one weekwait to get other appointments

· Wait time for patients not to exceed 10 minutes past their appointmenttime

These measures have kept the team focused on business needs.

Step 2: Launch. At the first meeting with the team, the managers walkedthe new members through the team charter as we clarified and negotiatedthe details. We also established team agreements for how to operate.

Step 3: Redesign. Now we are in the process of mapping their work processes,gathering baseline data on their processes, and generating ideas for meetingthe measures. (Tip: Use pictures to map out the process. These icons keeppeople from tunneling too deeply into any one step.) We are expecting tocomplete the project with 8 half-day meetings, fully expecting to turn aroundtheir business.


How are your measures working?

Are you getting the most out of your measurement system and avoidingthe pitfalls our respondents highlighted? Take this test and see. If youanswer "NO" to any of these questions, we're ready to help youset your teams back on track.



 Best Practices Assessment


 Do your teams track a balanced set of key performance indicators?

Are those indicators linked to the organization's and team's mission?

Are your teams tracking fewer than six measures?

Do your teams get performance data related to their specific performance?

Do your teams gather and report this information themselves?

Do your teams review all their data regularly as a team?

Do your teams use their data to identify new goals and improvement projects?

Do team members use performance data to give feedback to one another?

Are you consistent about celebrating the successes and learnings indicatedby your performance results




Here are some of the books and materials survey respondents reportedas having been helpful:

· Johana L Howell, Tools for Facilitating Team Meetings

· Barder, Bloom and Chang, Measuring Team Performance

· Mark G. Brown, Keeping Score

· DDI's Team Start-Up Kit

· Hitchcock and Willard, Why Teams Can Fail and What to Do AboutIt

· Katzenbach and Smith, Wisdom of Teams

· Maureen O'Brien, Who's got the Ball?

· G. Rummler, Improving Performance

· Sullivan and Harper, Hope is Not a Method

· Peter Scholtes, The Team Handbook


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


This entry was posted on July 31, 2013 by in Articles/Posts and tagged , , , , .

Enter your email address to follow this blog and receive notifications of new posts by email.