Designing Metrics (Dr. Bob Frost)

Designing Metrics is a short (77 page) introduction to performance metrics- those measures that companies use "to define and guide performance."  In it, Dr. Bob Frost discusses measuring frameworks, using them, and staying on track.

There's a lot of good information in this small volume.  Here are is an overview (any quotes are from the book):

Introduction

Measuring performance is important and has been for some time.  Private and non-profit organizations alike have a need to quantify their operations and measure progress towards their goals.  Still, when we talk about metrics, it's easy to be led astray.  Some general principles to stay on a good path are the need for multiple perspectives and balance.

The need for multiple perspectives.
When we focus on one element of operations (like, say, cost) and ignore others, problems ensue.
Today there is "wide acceptance of the idea that financial metrics alone do not provide enough help in understanding and managing performance" . . . "balanced, multi-perspective metrics provide a richer, truer understanding of group performance."

The need for balance.
Hand in hand with multiple perspectives is the idea of balance.  "Inside every organization, there are certain forces, goals, or value systems that tend to oppose one another . . . if any of these worthy goals becomes the sole guiding light of the organization, many other legitimate goals will be sacrificed- resulting in imbalanced efforts and, often, an early demise for the organization."

"The goal of management, and the reason excellence in management is so difficult, is balanced achievement of multiple goals, not the optimization of one thing at the expense of all else."

"Anything not measured is subject to being sacrificed for things that are measured."

Measurement Frameworks

"Your chief tool in designing [balanced] metrics is a measurement framework."  "They serve as guides in thinking about what performance actually means in your situation."  This will help you determine what to measure, what to balance, and the path to high performance.    A number of frameworks are in use today, and are introduced below.  Note that it's wise to use several frameworks to encourage multiple perspectives and balance:

Balanced Scorecard
Popular in industry today, the balanced scorecard framework looks at four measurement perspectives:
- Financial ("how do we look to our owners?")
     - Government/non-profits may redefine this perspective as "Mission Success"
- Customers ("how do we look to customers?")
- Internal ("Are we improving how we operate?")
- Learning/Growth ("Can we sustain growth & position?")

The balanced scorecard approach is considered long-term/strategic.

Stakeholder Framework
This framework focuses on the customers (or stakeholders).  It asks three main questions:
- Who are our stakeholders?
- What do they care about? [don't assume here]
- How can we measure it?

The stakeholder framework approach focuses on the here-and-now, and can be a useful complement to the long-term strategic focus of the balanced scorecard.

Program Logic Model
This approach "captures and succinctly illustrates the big picture view of an entire organization, program or effort."  It captures the following:
- Inputs
- Activities
- Outputs
- Outcomes

This approach works best when applied to things with very targeted missions.

Enterprise Performance Framework
This perspective "pays particular attention to the competitive and economic pressures behind performance in today's economy."  It looks at three fundamental ideas:
- Effectiveness ("How well are we fulfilling our mission?")
- Efficiency ("How efficiently are we operating?")
- Strategic Improvement ("How well are we preparing for success tomorrow?")

Cascading Framework
"This framework recognizes and uses the inherent structure of your organization."  It helps "ensure alignment of effort within an organization."
For this framework, each small unit in an organization "must look to the goals and Critical Success Factors of the next higher level" . . . and looks for its role in achieving those larger objectives.  The primary advantage of this approach is alignment of effort.

Translating to Measures

The author advocates a 3-step method for translating your measurement framework(s) into measures and indicators.

Step 1: "list a set of perspectives or performance topics" based on the results of your measurement framework(s), which you used to conceptualize performance.  A performance topic could be something like financial performance, customer service, product development, or the like.

Step 2: For each performance topic, "craft a list of Critical Success Factors."  CSFs are "the 'must do' factors that determine whether you are successful today and whether you will succeed in the future."

Step 3: For each CSF, define specific performance measures to track success.


Final Steps/Thoughts

Like anything, coming up with a system of metrics is not a 'one and done' affair.  Follow a "continual improvement model" (design, use, audit, improve, and repeat) to evaluate effectiveness and ensure the measures "encourage the right kinds of behaviors and right kinds of decisions in your group or organization."  "We have to ask whether we are creating what was intended."


Rating: A-