Nov 15, 2023

Are we measuring the wrong thing?

Designing a Better World 

Building a company or creating meaningful change in a company requires some measurement to understand the impact of your work. Numbers are necessary to look at financial reports, the number of users, app downloads, or other numerical data.

While attempting to understand your company through data, that data could lead you astray from your company’s vision and goal. 

Photo of author Don Norman, provided by jnd.org

From the Lens of UX

Don Norman, the father of UX, challenges how companies have focused on the raw numbers to understand progress in their business. Norman has worked on significant projects like the Three Mile Island Nuclear Accident. He has worked for giants like Apple and Hewlett-Packer, in addition to working at major academic institutes. This experience has culminated in his latest book, Designing for a Better World, where he reflects on how design has caused problems for society and, more importantly, how we might fix those problems.

A few chapters focus specifically on data-driven decisions and their impact on our world. Norman focuses on the GDP and government decisions around economic growth, but the principles he sets forth are relevant to any organization of any size. Even more so, the underlying principles that power the GDP to grow may influence how your company runs.

GDP as an Example

The GDP (Gross Domestic Product) is a way for countries to measure their economic growth and, therefore, can give insight into how well a country is doing. Much like KPIs (Key Performance Indicator) for a business, the data around GDPs are often shown as an indicator of the health of a country. 

However, GDP has a narrow degree of measurement. The focus of the GDP is primarily on goods and services produced by a country. Further, it is a single number that attempts to sum up the quality of life for citizens of a country. How the GDP combines multiple figures to give a simplified number removes any nuance or context – removing the meaningfulness of the number itself. Further, it doesn’t even include data points many would find relevant when referencing a country's “health.”

The World Economic Forum believes that other measures could be relevant to the discussion of how a country is doing –

  • Good Jobs
  • Well Being
  • Environment
  • Sustainability 
  • Fairness and equity
  • Health

Norman includes his own list, which includes more points like education, freedom from hunger, quality of governance, and happiness – to name a few. However, he points out that it would be difficult, if not impossible, to combine all of these metrics into a single number that accurately portrays the quality of life for a particular country. Norman doesn’t even want you to try.

Instead, he suggests that we need a much more complete dashboard to understand what is happening in a country and where the focus should be going forward. 

Getting Down to Business

Like the GDP, businesses may make the primary key indicator of revenue growth year-over-year. At first blush, this type of measurement seems relevant. A growing company is probably doing many things right, indicating health. However, this approach is missing some key aspects.

Measuring What Matters

If profits are the primary reason for a company’s existence, then it is reasonable that this would be the primary measure of success. However, most organizations exist beyond making buckets of cash. Most have a vision to influence the world positively. In companies like these, using profit growth as the success metric doesn’t take the company's mission fully into consideration. 

Similar to the GDP being inadequate, companies should consider what they are attempting to do in the world – their why- and work to find critical measurements relevant to that success. Certainly, staying financially responsible should be part of that mix, but it should not be the primary indicator of success.

Students in the LaunchX Entrepreneurship Summer Program must go through the process of dinging their mission and vision – their why. This is a critical step that aligns teams together and begins to illuminate the definition of success for the company.

Even for a company that primarily exists to make as much money as possible, profits as the primary measurement of success can cause problems. Businesses can make short-term decisions that inflate profits temporarily but cost the company long-term. 

Generated by AI – DALLE-3

Profits Alone and Shortsightedness

Imagine a company that sells a revolutionary new calendar software. It is the smartest way to schedule and keep track of schedules. It is a boom, and many consumers are purchasing it despite the high cost because the software is just that good. 

If the company primarily focuses on profits, they may see high company success in the first year. You can imagine the headlines – “Calendar Unicorn Breaks 20M Subscribers in 2 months!” Profits soar, investors are happy, the CEO gets a big bonus – the company is successful. 

However, what if this fictional company decided to get the product off the ground by any means possible? To ship quickly, they pushed the developers to 70-hour work weeks. They determine that it is cheaper to hire new salespeople fresh out of college and underpay them until they wise up and leave. And their support team is anemic because that maximizes revenue.

These decisions seem to align with the company’s primary goal –making money. However, the shortsightedness of these decisions means that in a few months, developers will quit, quality will wain, sales will drop, and support will be abysmal. Ultimately hurting profits and eventually killing the company.

Holistic Health Indicators

To generate valid quality indicators, there needs to be a wide variety of numbers that give insights into different aspects of the company. For our fictional company, what if they measured beyond just profits –

  • Profits
  • Team Culture Health
  • Turnover rate
  • Industry-standard Pay Rate
  • Customer Betterment 
  • Software quality

These measurements (not summed up into a single number) can help leadership pull levers that balance the company both here and now and in the future – creating a longevity plan.

Measuring Isn’t Easy

These measurements listed above aren’t all easy to obtain. There are more straightforward measurements, like the Industry–Standard Pay Rate, that you can measure up against. But points like Team Culture Health and Customer Betterment are much more difficult to pinpoint.

Norman argues that this is one reason meaningful measurements are left out – it is too difficult to obtain.

He sees the primary reason for this attitude as the overuse of the scientific method as the primary way of measuring. Our focus on driving a KPI to a succinct and relevant single number is unrealistic. 

Instead, the goal should be to combine quantitative data, such as the Industry Pay Rate, with qualitative data, such as Customer Betterment. Getting relevant data about whether customers’ lives are better from the product may require ethnographic interviews with customers, looking at what is important to them, and, perhaps more importantly, whether the product produces adverse outcomes detrimental to the longevity of the customers – which is harmful to the longevity of the company.

This approach is much more difficult. It is also far more humanitarian. 

If this is critical to companies whose primary goal is profit growth, how much more relevant is it to companies who want to positively impact the world?

Generated by AI – DALLE-3

Wrestling Through What to Measure

This shift toward more relevant indicators is a gradual change. This framework takes time to figure out what actually matters to the company, what measurements are truly relevant (and not just proxies of meaningful measures), and how to measure those aspects effectively.

Presenting Key Indicators

The final stage of the process is presenting these indicators in a relevant way that offers insight to the company – allowing the company to make better decisions. As the GDP is too simplistic of an indicator to measure the well-being of a country effectively, businesses must find ways to communicate the health of their organization. 

One method that Norman suggests is including stories as part of the measurement. When talking about statistical data –

“Numbers, statistics, dates, and historical records are important, but they are abstract, devoid of meaning. Stories are the key to adding context, the environmental setting, and meaning to the otherwise dry, impersonal abstraction of data.”

As we all probably do, Norman recognizes that numbers don’t give the whole story. In fact, without context, numbers can become meaningless figures. Instead, Norman suggests borrowing from the social sciences to include qualitative data alongside the imperial numbers to provide relevant context and meaning to what is happening.

Further, Norman suggests a single way of presenting the information is insufficient. You may have an incredible dashboard that offers insight into many aspects of your business. Still, it is mostly likely insufficient in giving a wide enough context to make vital business decisions. Instead of creating 1 mega-dashboard of relevant information, the goal should be to provide an adequate snapshot of how the company performs against its vision and mission – which will most likely take more than 1 PowerPoint slide.

Though this is difficult design work, developing relevant indicators and presenting them well is never easy and perhaps is never finished. However, Norman suggests this is the right work to establish a good foundation for organizations to flourish – and ultimately to make the world a better place.  

For the Sake of the Business and Humanity

Companies and organizations must commit to intentionally designing or redesigning what they measure to become more relevant to humanity. These nuanced indicators are a way to reshape businesses to help influence the world for the better while building sustainable and relevant companies.

Thomas Clapper
Thomas Clapper
Marketing and Brand Manager
Follow us on:

Subscribe to our newsletter