Measuring the Impact of Design

Using independent customer research to understand where design was creating value

Executive Summary

Design impact is often difficult to quantify. Teams can demonstrate what they have shipped, how many customers have used a feature or how much work has been completed, but those measures don't necessarily tell us whether the experience has actually improved.

At PensionBee, I wanted a clearer view of whether the work my team and I were doing was changing how customers perceived the product.

Every six months, Boring Money independently reviewed the major investment platforms in the market, surveying customers and sharing the results with the companies featured. Because the research also captured how customers perceived our competitors, it provided something particularly valuable: an impartial view of where PensionBee was performing well, where there were opportunities to improve and how our position was changing relative to the wider market.

Rather than treating the research as a market report, I used it as an ongoing measure of design impact.

Four measures became particularly important to us: ease of use, app experience, website experience and tools. A fifth measure, guidance, became significant when we introduced a major new content experience into the app.

Across the year, all five improved.

Ease of use: +1
App experience: +7
Website experience: +3
Tools: +5
Guidance: +12

The value wasn't simply in seeing scores increase. The changes gave us an independent way to connect customer perception with the strategic work being undertaken across Product and Design.

Measuring What Matters

As a design leader, I wanted the team to understand the impact of our work beyond the number of features we delivered.

Traditional product metrics could tell us whether customers were completing journeys, transferring pensions or using particular features. But they didn't always tell us whether customers felt the product had become easier, clearer or more useful.

Boring Money's research provided a complementary perspective.

It gave us an independent assessment of how customers perceived PensionBee, alongside comparable data from competitors.

That made the movement between reviews particularly useful.

A score increasing was positive.

A score increasing while competitors were declining was even more significant.

The research therefore became one of the ways I assessed whether our strategic design decisions were translating into a better customer experience.

Ease of Use → +1

Ease of use had consistently been one of PensionBee's strengths.

The relatively small increase wasn't disappointing. It demonstrated that as the product evolved, we had maintained a strong baseline rather than allowing increasing functionality and complexity to erode usability.

For me, this was an important measure of design maturity.

Improvement isn't always about dramatic increases.

Sometimes the achievement is ensuring that greater capability doesn't come at the expense of simplicity.

As more functionality was introduced, we continued to focus on information hierarchy, navigation, interaction patterns and the clarity of individual journeys.

The result was a product that continued to be perceived as easy to use while becoming more capable.

App Experience → +7

This was one of the clearest signals that the larger strategic changes we had been making were being felt by customers.

Two major projects during the year focused on fundamental improvements to the app experience.

Rather than treating the app as a collection of individual features, we looked at the experience more holistically: how customers understood where they were, how they navigated between information and actions, and how confidently they could manage their pensions.

A seven-point increase provided independent evidence that those changes were moving the experience in the intended direction.

The significance wasn't simply the number.

It was the correlation between the customer perception data and the strategic work being undertaken by the product and design teams.

Website Experience → +3

The website improvements were more incremental.

We worked on the information architecture and navigation alongside a series of smaller improvements intended to make it easier for customers to find and understand what they needed.

On its own, a three-point increase might appear modest.

The competitor data provided important context.

While PensionBee's score increased by three points, the average competitor score fell by two.

That created a five-point relative movement and strengthened the evidence that the changes we were making were improving the customer experience rather than simply reflecting a broader market shift.

It reinforced an approach I use consistently as a design leader:

Impact needs context.

A metric rarely tells the whole story by itself. Understanding how it moves relative to the market can reveal whether an apparent improvement is actually meaningful.

Tools → +5

PensionBee's retirement planning tools were an important part of the customer experience.

Rather than rebuilding them wholesale, we made smaller, focused improvements over the year to increase their usefulness, quality and the insight they provided to customers.

The five-point increase suggested that these incremental improvements were being recognised.

It was another useful reminder that product impact doesn't always come from a single transformational feature.

Small improvements, consistently applied to something customers already value, can compound into a significantly better experience.

Guidance → +12

This became the most significant movement in the research.

Guidance hadn't previously been one of the measures we focused on particularly closely.

That changed because we undertook a major piece of work to bring high-quality pension guidance and content directly into the app.

Previously, much of this information wasn't readily accessible within the product itself.

The challenge wasn't simply creating more content.

It was understanding what information customers needed, when they needed it and how to make it accessible within an experience already designed around managing a pension.

The twelve-point increase provided a strong indication that the strategy was working.

Customers weren't simply being given more information.

They were finding the guidance valuable enough for it to materially change their perception of the product.

From Metrics to Design Leadership

The real value of the research was what happened after we received it.

I used the results to help the team understand where our work was creating measurable value and where we needed to maintain or improve our focus.

It created a shared language between Design, Product and the wider business.

Instead of discussing whether a design was "better", we could increasingly ask:

What evidence do we have that this creates a better customer experience?

That distinction matters.

It moves design conversations away from personal preference and towards customer outcomes.

It also helped demonstrate the value of design at an organisational level.

The research provided an independent reference point that could be used alongside commercial and product metrics to understand the effect of strategic design decisions.

What the Data Told Us

Across the year, the movement was consistently positive:

Ease of use: +1
App experience: +7
Website experience: +3
Tools: +5
Guidance: +12

The pattern was particularly encouraging because the improvements reflected different types of design intervention.

Some were large-scale product changes.

Some were incremental improvements.

Some involved navigation and information architecture.

Others involved bringing entirely new forms of guidance into the product.

The common thread was a focus on understanding what customers needed and then using design to make those needs easier to fulfil.

Reflection

The most valuable part of this work wasn't the scores themselves.

It was creating a mechanism for holding Design accountable to customer outcomes.

As a design leader, I don't want to measure success by how much design work my team produces. I want to understand whether that work makes a meaningful difference to customers and the business.

Independent research made that easier to see.

It also reinforced an important principle in how I lead design teams:

Design should be able to explain not only what it changed, but why the change mattered.

The Boring Money research gave us one way to do that.

It allowed us to step outside our own organisation, compare ourselves against the market and see whether customers were actually experiencing the improvements we believed we were making.

For me, that is the real measure of design maturity: moving from demonstrating the quality of the work to demonstrating the value of the outcome.