SAP built intelligent claim matching into its trade claims product. The analysts it was built for did not trust it and mostly did not know it was there. Six months of research and prototyping to work out whether gamification could change that, and what would happen if we got it wrong.
(view larger)
Context
What the ITCM does
SAP Eureka launched the Intelligent Trade Claim Management system in 2020. It sits between consumer products companies and the retailers they sell through, handling the claims and promotions that pass between them.
The pitch was automation. Smart matching, document handling, and a lot less of the manual work the job used to take.
Who was using it
Claims analysts. Before the ITCM they worked out of spreadsheets, email, printed statements and their own memory, and a lot of them still did after it arrived.
They are measured on volume and accuracy, and most of them have been doing the work long enough to be fast at it their own way.
My role
I ran the research and the visual design across a six month engagement, working with SAP stakeholders and a small project team. The output was a gamification strategy, a playbook SAP could apply to other products, and a working proof of concept.
Why nobody used the smart features
Talking to analysts first
The features worked. That was not in question. What we did not know was why people who would obviously benefit from them were leaving them alone.
What we asked about
What a typical day looks like, and where the time goes.
Who sets their goals, who rates their performance, and how both get measured.
Which ITCM features they use, which they have never opened, and why.
(view larger)
Three things came back
01 Habits
Seasoned analysts have a way of working that already gets them through the day. Nothing in the tool gave them a reason to change it, and changing it has a cost they pay immediately while the benefit arrives later, if at all.
02 Trust
The smart matching produced results people did not believe. When software proposes a match on a claim you are accountable for, an unexplained answer is worse than no answer, because now you have to check it anyway.
03 Discoverability
Even the analysts who wanted to adopt the ITCM could not find the full set of features. Willingness was not the constraint. Knowing what existed was.
Awareness against value
Two factors kept surfacing in the interviews and they were not the same factor. Someone can know a feature exists and think it is useless. Someone else can want exactly what it does and have no idea it is in there.
Plotting them against each other gave us four groups instead of one undifferentiated pile of non-adopters, and each group needs something different.
Why it mattered
It stopped the team treating adoption as a single problem with a single fix. The person who has never heard of automatch and the person who has tried it and does not rate it are not going to respond to the same intervention.
(view larger)
The problem, stated
One paragraph everyone signed off on
The ITCM provides time saving intelligent features that increase the volume and accuracy with which analysts can process claims. Because of low perceived value and low awareness, those features are chronically underused.
How might we increase the usage and discoverability of these features?
01 Increase usage
Get more analysts using the features that already exist, rather than building more of them.
02 Encourage discovery
Give people a reason to go looking, and something to find when they do.
03 Acknowledge completion
Close the loop on work that finishes. Analysts told us the job mostly runs on nobody noticing until something goes wrong.
What we agreed to measure
Three KPIs
Task completion rate.
Engagement with the newer features.
Conversion into the ITCM features from wherever the analyst started.
Why write them down early
Gamification is easy to declare a success. Points go up, badges get awarded, and none of that tells you whether more claims got processed. Agreeing on the numbers before we designed anything meant the pilot had a way to fail.
Six ideas, one direction
Blue skies, then a filter
We ran an open workshop with SAP stakeholders on the rule that nothing gets rejected in the room. Six solution spaces came out of it, and we scored each on user value, business value and technical feasibility before committing.
The six
A plan generator, in app gamification, an AI task guidance assistant, goal tracking, a wizard for breaking down complex actions, and a visual process builder.
Why gamification
It was the only one that landed on all three axes. Users wanted their work recognised, SAP wanted feature usage up, and it could be built on top of the ITCM without waiting on the AI roadmap.
The risk we took on
Gamification applied badly does not do nothing. It actively works against you. Points attached to the wrong action teach people to do that action, and if the action is not the one that matters you have paid to make the problem worse.
So we read first. An extensive literature review of what has worked, what has failed and why, before anyone drew a badge.
The vocabulary we settled on
Dynamics are the psychological effects a tactic produces. Mechanics are how a dynamic gets delivered in an application. Gamification is the adaptation of game design principles to workplace processes and behaviours.
Splitting those apart kept the conversation off badge shapes and on what behaviour we were trying to move.
(view larger)
Two deliverables
A playbook
SAP wanted anything we learned to be reusable across their other products, so the principles, definitions and expected outcomes were written up as a standalone resource rather than living inside one prototype.
A proof of concept
Principles on their own do not survive contact with a stakeholder review. We picked a handful of real ITCM tasks and workflows, applied the tactics to them, and tested the result.
Designing the game
Three target areas
The goal was lasting habit change, not a spike. These were the three areas where a game layer had a plausible route to it.
Goal tracking
Analysts are already measured against targets they cannot see day to day. Surfacing progress costs nothing and answers a question they were asking anyway.
Achievement and validation
Recognise work that gets done well, publicly, through kudos and rewards rather than a private counter.
Community and belonging
Claims work is more collaborative than it looks from outside. Peer to peer interaction makes a tedious task more tolerable, which is a lower bar than making it fun and a more honest one.
Two people to design for
The user base is small and specific, so we wrote two personas rather than a spread. Collette processes claims. Craig manages the people who do.
Their motivations pull in different directions, and the settings work later in the project exists entirely because of that gap.
What each one wants
Collette wants accuracy, speed, collaboration, and to have the effort noticed.
Craig wants his team to actually use the tool the company bought, wants performance he can track and reward, and wants a team culture worth staying in.
-
(view larger)
Collette, claims analyst -
(view larger)
Craig, finance team manager
Introducing the game
Three workflows carried most of the weight. The first is the one that decides whether anybody engages at all, which is how a user finds out the game exists.
Key aspects
An in app announcement, sized to the persona rather than the same modal for everyone.
From there a user can go to the dashboard, to the achievements tab, or straight into configuring the game if they have the role for it.
Craig's route ends in setup. Collette's ends in her own progress.
(view larger)
Giving props
The social flow. Analysts depend on each other to close claims and there was no way to say so inside the product.
Key aspects
Props start from the achievements dashboard, not from a separate social area nobody would visit.
Pick a colleague, pick the kind of props, and the message is optional. Making it mandatory would have cut the number of people who bother.
(view larger)
Settings and configuration
Managers and administrators needed to run the game, not just play it. We laid out the settings architecture alongside the flows for changing point values and adding bonuses.
Key aspects
Points, badges and bonuses live under one Achievements settings area rather than scattered through business rules.
A manager can boost a specific action for a fixed window, which is the lever Craig actually asked for.
Everyone affected gets told when the rules change. A game whose scoring shifts silently is a game people stop trusting.
(view larger)
What testing changed
01 The rules were hidden
The first build put the scoring behind a button labelled Show Rules, which opened a modal explaining how to earn points. It got no engagement at all.
What changed
Nobody presses a button to find out whether a thing is worth their time. The rules had to be readable before the decision to engage, not after it.
We moved the rubric onto the dashboard as a card. Same content, no click, and it turned into the introduction to gamification that the modal was supposed to be.
02 Too many knobs
We tested a settings screen that let managers define every badge, its criteria and its point value. Managers could do anything, and that turned out to be the problem for two separate reasons.
What we heard
Managers took a long time to get through setup, which delays rollout, and the engineering cost of full customisation was not something SAP was going to sign off.
The second version fixes the number of badges and lets managers set values against them. Slower to outgrow, much faster to turn on.
-
(view larger)
Every badge configurable -
(view larger)
Fixed tiers, editable values
The gamification system settings needs to have some level of tiers to make it feasible from a development perspective. We can't just make everything and anything customizable.
03 The game got loud
Every kudos and every badge fired a notification. In testing, analysts started missing notifications about their actual claims because the game was filling the same tray.
This is the failure mode the literature review had warned about, arriving on schedule. We consolidated the game notifications and moved them onto SAP's Fiori patterns so they read as one stream instead of a stack of interruptions.
Key aspects
Game events are grouped rather than announced one at a time.
Task notifications keep their existing weight and colour, so the thing you are paid to look at still wins.
(view larger)
Where it landed
The final usability test
With the six months nearly up we ran a last round against three questions. Did people understand the point system, could they use it to read their own behaviour, and could a manager configure it for a real business.
What worked
The point system was functional and legible. Analysts could see where they stood in the month without asking anyone, which was the goal tracking bet paying off.
Badges got a warmer reception than I expected from a room of people processing trade claims.
-
(view larger)
Finding out the game exists -
(view larger)
The achievements dashboard -
(view larger)
A manager boosting an action for a week -
(view larger)
Points configuration
I'm liking how it lets me see where I am at in the month.
What did not work
First run experience
The announcement told people the game existed and then left them to it. Onboarding would have given users more confidence that they could work the system, and we ran out of time to build it.
Inconsistent components
We added components Fiori did not cover, and the new call to action styles did not match the old ones. Users could not tell which elements were interactive. The fix is a UI audit against Fiori, which we flagged rather than finished.
The points feed
People liked the idea of it and could not read it. The card did not match the mental model anyone brought to it, and it needed another pass we did not get.
What we handed over
The engagement ended at the proof of concept. These were the recommendations that went to SAP with it.
Interview after launch
Everything we know about how analysts feel about the point system comes from testing a prototype. The qualitative read after it is live is a different question and it will move the subjective calls, badge design among them.
Watch the behaviour data
The pilot generates the first real evidence about which point actions actually drive feature usage. Some will do nothing. Prioritising the ones that work is how the system matures instead of ossifying.
Gamify what comes next
As the ITCM grows, the same tactics apply to new features. That is what the playbook was for.
Reflections
01
Adoption problems get treated as one problem and they are usually two. Not knowing a feature exists and not believing it is worth using look identical in the usage numbers, and nothing you build fixes both. The awareness matrix was the cheapest artefact on this project and it changed more decisions than anything else we made.
02
Gamification is not a reward you bolt onto software people dislike. Every failure in the literature came from points attached to the wrong action, and the notification problem we hit in testing was the same mistake in miniature. The research up front is what made that a two week fix instead of a launched product teaching analysts to ignore their own alerts.
03
This was a proof of concept and it ended as one. I would rather say that plainly than dress six months of research and prototyping up as a shipped outcome. What it produced was a tested design, a playbook SAP could reuse, and a clear list of what to fix first, which is what the engagement was scoped to produce.