The game of telephone

Recently, I was working with our support team on triaging an issue that they were reporting. Like others in the software space, I am sure I am not alone in finding the steps to replicate somewhat lacking in detail needed. Luckily, our software team finished installing a third-party application that monitors and records our users’ interactions within our application. This tool proved to be a lifesaver in that we were able to find the problem that the user was happening.

As it turns out, the information we were getting was not at all the problem that we needed to solve. It was a game of telephone where the original message is warped as it passes through various people until the person at the end of the line gets a message that has nothing to do with the original. That was the case here.

Bringing in actionable data

Close contact with users is preferred. Getting the information right from them yields actionable insights to improve the adoption and usage of your application. But many users don’t live to make your application better. You have to remember that getting the information you need to make your product better comes at a cost to your users’ time, attention, or workflow. A cost that some may not want to pay. That makes the job of finding value much harder on your own unless you have a tool that provides you insights inside the product.

When you are able to talk to people, having data that you can search and gain insights that may back up what you are hearing from a user is a great way to confirm that feedback and confirm value. Not all suggestions are issues or ideas that should be implemented. You can take these suggestions or ideas as a starting point to look into if it lines up with what you are seeing in your total user base. If it is, you may have something valuable to put in place. If it isn’t, you may have found an outlier case that needs to be further considered and vetted. In either case, if you do not have a tool that provides you actionable data and insights into your product, you are operating off assumptions. You may be able to make some great changes on an assumption alone, but vetting those assumptions with data is the better way.

Empiricism

In the scrum framework there is emphasis on the concept of empiricism where you make decisions on experience, facts, and evidence. This is achieved in the three scrum pillars of empiricism: transparency, inspection, and adaption. Now, typically this refers to the process by which a team is producing increments of value. But, I’m going to take these pillars and adapt them to the product itself. that these three pillars apply to what your team is producing in relation to what your user is doing with your application.

Transparency

Rather than making decisions based on what you intend to happen or what you think your user is doing, you need to be able to observe your user’s behavior, collect data on many points, and adapt your application based on what your user is experiencing.

If your product doesn’t allow you to see what your user is doing (either by collecting usage data and presenting that to your team, or by actually recording various interactions by your user, you don’t have product transparency. Any decisions you make will be based on a guess. Even if you interview all your users and diligently record and collect their feedback, you are still guessing. That’s because you cannot totally believe what is being told to you. You have to actively observe the behavior. A good case in point is this video. You have to verify that feedback with observable behavior or data to truly understand what is going on in your application.

Inspection

You may be able to access data points left behind from your user, but if it isn’t consumable, you are not achieving the necessary transparency. That brings us to the point of inspection. You can collect all the data in the world, but unless you have someone reviewing that data, seeing the behavior you are not operating under empiricism.

Someone needs the time and capacity to review this information. You cannot collect solid insights without that inspection.

Adaption

The final point is after seeing the behaviors, collecting, and verifying the data, you have to be able to adapt your work to fit the observed reality. In some cases you may want to reinforce behaviors, in other instances you may want to change a user’s behavior. But, if your application is too rigid, or your culture/environment or directives are unchangeable then you will fail to adapt and will create an unremarkable product. The ability to adapt must be at all levels of the company Leadership must be able to change the objectives or change the objectives based on the insights gained just as much as teams need to be able to change their direction and backlogs so that they can find the things are going to be of most value to the customer.

Why Empiricism Matters

It has been said that the more you can delight your customers, the more successful your product will be. Too often we settle for mediocre interactions because we cannot observe what is going on with our users. We are not operating under empiricism. We are guessing. We may even be getting feedback, but unless we can confirm that feedback with solid insights into our end-users actual behavior, we are still guessing to some degree.

If you found this helpful, please share with your colleagues, friends, boss, etc. Also, feel free to start reading my Agile Laws series here.

Thanks for reading!

Agile ↑