If an engineering or product manager tells you that their end users are doing exactly what they expected them to do, they’re either clairvoyant, in the dark, or seriously massaging the truth. Any R&D veteran will flat out tell you that when they meet end users face to face they are constantly surprised by not only how users are using their applications, but what those users are doing with their applications. Understanding exactly how users are using your apps is the holy grail for R&D and product engineering.

Image Source

Over the years we adopted various methods that help us learn what users are doing: such as tools like Google Analytics that provide page monitoring for web applications, and monitoring tools that map which APIs and flows are used. These tools provide a good start and general understanding of user activities but have an Achilles heel.

1. We can only understand activities that we have instrumented in advance. Meaning that we need to understand what events are important in advance and build and maintain the right event triggers within our code to understand our user’s flows.
2. These reports are based on selective instrumentation, the user insights provide only a partial picture. As a result, it is possible to arrive at the wrong conclusion.
3. This type of instrumentation provides a good understanding of frontend paths but provides little to no visibility into backend code execution.

Functionality vs Architecture Planning

The obvious use of end-user production insights is feature/ functionality planning. The often overlooked cousin, and no less important mind you, is software architecture planning. Its end goal is to optimize code implementation down to the method level. With comprehensive production insights, it’s possible to eliminate dead code and understand exactly which code areas are risk sensitive. It provides a whole new context to manage, and even better, write off our technical debt.

Production user insights and monitoring are nothing alike. Production insights are to motion sensor lights what monitoring is to a spotlight. There is a big difference between flooding a part of the application with light and lighting up the paths taken. The trick here is to keep the power consumption down = we need a cost-efficient method to gain visibility that will not result in performance degradation or incur additional code development and maintenance overhead.

Production User Insights

With the ability to track and understand exactly which code is executed in our production environment we can run correlations against the activities and decisions we are making on a daily basis. Making it easy to see if our engineering activities are aligned with our users’ activities and highlight areas in our code that require our immediate attention.

By creating a dynamic code execution blueprint, it’s easy to plan effective work, like maintaining and testing only code that is actually being used and avoid investment around dead code areas. Using production insights has accelerated our development velocity significantly by prioritizing our testing and stopping the testing of dead code areas.