We developed our feature management tool to provide tech teams with the capabilities to deliver frictionless customer experiences more effectively. Naturally, we also use this tool at AB Tasty, but in the past, we also had to master our development cycles without the tool.
In this article, I’d like to give you insight into how our tech teams’ work has changed thanks to our Feature Experimentation and Rollouts solution. How did we work before? What has changed, and why do we appreciate the tool? Without further ado, let’s find out!
What a typical development cycle without our feature management platform looks like
The beginning of a typical development cycle is marked by a problem, or user need that we want to solve. We start with a discovery phase, during which we work towards a deep understanding of the situation and issues. This allows us to ideate possible solutions, which we then validate with a Proof of Concept (POC). For this, we usually implement a quick and dirty variant – the Minimum Viable Product (MVP) – which we then test with a canary deployment on one or two clients.
When the solution seems to be responding to customer needs as intended, we start iterating the MVP. We’re allocating more resources to the project to get it into a robust, secure, and user-friendly state. During this process, we alternate between developing, deploying, and testing until we feel confident enough to share the solution with our entire user base. This is when we usually learn how most of our users react to the solution and how it performs in a realistic environment.
The pitfalls of this approach, or: Why we developed a server-side solution
Let’s see why we weren’t happy with this strategy and decided to improve it. Here are some of the main weaknesses we discovered:
Unconvincing test results.
A canary release with one or two clients is great for getting first impressions but doesn’t provide a good representation of the solution’s impact on a larger user base. We lacked qualitative and quantitative test data and the ability to use it simply and purposefully. Manual trial and error slowed us down, and our iterations didn’t always produce satisfactory results that we could rely on.
Shaky feature management.
Developers were often nervous about new releases because they didn’t know how the feature would behave under a higher workload. When something went wrong in production, it was always incredibly stressful to go through our entire deployment cycle to disable the buggy code. And that’s just one example of why we needed a proper feature management solution.
We see tech teams around the world know and fear the same difficulties. That’s why we created a server-side and feature flagging solution to help them – and us – innovate and deliver faster than ever before while reducing risks and headaches.
I spoke to some of my tech teammates to determine how their work lives have changed since we started using our new tool. I noticed some major themes that I’d like to share with you now.
We know the impact of a new feature
Our product teams need to make a clear connection between a business KPI and a feature under test. In the past, we’ve fiddled with Google Analytics for a rough idea, but without distinct control and test groups for our experiments, we couldn’t know if our changes made a difference.
With our feature management platform, we no longer have to guess and can follow a scientific approach. We now know for sure whether a business KPI is positively impacted by the feature in question.
Suppose we publish a new feature while the marketing team starts a campaign without us knowing about it. We may get abnormal test results such as increased traffic, engagement, and clicks because of this. The problem: how can we measure the real impact of our feature?
The platform lets us define control groups to reduce this risk. And thanks to statistical modeling (Bayesian statistics), we get accurate data from which we can make a reliable interpretation.
The discovery phase lives and dies with qualitative information – but how can you get reliable data? Our answer is to conduct controlled experimentation and progressive deployments.
One time, we worked on a new version of one of our APIs and used our server-side solution for load testing. Fortunately, we found that the service crashed at some point as we gradually increased the number of users (the load). The problem wasn’t necessarily the feature itself. It had to do with changes in the environment, which can be easy to miss with traditional web testing strategies. However, we could stop the deployment immediately and prevent end-users or our SLAs with customers from being harmed by the API changes. Instead, we had the opportunity to further stabilize the API and then make it available to all users with confidence.
We iterate faster by decoupling code releases from feature deployments
We often deploy half-finished features into production – obviously, we wrap them in feature flags to manage their status and visibility. This technique allows us to iterate so much faster than before. We no longer have to wait for the feature to be presentable to do our first experiments and tests. Instead, we enjoy full flexibility and can define exactly when and with whom to test.
Additionally, we no longer have to laboriously find out who can see what in production during feature development, as we don’t have to integrate these things into our code anymore. Instead, we use the Decision API to connect features with the admin interface through which we define and change the target groups at any time.
What’s more, everyone in the team can theoretically use this interface and see how the new feature performs without involving us developers. This is a huge time saver and lets us focus on our actual tasks.
“Our Feature Experimentation and Rollouts solution helps me take back control of my own features. In my old job, I was asked to justify what I was doing in real-time, and I sometimes had trouble getting my own data in terms of CDP MOA, now I can get it.”
Julien Madiot, Technical Support Engineer
We can rely on secure deployments
Proper feature management has definitely changed how we work and how we feel about our work. And by managing our feature flags with our feature flagging platform, the whole process has become much easier for our large and diverse teams.
EVERY feature has to be wrapped in a feature flag – this is possibly one of our most essential rules in development. But it pays off:
- They’re ON/OFF switches. Let’s not lie: we still make mistakes or overlook problems. But that’s not the end of the world. Especially not if our code is enclosed in a feature flag so that we can “turn it off” when things get hairy! With our feature flagging platform as our main base for feature management, we can do this instantly, without code deployments.
- They help us to conduct controlled experiments. We use feature flags to securely perform tests and experiments in real-world conditions, aka in production. A developer or even a non-tech team member can easily define, change, and expand the test target groups in the dashboard. Thanks to this, we don’t have to code these changes or touch our codebase in any way!
They cut the stress of deployments. Sometimes we want to push code into production, but not yet for it to work its magic. This comes in handy when a feature is ready, but we’re waiting for the product owner’s final “Go!”. When the time comes, we can activate the feature in our dashboard hassle-free.
DevOps engineers have many responsibilities when it comes to software delivery. Managing our feature flags with our server-side solution is an effective way to lift the burden off their shoulders:
I honestly sleep better since we started using our server-side solution 🙂 Because I’m the one that puts things in production on Thursdays. When people say ‘Whoops, we accidentally pushed that into production,’ now I can say, ‘Yeah, but it’s flagged!’
Guillaume Jacquart, Technical Team Leader
Wrapping up
I hope you found the behind-the-scenes look at AB Tasty both interesting and informative. And yes, if there was any doubt, we actually use AB Tasty’s Feature Experimentation for all AB Tasty feature development! This helps us improve the product and ensure that it serves its purpose as a valuable addition to modern tech teams.
