Multi-tenant Feature Flag Manager

I’ve been searching on the Internet for some already builtin Feature Flag Manager for multi-tenant systems, with not much luck. There are some companies and products that offer Feature Flag as a Service (let’s call it like that), which permits managing this contraption for your microservice architecture, but unsure about if they support multi-tenancy.

In this article, I’ll propose a system for a real multi-tenant Feature Flag Manager system. Engage!

Feature Flag

From the Wikipedia

A feature toggle (also feature switch, feature flag, feature gate, feature flipper, conditional feature, etc.) is a technique in software development that attempts to provide an alternative to maintaining multiple branches in source code (known as feature branches), such that a software feature can be tested even before it is completed and ready for release

https://en.wikipedia.org/wiki/Feature_toggle

These days, where microservices and fast development are key, the concept of Feature Flag becomes something really important. Moreover, the capability for deploying new features, very fast, in production, cannot happen without any control.

Multi-tenant system

In this section, I continue writing some context, before going to the proposal. Let’s define “Multi-tenant” concept. The one provided by Wikipedia, it’s enough valid for this article:

The term “software multitenancy” refers to a software architecture in which a single instance of software runs on a server and serves multiple tenants.

https://en.wikipedia.org/wiki/Multitenancy

If we get this concept down

• “a single instance” could be a container or Pod for each microservice of your system.
• “runs on a server and servers” could be a cloud environment driven by either virtual machines or a Kubernetes cluster.
• “multiple tenants” could be each of the customers that are using your system.

In this article, the tenants are the customers for a single (your) company.

Start with the why

Different features apply to different Tenants, based on multiple reasons. Only our beloved beta customers are willing to try “that” new feature that we plan to deliver this week.

The goal pursued in this proposal is that an undetermined number of microservices are able to act differently, based on the feature flags they are aware of, for the features they have to offer to the different Tenants in the whole system.

Proposal

In this section, you will find the description of the system proposed to satisfy the use case indicated in the previous section.

First, we have the Operator of this Feature Flag Manager using the brand new UI to manage each feature flag for each of the customers (tenants). This UI application has, of course, a backend to rule the actions. To know which feature changes for which Tenant, a special header is propagated and contains the Tenant-ID, which is the unique identifier of a tenant. All the information about the tenants should come from some kind of Customer Database.

Second, the Feature Flag Manager backend acts as both producer and consumer with a message-broker system. The information about each feature flag, together with the Tenant-ID, flows across the message-broker until “the other side”. This “other side” is a microservice, let’s call it Feature Flag Orchestrator, that consumes what the Feature Flag Manager backend produced. This microservice is in charge of ensuring the correct storage of what it consumes.

Third, the storage. At this point, the Feature Flag Orchestrator must ensure the storing of the feature flags. For this, we can use a Key/Value storage system, using the Tenant-ID as the key.

Finally, we have the list of microservices used in some way by the customers, the tenants. These microservices will work in a different way based on the tenant that requests the service. A tenant can be identified by a Tenant-ID, as we said before. With this information, each of the microservices can ask for the set of feature flags (os specific feature flag) for the request of the tenant at that time.

Architecture

Technology

In this chapter, I’ll describe the technology that I’d choose to implement this proposal.

Environment

I would deploy all microservices in a Kubernetes cluster managed by some Cloud Provider, such as AWS or Azure. Other services, like the message-broker, the Customer Database, and the Consul, could be out of the Kubernetes cluster.

Microservices

Nowadays, I’m very happy and comfortable with the Java Micronaut Framework. It’s very fast for implementing microservices and the performance you get is really good in Cloud environments. Moreover, you wrap them in a HELM chart, and you have a nice and beautiful microservice ready to run in any K8S cluster.

In addition, these microservices will have READ-ONLY access to the feature flag information.

Message Broker

The fact that this proposal contains a message broker is for decoupling purposes. I want to have all different responsibilities enough separated and not blocking each other. Kind of streaming platform.

My choice here would be NATS. You may remember my previous article “Enter, The NATS” where I go a bit deep into this technology. The size of the information to move is really small, and NATS is meant for that small sizing. Moreover, the performance is really good and it’s very easy to deploy either in a Kubernetes cluster or in virtual machines. Using NATS in pub/sub mode will make this work.

However, if you already have another pub/sub system in place, just reuse it. You will save time and reduce uncertainty.

Feature Flag Storage

As I mentioned in previous sections, I propose to use a Key/Value store to save the information of the feature flags per customer (tenant). In my article “Adopting a distributed key/value storage“, I make an introspective about 3 different Key/Value storage systems, and one of them is Consul (from Hashicorp).

I consider Consul as a good choice for this proposal for many reasons. First, it’s simple to put in production. You have heard this is important for me many times. Second, it’s very resilient thanks to its distributed architecture. Above all, Consul offers the capability to watch for changes in a Key/Value. This feature facilitates the work of the microservice Feature Flag Orchestrator.

Also, it is important to reinforce that, the fact of using a Key/Value store facilitates the classification of the feature flag information across all tenants, using the Tenant-ID as key for the values.

Summary

You can see this proposal defines a streaming-oriented feature flag manager system to run in a multi-tenant context. The key pieces here are the message-broker and the storage system. For those, I selected NATS and Consul, but you can switch them by others similar.

The most strong development effort comes in the UI application, together with the deployment in production of the whole system. The implementation of the microservices to interact with NATS and Consul should be reasonably easy and fast.

I hope you enjoyed this reading as a much I did writing it.

Next Steps

A way to group the feature flag values could be useful, in the sense that you could have a group called “beta-customers” and you could change feature flags for a set of customers. I believe this is a real use case and with Consul as Key/Value store, this could work as well.

And, of course, Feature Flag Manager as a Service. Implement this system not only for your company but for any other companies as a cloud-based service.