Managed Kubernetes as a Service in the Bioinformatics Cloud
The mission of the Berlin Institute of Health (BIH) is medical translation: findings from biomedical research are transferred into new approaches for personalized prediction, prevention, diagnostics and therapy; conversely, observations in everyday clinical practice lead to new research ideas.
To this end, the BIH, as a translational research unit at Charité, is establishing a comprehensive translational ecosystem, focusing on a cross-organ understanding of health and disease and promoting a translational cultural change in biomedical research. BIH was founded in 2013 and is 90 percent funded by the Federal Ministry of Education and Research (BMBF) and ten percent by the State of Berlin. The founding institutions Charité - Universitätsmedizin Berlin and Max Delbrück Center were independent members of BIH until 2020. Since 2021, BIH has been integrated into Charité as a so-called third pillar, and the Max Delbrück Center is a privileged partner of BIH.
Challenge
Providing Kubernetes clusters for bioinformaticians
The de.NBI cloud environment at the Berlin Institute of Health has been established since 2017 and provides resources for bioinformaticians as an Infrastructure as a Service. Users are able to independently create a complete environment with virtual instances, storage as well as network and load balancer within the scope of their project, either via user interface or automation. Now that containers and Kubernetes are no longer just a trend, but have really arrived in IT and thus also in bioinformatics, the demand from bioinformaticians to also operate such Kubernetes clusters on the de.NBI Cloud at the Berlin site is increasing.
Thanks to the existing options within the de.NBI Cloud, it is easily possible to provision a Kubernetes cluster yourself within a short time. However, the management, e.g. the update of such a cluster, also requires a certain amount of time, which the bioinformaticians cannot devote to their actual research work. At the same time, the operators of the de.NBI Cloud are interested in offering users the simplest possible user experience and standardized services that generate added value and not additional effort for the projects.
Solution
Kubermatic Kubernetes Platform
Together with the cloud team at the Berlin Institute of Health at Charité, the SVA experts first evaluated various approaches and solutions and the decision was ultimately made in favor of the Kubermatic Kubernetes Platform (KKP). With KKP, it was possible to build a central control plane that can be scaled flexibly and then also connect different cloud providers - in this case, the OpenStack-based cloud at BIH.
The architecture of KKP is itself based on a Kubernetes cluster, which is integrated into the existing OpenStack environment by means of so-called cloud controllers. With KubeOne, Kubermatic offers a solution to initially create and manage this base cluster. This enables the KKP platform administrators to configure the underlying infrastructure in a fully automated way, e.g. to dynamically add further resources required for the control plane.
The end users (bioinformaticians) only see the KKP user interface as a product, into which they log in with the existing OpenID Connect-based authentication to provision Kubernetes clusters directly into their assigned project within OpenStack. A key advantage is that end users do not need to have any knowledge of the underlying infrastructure, but simply choose how many resources they need and which node types (e.g. GPU, high memory) should be available in their cluster. Even updates to the clusters can be carried out by the end users with a click.
The SVA experts were able to successfully accompany the project from the creation of the concept, through the set-up and integration into the existing OpenStack environment, to operational readiness. SVA continues to support de.NBI in the further development of the environment and works with the customer as a joint "cloud team".
A decisive advantage is that the Kubernetes control plane, i.e. the masternodes are not set up as three virtual machines for each cluster, but are hosted centrally.
The administration of the clusters can be done completely by the users after the projects have gained access to KKP, which is extremely end-user friendly.