xor22h.{dev}

Kubernetes on a Budget: Running a Robust Cluster for Under 80€ a Month with K3S

Julius Lisauskas -

Kubernetes is often seen as a powerful but expensive platform for orchestrating containers. However, with the right approach and choice of tools, it's possible to run a fully functional Kubernetes cluster without breaking the bank. In this blog post, I'll share how I run a Kubernetes cluster for under 80€ a month using Hetzner as my cloud provider. We'll delve into persistent data solutions with Hetzner CSI, and the powerful monitoring and logging stack composed of Grafana, Prometheus, Loki, and Promtail. Additionally, I'll discuss setting up OpenVS Code Server for remote development. The key to making this all work efficiently and cost-effectively is using K3S, a lightweight Kubernetes distribution.

Why Hetzner?

Hetzner is a popular European cloud provider known for its competitive pricing and reliable infrastructure. By choosing Hetzner, you get robust hardware and excellent network performance at a fraction of the cost of other major cloud providers.

Why K3S?

K3S is a lightweight, fully compliant Kubernetes distribution by Rancher Labs. It's designed for resource-constrained environments, making it perfect for running on a budget. K3S reduces overhead by bundling required components into a single binary, using SQLite instead of etcd by default, and supporting external databases.

Setting Up Your Cluster with K3S

To get started with K3S on Hetzner, you'll need to follow these steps:

  1. Create a Hetzner Account: Sign up and set up your billing information.
  2. Deploy Virtual Machines: Provision a few virtual machines (VMs) to form your Kubernetes cluster. Depending on your needs, you can start with a basic setup of three VMs: one master and two worker nodes.
  3. Install K3S: Use the K3S installation script to set up K3S on your VMs. The lightweight nature of K3S makes the setup process quick and easy.
curl -sfL https://get.k3s.io | sh -

Persistent Data with Hetzner CSI

For applications requiring persistent storage, Hetzner's Container Storage Interface (CSI) driver is an excellent choice. Hetzner CSI allows you to dynamically provision persistent volumes backed by Hetzner Cloud Volumes. This is how you can set it up:

  1. Install Hetzner CSI: Deploy the Hetzner CSI driver in your Kubernetes cluster. This typically involves applying a set of YAML files provided by Hetzner.
  2. Create Storage Classes: Define storage classes that specify the types of volumes you want to use.
  3. Provision Persistent Volumes: Use persistent volume claims (PVCs) in your applications to automatically provision and manage storage.

Hetzner CSI makes managing persistent storage straightforward and integrates seamlessly with Kubernetes, ensuring your data is safely stored and easily accessible.

Monitoring and Logging with Grafana, Prometheus, Loki, and Promtail

A robust monitoring and logging stack is crucial for maintaining the health and performance of your Kubernetes cluster. I use Grafana, Prometheus, Loki, and Promtail for this purpose.

Prometheus and Grafana for Monitoring

  1. Install Prometheus: Prometheus is a powerful monitoring and alerting toolkit. Deploy it in your cluster to collect metrics from your applications and Kubernetes itself.
  2. Set Up Grafana: Grafana provides a beautiful and customizable dashboard to visualize metrics collected by Prometheus. You can deploy Grafana alongside Prometheus and configure it to use Prometheus as a data source.

Loki and Promtail for Logging

  1. Deploy Loki: Loki is a log aggregation system that integrates with Grafana. It's designed to be cost-effective and easy to operate.
  2. Install Promtail: Promtail is an agent that ships the contents of local logs to a Loki instance. Deploy Promtail on your Kubernetes nodes to forward logs to Loki.

With this setup, you can easily monitor the health of your cluster, analyze logs, and set up alerts to notify you of any issues.

Remote Development with OpenVS Code Server

OpenVS Code Server allows you to run Visual Studio Code on a remote server and access it through a web browser. This is particularly useful for development within a Kubernetes cluster, as it lets you work directly within your deployment environment.

Setting Up OpenVS Code Server

  1. Deploy OpenVS Code Server: You can deploy the OpenVS Code Server as a Kubernetes deployment. This will allow you to access VS Code through your browser.
  2. Configure Access: Set up authentication and configure the server to ensure secure access to your development environment.
  3. Connect to Your Cluster: Use the OpenVS Code Server to connect to your Kubernetes cluster, enabling you to develop and debug applications directly in your production environment.

Consider ARM Nodes

To further optimize costs and improve efficiency, consider using ARM-based nodes. ARM processors are known for their energy efficiency and lower costs compared to traditional x86 processors. Integrating ARM nodes into your cluster can provide significant cost savings and performance benefits, especially for certain types of workloads.

Mixed Architecture Cluster

In my setup, I use a mixed architecture cluster with some nodes being ARM and others being AMD-based. This hybrid approach allows me to leverage the strengths of both architectures, optimizing for cost, performance, and energy efficiency. Here are some steps to integrate ARM nodes:

  1. Provision ARM Nodes: Hetzner offers ARM-based servers, which can be provisioned alongside your existing AMD nodes.
  2. Configure K3S for Mixed Architecture: K3S supports multi-architecture clusters, allowing you to seamlessly integrate ARM nodes into your existing setup.
  3. Deploy Workloads: Deploy your workloads, taking advantage of the cost-efficiency and performance benefits of ARM nodes.

Conclusion

Running Kubernetes doesn't have to be expensive. By choosing Hetzner as your cloud provider, leveraging tools like Hetzner CSI, Grafana, Prometheus, Loki, Promtail, and OpenVS Code Server, and utilizing K3S, you can maintain a cost-effective yet powerful Kubernetes environment. Incorporating ARM nodes into your cluster can further optimize costs and improve performance.

With careful planning and the right tools, you can keep your monthly costs under 80€ while enjoying the full benefits of Kubernetes for your applications. Remember, the key is to optimize resource usage, choose the right tools, and continuously monitor your setup to ensure everything runs smoothly. Happy orchestrating!