k3s is a lightweight Kubernetes distribution “for IoT and Edge Computing”.


dnf install -y container-selinux selinux-policy-base
rpm -i https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm

curl -sfL https://get.k3s.io | sh -
# wait....
k3s kubectl get node

You’ll need to be root to interact with the cluster:

sudo k3s kubectl get node


Kill, uninstall:



How DNS works

  • Uses coredns using image docker.io/rancher/coredns-coredns
  • This runs as a ReplicaSet in namespace kube-system
  • CoreDNS gets settings from the configmap coredns in namespace kube-system - this is mounted as /etc/coredns/Corefile in the container
  • Each container uses CoreDNS for DNS resolution, due to nameserver <kube-dns Service IP> in /etc/resolv.conf

How Networking/Ingress works

  • IP addresses:
    • Pod IP addresses are allocated from a default CIDR of (this can be configured on startup with the --cluster-cidr option) 1
    • Service IP addresses are allocated from a default CIDR of
  • ServiceLB (formerly known as Klipper) is used for load balancing. It watches Kubernetes Services with the spec.type field set to LoadBalancer. 2
  • ClusterIP service and Ingress: To expose an app outside the cluster, you can just create a Service of type ClusterIP and expose it with an Ingress.
  • LoadBalancer service: Alternatively, create a Service of type LoadBalancer. This will create a new klipper load balancer DaemonSet (svclb-*) in the namespace kube-system. However, your Service must expose a port which isn’t already in use. For example, Traefik occupies ports 80 and 443, so pick a different port.

Middlewares with Traefik

Example Ingress and Middleware for an HTTPS service

You can also tell Traefik to redirect clients to HTTPS by creating a Middleware resource:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
  name: redirect-https
    scheme: https
    permanent: true

Then create the Ingress, with the appropriate annotations so that Traefik will use it. This assumes you’re using cert-manager in the cluster to issue Let’s Encrypt TLS certificates:

apiVersion: networking.k8s.io/v1
kind: Ingress
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/router.middlewares: default-redirect-https@kubernetescrd
  name: myapp
  namespace: plausible
  - host: myapp.example.com
      - backend:
            name: myapp
              number: 8000
        path: /
        pathType: Prefix
  - hosts:
    - myapp.example.com
    secretName: myapp-tls


Monitoring with Kubernetes Dashboard

Add the Kubernetes Dashboard:

VERSION_KUBE_DASHBOARD=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
sudo k3s kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml

Using k9s with k3s

If you want to view the cluster using the excellent k9s, you will need to pass the location of the rancher kubeconfig file:

k9s --kubeconfig /etc/rancher/k3s/k3s.yaml


Some more info about k3s:


metrics-server fails, with error “Failed to scrape node … http://your_ip:10250 no route to host”:

  • Likely firewall is preventing traffic from the pod network to your host.
  • Add a rule: firewall-cmd --permanent --add-port=10250/tcp && firewall-cmd --reload

Can’t seem to use kubectl with any other Kubernetes clusters except k3s: “error: error loading config file “/etc/rancher/k3s/k3s.yaml”: open /etc/rancher/k3s/k3s.yaml: permission denied”:

  • The kubectl that’s distributed with k3s is modified to always load config from /etc/rancher/k3s/k3s.yaml 3
  • But the default behaviour for a regular kubectl binary is to read config from ~/.kube/config.
  • To work with other clusters: install your own kubectl binary (e.g. from the Kubernetes website, or via a package) then symlink it, so that it overrides the k3s kubectl binary in your PATH:
    • sudo dnf install -y kubernetes-client --repo fedora
    • ln -s /usr/bin/kubectl ~/.local/bin/kubectl
  1. https://docs.k3s.io/reference/server-config 

  2. https://docs.k3s.io/networking#service-load-balancer 

  3. https://github.com/k3s-io/k3s/issues/1541#issuecomment-672099924