Install and Configure Kubernetes
MemVerge.ai supports multiple kubernetes flavors. Use the installation guides to provision Kubernetes in the AWS environment.
- K3s: Follow the K3s Quick-Start Guide.
Installing K3s
This guide follows https://docs.k3s.io/quick-start.
K3s provides an installation script that is a convenient way to install it as a service on systemd or openrc based systems. This script is available at https://get.k3s.io. To install K3s on the management node using this method, run:
Example output:
$ curl -sfL https://get.k3s.io | sh -
[INFO] Finding release for channel stable
[INFO] Using v1.31.5+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.31.5+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.31.5+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
Run systemctl status k3s to verify the service is running. For example:
$ systemctl status k3s
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2025-02-20 23:56:47 UTC; 39s ago
Docs: https://k3s.io
Process: 1227 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service 2>/dev/null (code=exited, status=0/SUCCESS)
Process: 1229 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 1231 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 1232 (k3s-server)
Tasks: 93
Memory: 1.3G
CPU: 33.197s
CGroup: /system.slice/k3s.service
├─1232 "/usr/local/bin/k3s server"
├─1257 "containerd " "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "">
├─1985 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id 2f54140>
├─2018 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id 92456dd>
├─2157 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id 5e2b356>
├─3028 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id a88ccf7>
└─3118 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id ebd8daa>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908532 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="middlewares.tr>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908548 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="tlsoptions.tra>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908563 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="ingressrouteud>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908578 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="middlewaretcps>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908597 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="middlewaretcps>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908616 1232 resource_quota_monitor.go:227] "QuotaMonitor created object count evaluator" resource="ingressroutetc>
Feb 20 23:57:22 mvai-mgmt k3s[1232]: I0220 23:57:22.908792 1232 shared_informer.go:313] Waiting for caches to sync for resource quota
Feb 20 23:57:23 mvai-mgmt k3s[1232]: I0220 23:57:23.009009 1232 shared_informer.go:320] Caches are synced for resource quota
Feb 20 23:57:23 mvai-mgmt k3s[1232]: I0220 23:57:23.323885 1232 shared_informer.go:313] Waiting for caches to sync for garbage collector
Feb 20 23:57:23 mvai-mgmt k3s[1232]: I0220 23:57:23.323937 1232 shared_informer.go:320] Caches are synced for garbage collector
To install additional agent nodes and add them to the cluster, run the installation script with the K3S_URL and K3S_TOKEN environment variables. Setting the K3S_URL parameter causes the installer to configure K3s as an agent, instead of a server. The K3s agent will register with the K3s server listening at the supplied URL. The value to use for K3S_TOKEN is stored at /var/lib/rancher/k3s/server/node-token on your server node.
Here is an example showing how to join an agent:
Example command and output:
ubuntu@mvai-nvgpu01:~$ curl -sfL https://get.k3s.io | K3S_URL=https://mvai-mgmt:6443 K3S_TOKEN=K10f125cd21ccff1f40d93fc85ee2080da622a7153b8b9d95f28ff2d8653817ee5e::server:719056123428a646178043d311a799d0 sh -
[INFO] Finding release for channel stable
[INFO] Using v1.31.5+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.31.5+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.31.5+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent
Confirm the service is running using systemctl status k3s-agent
Example output:
ubuntu@mvai-nvgpu01:~$ systemctl status k3s-agent
● k3s-agent.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2025-02-21 00:05:07 UTC; 1min 8s ago
Docs: https://k3s.io
Process: 1260 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service 2>/dev/null (code=exited, status=0/SUCCESS)
Process: 1262 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 1264 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 1266 (k3s-agent)
Tasks: 43
Memory: 347.1M
CPU: 3.850s
CGroup: /system.slice/k3s-agent.service
├─1266 "/usr/local/bin/k3s agent"
├─1295 "containerd " "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "">
└─1703 /var/lib/rancher/k3s/data/da3ffc1d30a49a23449847b31d95bf4c96c8551396573c18886c9d0c4a63c710/bin/containerd-shim-runc-v2 -namespace k8s.io -id 79e20ba>
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.225029 1266 iptables.go:125] Setting up masking rules
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.228352 1266 iptables.go:226] Changing default FORWARD chain policy to ACCEPT
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: time="2025-02-21T00:05:08Z" level=info msg="Wrote flannel subnet file to /run/flannel/subnet.env"
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: time="2025-02-21T00:05:08Z" level=info msg="Running flannel backend."
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.230707 1266 vxlan_network.go:65] watching for new subnet leases
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.230741 1266 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIP>
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.230867 1266 vxlan_network.go:100] Received Subnet Event with VxLan: BackendType: vxlan, PublicIP: 172.31.25.21>
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.243220 1266 iptables.go:372] bootstrap done
Feb 21 00:05:08 mvai-nvgpu01 k3s[1266]: I0221 00:05:08.251534 1266 iptables.go:372] bootstrap done
Feb 21 00:05:11 mvai-nvgpu01 k3s[1266]: I0221 00:05:11.046769 1266 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/svclb-traefik>
Configuring kubectl Access for Non-Root Users
After installing K3s, you may encounter permission issues when running kubectl commands as a non-root user. This section guides you through setting up kubectl access for your user account without requiring sudo privileges.
- Set up the KUBECONFIG environment variable
Add the following line to your .bashrc file to set the KUBECONFIG environment variable:
This step ensures that kubectl knows where to find your configuration file.
- Create a local copy of the K3s configuration
Run these commands to create a local copy of the K3s configuration file:
This creates a .kube directory in your home folder, copies the K3s configuration to it, and sets appropriate permissions.
- Verify the setup
Test your configuration by running:
You should now be able to see the list of nodes in your cluster without any permission errors. For example:
ubuntu@mvai-mgmt:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
mvai-mgmt Ready control-plane,master 53m v1.31.5+k3s1
mvai-nvgpu01 Ready <none> 45m v1.31.5+k3s1
Troubleshooting
If you encounter issues after following these steps, verify that the K3s service is running:
If the service is not active, start it with:
Security Note
This configuration creates a user-specific kubeconfig file, avoiding the need to modify system-wide permissions. Always ensure that your home directory has appropriate permissions to protect the kubeconfig file.