2

I have been accruing some old gaming PCs that my friends no longer need, and I would like to turn them into a place to run Kubernetes clusters. I am hoping to come up with a somewhat high availability and want to be efficient in using the hardware I have. Each of these computers are full fledged desktops, CPU, GPU, decent RAM, SSD, disk, personal power supply. I see two options and was wondering what is going to be the best for my goals:

  • Keep each computer as an individual, run Ubuntu server on each one, and then connect each one into a cluster as a node. Questions about this approach:
    • This would restrict my clusters to being as many nodes as I have computers.
    • Can a computer run as a node on several different clusters? For example can I have machine 1 as a control plane node for cluster 1 and a worker node for cluster 2?
    • It is my understanding that running a cluster like this on bare metal squeezes more out of your hardware, is this accurate?
  • Attempt to raid together the SSD's and disks and run Proxmox or some other hypervisor as a main OS. Then run VM's as the nodes of the Kubernetes clusters.
    • Is this possible? Can Proxmox use the distributed (and notably different CPU/GPU/RAMs) and provision VMs across it?
    • Can Proxmox efficiently use these resources without much overhead? Will it be able to interface with the GPU's and split them among the Kubernetes clusters as well?
    • Is it possible that a VM might get provisioned on hardware from different computers, like a CPU from computer 1 and GPU and RAM from computer 2? Could this hurt my availability as one of the computers experiencing an outage could take down almost all of the VMs rather than just the ones provisioned on it?

I am just starting my at home server and K8s journey so any advice would be appreciated.

drjellybean
  • 71

1 Answers1

5

Found a good answer on the Kubernetes Slack, was probably a better place to post. Here it is for anyone looking for the answer: Both approaches have their advantages and drawbacks, so the best choice for you depends on your specific goals and requirements. Let's briefly compare the two options:

Running Ubuntu server on each computer (bare metal nodes):
Pros:

  • Easier setup and less complexity.
  • More efficient, as there is no virtualization overhead.
  • Better performance for hardware-dependent workloads, as the hardware is directly accessible.

Cons:

  • Limited flexibility in managing resources.
  • Less efficient use of hardware resources, as each computer is treated as an isolated unit.
  • Potential underutilization of resources.

Running Proxmox and virtualizing nodes:
Pros:

  • Better resource utilization, as resources can be allocated and balanced across nodes.
  • Increased flexibility in managing resources and deploying workloads.
  • Easier to scale and manage in the long run, as additional hardware can be added seamlessly.

Cons:

  • More complex setup and management.
  • Some performance overhead due to virtualization.
  • Potentially steeper learning curve, especially if you're new to virtualization.

If you prioritize simplicity and efficiency, the bare-metal approach might be the better choice. However, if you want more flexibility and scalability, using Proxmox for virtualization could be the way to go. Keep in mind that running a Kubernetes cluster at home can be quite a learning experience, so consider what you'd like to learn and what kind of challenges you're willing to face while setting up and managing your cluster.

Hannu
  • 10,568
drjellybean
  • 71