↓ Skip to main content
Janusworx

2026-11-06: Notes and Research, while I fiddle with the servers

Notes-and-Miscellanea

Collection of notes over the past week when I attempted to try move my Hetzner VM to some immutable operating system and then decided not to. Update: I seem to have buggered this up as well. I needed to write down the failure as to why I have not move and I cannot seem to find that post anywhere. Sigh …

2026-10-22
#

Temporarily writing here while I move servers.
#

These will move to my main blog when I’m done

2026-10-22
#

Notes while I fiddle with the servers (2026-10-22)
#

The Big Idea
#

  • Move from Debian to Flatcar on the main server.
  • I wanted to move to something completely black box and appliance like Talos, but I am not that brave. I still want my K3s, and my instance is not that large that I can run full fat Kubernetes.

2026-10-22 09:45
#

  • Good I decided to review my files. If I had just generated a new instance on Hetzner, I might have paid double for the instance I use (CPX32)
  • Moved my terraform state from s3 to local.
  • S3 meant my self hosted Garage instance. Found that I did not use it as much as I intended. So will retire it. (That will probably be phase two when I revamp the home network). Since last December I’ve hardly touched the state of the VM. So makes sense for me to maintain state locally.
  • Found that OpenTofu can encrypt at rest local tf state files. Time to move from Terraform to OpenTofu
  • 2026-10-22 10:45: Buggered it up enough, between backups and moving VMs that the current thing will destroy my VM. that is what I don’t want! will need to see what to do. But the dream of just changing enough stuff is gone now.
  • my dream of an entirely gitops driven workflow is undone by the ai-pocalypse
  • options? am too used to this machine to downgrade. so will manually nurse it.
  • worst case option? will downgrade if it comes to that.

2026-10-22 11:05
#

  • Just realised that the PHP world is on fire.
  • Taking a break from doing things above to adapting two images for my containers
  • 2026-10-22 11:31: mistakenly created the wrong server. twice Grr!
  • 2026-10-22 12:17: New server up and running. Had to change my ip in just two places. Good boy, old Jason for using variables everywhere. Break time.

2026-10-22 13:20
#

  • Back to thinking ’bout what to do with Flatcar now
  • 2026-10-22 13:55: Just realised I should go back and figure which of my workloads run PHP, and then upgrade them to the latest safe images/versions
    • Sendy (14:37: done!)
    • Baikal (17:45: done! vendored and adapted the unofficial, yet official ckulka image)
    • Kanboard (left as is and will let flux automatically handle the upgrade. fguillot is pretty good at keeping his work, shipshape) (Update 2026-10-23 11:25): I was right. A new image shipped and the cluster moved to the patched version overnight

2026-10-22 17:50
#

  • enough for today. will figure out the new node and flatcar tomorrow.

2026-10-23
#

Notes and Research, while I fiddle with the servers (2026-10-23)
#

The Big Idea
#

  • Move from Debian to something on the main server.
  • Noodling on what that something might be …
  • Two main yet conflicting needs are
    1. It should be stable and boring and well known like Debian
    2. It should be amenable to being controlled with code
  • One approach will be to just having an OS that is completely configurable.
    NixOS beckons. But I think that is a project for a machine at home, not for something that I want to run on the wide net and in production. (this is just a matter of comfort and opinion. Not fact. Other folks routinely do run workloads on the net with NixOS)
  • The other approach is what I’m doing. Nothing matches what I’m looking for more than Talos. But I’m a bit hesitant, because at least now, Talos (despite the MPL) is Sidero Labs. Not community owned like Debian.
  • After a bit of thought, well, if this is the risk to take, I’ll take it. And see how it goes. After all, I say I want something long term, but in the long term I figure out said something’s shortcomings (like I hate not being able to update Debian like an appliance) and then want to switch anyway.
  • So, I already learned all about containers and how to create and write them last year. So this year, the big change, in addition to Talos is Cilium.
  • The other thing I’m dropping is provisioning the Hetzner instance with OpenTofu/Terraform. A single node does not quite warrant that level of work.
    Am pushing this to when I scale this up to AWS.
  • Think the VM should handle this. My current idle load seems to hover at a few % cpu and about 50% of my 8gb ram

Tentative Plan
#

  • Create a Hetzner instance with Talos and Kubernetes on it
  • Cilium
  • Flux
  • Traefik
  • Letsencrypt
  • Move workloads from the other cluster to this one. (Examine and drop the ones I don’t need)

2026-10-23 17:20
#

  • Got the instance up with Talos installed. Now to figure out more next week.

Feedback on this post?
Mail me at feedback at this domain.