Upgrading kubernetes 1.30.x to 1.31.x with kubespray
I recently used release v2.27.0 of kubespray to upgrade my kubernetes cluster and I ran into an interesting issue. Control Plane Upgrade Failure I just kicked off the upgrade as I usually do: > ansible-playbook -i inventory/home/hosts.yaml -b upgrade-cluster.yml And it failed on...
Gentoo: Converting systemd networkd with wpa_supplicant to NetworkManager with iwd and iwlwifi issues
iwlwifi module issues I came back home one day and my machine running gentoo was having issues connecting to the internet. I looked at the logs and I saw the following: Jan 05 17:44:16 nuc systemd-networkd[645]: wlp0s20f3: Gained carrier Jan 05 17:44:16 nuc wpa_supplicant[878]: wlp0s20f3:...
velero with rook ceph CSI
After switching to rook-ceph I needed to update my velero configuration to make sure I can take backups of my PVCs. In the process I ended up learning a lot about the velero backup flow. Velero Snapshot Data Movement By default velero...
Using rook ceph
With the recent news about openEBS getting archived by CNCF OpenEBS: Lessons We Learned from Open Source, I decided this was the time to try out another storage provider and I decided to try rook-ceph. I’ve heard so much about ceph in the past but I never really...
Kubespray random networking troubleshooting
I did my regular kubespray upgrade and after the upgrade some things stopped working. Here are some issues I ran into. DNS failures After the upgrade I started seeing the nodelocaldns pods show a bunch of permission denied messages: As...
Simple Pipelines with the Harness CI module
I have been using Tekton for a while now and I wanted to try another tool just to compare. After some research I decided to try harness. They have a free offering with some limits, but I think the limits are pretty good. One last note is...
Sending Logs To Loki
I had a couple of use cases, let’s try to cover them one by one. Sending Kubernetes Logs and events to Loki There are instructions in Send Kubernetes metrics, logs, and events to Grafana Cloud with Helm and Argo CD. I was surprised to see an example for...
Upgrading pfSense to 2.7.x
I tried to do my usual upgrade of pfSense from the UI but it timed out and got me into a weird state. So I logged into the device and tried to perform the upgrade manually: [2.7.0-RELEASE][root@pf.kar.int]/var/log: pfSense-upgrade -d ld-elf.so.1: Shared object <span...
Migrating From ESXi to Proxmox
As I kept migrating to kubernetes, I stopped needing VMware to host my VMs. All I needed was just a hypervisor that runs the VMs, but I didn’t need the fancy bells and whistles any more. I kept reading good things about proxmox and I wanted to give it...
Kubespray Post Install Operations
I’ve been using kubespray for a couple of months now and I’ve run into a couple of corner cases that I wanted to share. Changing the OS of the worker nodes I initially started with ubuntu and I quickly realized there are just too many updates to keep...