21 Months later


It has been 21 months since I wrote the 36 Months post and despite the number of drafts I have slowly brewing, I’ve never gotten around to actually hitting the publish button on a third post.

While this post is meant as a follow up to both previous posts with some quick updates, I’ve also decided to amalgamate my other hobbies and interests here as well.

Both for the sake of my sanity having fewer websites to maintain and to get more things documented for when the bus factor eventually hits. Almost all of my hobbies are linked to the HH environment or its components in some way.

I’ve got draft posts for building our own heat transfer system, our household renovations, building lighting control systems, modifying robot vacuums, fixing old tape robots, stalking pets and other random bits of DIY.

Looking back

In the previous posts I talked about my cycle of hosting things externally and then bringing them back to being self hosted, only to ship them out and bring them back over and over and over again.

Despite now having significant credit towards cloud hosting I have very little left outside of the HH environment. DNS is is going to stay external along with Disaster Recovery type hosts. There are a few other servers I’m hoping to move or replace in the near future but nothing particularly substantial.

I can happily say that our internet connectivity has been incredibly reliable, aside from one significant overnight outage caused by a fibre cut during the unplanned and poorly executed relocation of our ONT during some renovations.

Our power supply has also been incredibly reliable with almost all of its outages being caused by our house being re-wired or by firmware bugs in the UPS.

What’s new?

Personally I now work for a cloud provider, work from home, have no commute, am much more involved at home, have more time for hobbies and generally have a much better grasp on my life and ability to get things done.

Physical Changes

We’ve made a few physical changes including:

  • Building a wall between the HH Server Room and the rest of the garage
  • Upgrading our camera system
  • Installing a new alarm system
  • Adding a “for fun” access control system to the HH Server Room
  • More permanently installing the portable AC unit
  • Installing better heat extraction

Equipment and Hardware

We’ve also acquired some new things, broken some things, replaced some hardware and made other changes. These include:

  • Improving our networking infrastructure
  • Working towards having IPv6 parity
  • Added a second server rack
  • Adding UPS Bypass hardware and Automatic Transfer Switches for things without redundant power supplies
  • Moving from Ubuntu and KVM on three servers to five ProxMox VE hosts with SSD only hyper-converged Ceph storage
  • Starting to move away from having many low capacity hard drives in various disk shelves towards fewer higher capacity hard drives in a more appropriate server case
  • Working towards accepting that my love of high availability and fault tolerance doesn’t mean we have the funding of a tech giant
  • Adding a bunch of RAM and storage to the ProxMox VE cluster