What we’ve got – Hazardous Hosting

I took two days off work in the days immediately after the business Internet was provisioned and went through the rack. I rebuilt, reconfigured and rearranged things to make life easier. Let’s go through some of what we have in and around the rack.

The rack and power

The rack itself is an APC AR3100 or, more formally, the APC NetShelter SX 42U Server Rack Enclosure 600mm x 1070mm with sides in black.

The rack cost me nothing and it came with two APC AP8853 PDUs which are metered, mount in the rear sides of the rack so they don’t use any of the rack space and each provide 36 C13 and 6 C19 power connections.

I bought an unused HP R5000 UPS from a local computer recycler which arrived on a pallet and was briefly left in the rain. The recycler was nice and gave me a “I’m not a company and it’s going in my garage” discount.

Both PDUs and the UPS came wired to 30A plugs which a colleague of mine was kind enough to donate 3 sockets for. Two of these sockets are wired to mains power (each with a 16A circuit breaker), one goes directly to one PDU, the second provides power for the UPS and the third is wired back to a 16A socket on the UPS and powers the other PDU.

A large industrial style power socket — Three of these only they’re mounted to the wall nicely now

Networking

Networking in the rack is provided by two switches, a MikroTik CRS317 for 10G Fibre and a MikroTik CRS226 for 1G Ethernet. The CRS317 acts as the core switch for the other switches in the house, garage, as well as connecting directly to the servers. We connect back to a MikroTik RB3011 which acts as the router and firewall, however this is not currently in the rack.

A bad photo of the switches mid-cleanup of the rack

Environment monitoring

Two APC environmental monitoring units — Environmental monitoring is easier with donated hardware designed to do it

I was given an APC InfraStruXure® Manager Server (AP92200), an APC Environmental Management System (AP9320) and a more than reasonable amount of sensors to monitor the rack environment. I haven’t set these up yet so we’ll save that for another post.

Servers

There are five servers in the rack, three act as KVM hosts for public facing infrastructure, one hosts ‘essential’ but non-redundant services and one is for storage. The three KVM hosts are HP DL380 G7s, each with two CPUs, 56GB of RAM, 10G fibre and six SSDs. One for booting and the others in an array for VM storage.

The reality of running this stuff out of my garage with second hand gear is that things are going to fail. Any redundancy I can have is important as it should greatly reduce the day to day involvement needed from me and will hopefully mean I never feel obligated to go out to the garage in the rain at 3am in the middle of winter (this has happened) to fix something.

Another bad photo showing the servers, disk shelves and UPS

While there are a few things where only one VM makes sense for my purposes (like GitLab and NetBox) the rest are as redundant as possible. I’m traditionally the monitoring and backups person, so while I strive for decent uptime (balanced with being up to date with patching and new releases) I’ve also learnt to accept that knowing something redundant is broken with the service still working is better for my health than spending the night fixing it.