ops

Using git hooks to trigger Rundeck jobs

April 27, 2017
rundeck, puppet, hiera, automation, bash, sysadmin, ops

Using git hooks to trigger Rundeck jobs # 2017-04-27 At work, we keep our hiera yaml files in a git repo (encrypted using the excellent hiera-eyaml backend). I got really tired of doing a git pull on the puppetmaster each time I made a change to a hiera file. So, I wanted to set up a way to pull these changes automatically each time I did a commit. The first thought I had was to just set up some keys so the git server could ssh to the puppet master and run a git pull. ...

Some thoughts on toil

January 1, 2017
sysadmin, ops

Some thoughts on toil # 2017-01-01 The importance of distinguishing toil from other types of work and understanding its impact didn’t really click with me until recently when I read the Google SRE book. Toil is an insidious type of work. It’s impossible to completely eliminate, can be difficult to spot, and can have a huge impact on your productivity. If left unchecked, it can grow to consume the majority of your time meaning you’re left with little or no capacity for work which improves your environment. ...

Controlled Server Demolition

April 28, 2016
linux, sysadmin, ops

Controller Server Demolition # 2016-04-28 Today I tasked with one of those jobs which don’t come up very often - shutting down a service for good. This particular retirement posed an interesting challenge though. As per usual, I wanted to make sure that all traces of customer data was wiped from the systems, and, indeed, any of our own keys/users/passwords and so on. The challenge was, though, that I only had ssh access to the systems. ...

Latency And Mobile Sites

May 28, 2014
mobile, ops, sysadmin

Latency And Mobile Sites # 2014-05-28 I wrote an article for MobiForge entitled Less Is More - Why You Should Care About Latency For Your Mobile Site.

An Empirical Study of SNI Support in Different HTTP Clients

October 5, 2013
https, sni, legacy, linux, sysadmin, ops, tls

An Empirical Study of SNI Support in Different HTTP Clients # 2013-10-05 There’s a bit of a chicken and egg situation inherent in the way that SSL works. This can make hosting multiple SSL domains/certs on the same host problematic. Before an SSL client can make a request it must handshake with the server to set up a secure connection. Part of the handshake process is the server presenting it’s SSL cert to the client. ...

Automatically Signing Apt Repos with gpg-agent

August 7, 2013
apt, ubuntu, sysadmin, ops

Automatically Signing Apt Repos with gpg-agent # 2013-08-07 This is an enormous pain in the ass to get working correctly. To hopefully save you some butt pain, here’s how you actually do this, from end-to-end. So, the story is, you want to build an apt repo of your own stuff. In order to stop apt-get complaining every time you install something from this repo, you need to set it up as a signed repo. ...

Zabbix Agent Over an ssh Tunnel

December 4, 2012
sysadmin, ops

Zabbix Agent Over an ssh Tunnel # 2012-12-04 Today I set up Zabbix monitoring of a bunch of boxes. A couple have public IP addresses - the load-balancers - so they were pretty standard. However, most of them are sitting behind a NAT, so are a little trickier. I played around with Zabbix proxy for a while - whose purpose is to solve this exact problem. In the end though, just to be different/awkward, I opted to set up ssh tunnels and just pass the Zabbix traffic through the load-balancer boxes. ...

Failover Squid via HAProxy

November 29, 2012
sysadmin, ops

Failover Squid via HAProxy # 2012-11-29 At the moment I’m using Squid quite a bit as a forward proxy. The application in question pulls content from remote sites and does some processing on it. It’s handy to have a copy of the site ’nearby’ in case further processing is needed. So, the content is pulled through Squid for later use. Obviously, a single squid instance is no good. If it goes down, everything grinds to a halt. ...

root-kit-a-rama

February 14, 2011
security, sysadmin, ops

root-kit-a-rama # 2011-02-14 Last week, I got a complaint that one of our webservers, hosted in EC2, was responding very slowly. After some fiddling around, I could eventually get ssh access, the box was just dragging along a bit. So, I check the uptime and it’s hovering around 6. I check top and there are a few perl processes chewing up the CPU. At first I think this is some backend web app stuff - some DB processing or something. ...