In 2016, the engineering department here at Administrate grew significantly. At the start of the year we had just a handful of developers, and it quickly became apparent that to continue to develop and build our product in a reliable and sustainable way, we had to grow the team, and quickly.
Over the course of the year we hired a dozen more engineers, several more product people, and a project manager. This, of course, brought its own challenges!
- How do we get engineers on-boarded, and up to speed quickly?
- How do we deal with planning for this many people?
- How do we ensure we can scale our quality assurance process to support this?
- How do we communicate across the team?
- Do we need to split into smaller teams to focus on dedicated areas of work?
- How do we ensure other business needs and internal projects are being serviced?
That, in itself, is probably the subject of another series of articles, but for now I want to talk about 2017 and what we've learned now that the team has had a chance to settle down, and begin to gel together.
Sharing is Caring
The human brain starts working the moment you are born and never stops until you stand up to speak in public.
-- George Jessel
We did - as you would expect - need to split into several smaller teams to make this work. Immediately after LITE last year, we split the 13-strong team into two smaller teams. Arguments about what the teams should be called aside, this worked well, particularly when it came to reducing the load on those responsible for managing planning sessions (sorry, Iain!).
Team Cobalt and Team Lithium now operated as completely autonomous units, each with their own SCRUM masters, and shortly after, their own testers. Soon after, we added Team Zinc (spot the pattern!). One of the challenges of this was ensuring that we could have effective communication across the teams. We've struggled in the past with team members who've been here significantly longer becoming 'knowledge silos,' to the point where nobody else is comfortably able to work on certain parts of the product.
We've a few things in place to help deal with this now. The SCRUM masters have a weekly meeting, called the SCRUM of SCRUMs, where issues that each team is facing can be discussed, whether that be planning problems, or process issues. The recently minted 'mentor' role within each team also catches up regularly to ensure that we're sharing engineering knowledge and practice across the teams. These meetings are important, and because at Administrate we believe so firmly in #transparency, the meeting notes are written up and shared with the entire engineering department.
Outside of these regular, 'directed' meetings, we also have weekly lunch-and-learn sessions, in which team members can share knowledge and experience with some of the technology we're using, or where folk can talk about new concepts that their team is using in their latest project. This time slot is also used for our engineering 'book club,' where we split into team-agnostic groups and work through books, discussing what we're learning. This is a great way to ensure that people from different teams are interacting and sharing thoughts and ideas.
The chicken is just not going to be able to cross the road this month. Crossing requirements were due last Friday. She will have to take her place on the backlog. Maybe the chicken can cross the road in Sprint 9.
In a very small team, it can be easy to work without a lot of process in place. Collaboration is effortless when only two or three engineers are involved, but this changes quickly when you scale up to a team of almost 20. We need to be able to do our day-to-day jobs in a reliable and repeatable fashion, to ensure everyone knows how 'stuff works.' We have processes on how to check in code, processes on what a good code review looks like, processes for planning sessions, and processes for releasing software to production.
This - thankfully - is a lot less bureaucratic than it sounds. Most of it is common sense, but it's good to have a play-book. In my opinion, one of the worst things is being 'surprised.' Perhaps that's a bit of a broad stroke, but there's nothing more frustrating than trying to carry out a basic function of your job and finding out that it either isn't documented, there is documentation but it's not up to date, or that the process has changed completely and it hasn't been written down!
We have a few methods of ensuring this is handled well. Primarily, we keep a knowledge base which all engineers have access to which documents all of our common workflows. This is handy for seasoned engineers as a quick memory jog when needed, and also helps to spread the load, as we can use this to help newer engineers get up to speed. It doesn't matter if releasing new software to production 'sounds' like a daunting task, because it's well documented with a list of pre-requisites and common gotchas.
There is - of course - such a thing as 'too much' process. With many things, long step-by-step lists can be indicative of a process which can be automated. To stick with releasing code as an example, this used to be a long list comprising twenty or more steps. Now, for the most part, it's three steps, and even these can certainly be automated further in the future.
Another less obvious way which this can be improved is to avoid private messaging where at all possible. We use Slack for internal communication and the vast majority of our messages occur in public channels. There's no value in me asking our engineering lead how to do X in a direct message. He'll reply (or, more likely, tell me to ask in the engineering channel), and nobody else is able to learn from my question. We have a great culture of challenging everything at Administrate, and by making these questions public, everyone then has ownership of ensuring that we work out why a question was asked, and how we can document it to make sure that it's obvious for the next person.
Post-Mortems, Testing and Regressions, Oh My!
When things go wrong, what can you always count on? Your fingers.
Sometimes things do go wrong. We haven't always done a great job of formally documenting what went wrong, why it happened, and what we can do to mitigate it for the future. This is now part of our core process, and is incredibly important for learning from our mistakes and ensuring that we do not repeat them. Every time we, directly, cause a breaking change to the product, we log the problem, the cause, what we did to fix it, a more general overview of what we 'messed up,' and what changes we can put in place to ensure it doesn't happen again.
I could spend a great deal of time talking about legacy code, what is considered 'legacy,' and how hard it is to work with. One of our teams shared heroes is 'Uncle' Bob Martin, who wrote - amongst others - Clean Code, and The Clean Coder. He considers any code without automated tests to be legacy code, and unfortunately we do have a lot of that, particularly on older parts of the product, such as the legacy events system - some of its components were written when I was in high school!
In the context of software engineering, automated testing means writing 'extra' code which makes sure the code you already have does what you think it does. For example, given a function which adds two numbers together, a viable test would be a piece of code which passes in two, and two, and verifies the result is four. That means that if anyone changes the number-adder, and it no longer returns four for these inputs, something has been broken!
For a long time now, we've absolutely committed to ensuring that no new code is going to production without suitable tests wrapped around it. This means it's harder for us to break things accidentally, because our test suite will tell us before customers get to use it, and crucially means that we can make changes and be confident that we're not breaking other parts of the system in the process. Again, this comes back to the way in which our team is growing. Not all of our code is super-obvious to the uninitiated, so automated testing provides a safety-net for these scenarios, too.
This is something we're continuing to work on, and something that every single person in the engineering team is completely dedicated to.
If you can't hear a pin drop, then something is definitely wrong with your bowling.
Spending 32 hours a week with colleagues is great, but aside from the odd bit of pair-programming and planning meetings, the engineering room at Administrate is mostly a quiet place. One of the best ways - in my opinion - to help a team gel is to get out of the office and go do something completely unrelated to work. With our four-day work week, the usual start-up-esque option of Team Fortress 2 on a Friday afternoon is a little bit less appropriate, so we have regular engineering socials at least twice in the year, planned around our other company social events.
These are a great chance for the team to unwind, and get to know each other as human beings rather than just engineers. We typically head out and grab a meal, followed by some light-hearted bowling, which is great fun, and just a good chance to blow off steam. We're a competitive bunch by nature (looking at you, Heather), and even after just two sessions, there are already some rivalries starting to develop! I certainly need a few more of these before I can fully judge their effectiveness, you know... Sample size and all that...
There's so much more I could go into about this, but I've already completely destroyed the word count that Marketing gave me, so I'll wrap it up here for now. We've got a lot going on, and in-keeping with our values, we're #alwaysimproving, every single day, so I'll check back in soon. I can't wait.