Development environments

Sep 19, 2018 · 7 min read

It works on my machine

We have all heard it, and secretly, we have all probably said it too. Setting up your development environment really is a thing. About 10 years ago, I remember spending about two weeks getting a full ISP platform up and running on my host machine. There were a lot of moving parts I didn’t understand in my first week. I was fairly junior at the time, and this was the first time I was doing development, to this level, on a Linux platform. There was a wiki page with step by step instructions, updated by each new starter along the way. The people with the least knowledge of the platform were the passing guardians of one of the most fundamental documents in the department. About 4 weeks later we had an offsite where I had to stand up and give a talk about my experience of joining the department and business. When I stood there, rabbit in headlights, I explained that the first two weeks were a complete waste of time. There were some shocked faces. I was so new, I was just blindly entering commands from a wiki page, not really understanding the why. It really did not add much value to me, nor the company. I would have rather fixed my first software bug in those two weeks, and then started to learn the mammoth platform, when I had more context.

Fast forward a little bit. The tooling around development environments has moved on some what since those days.

What do we need

That wiki page was great, in the sense there was one. And to be fair, there were one or two dedicated senior engineers who did try and keep it up to date. But, it does get out of date. If you can define your development environment and version control it, you are going to be better off. This applies for every single software environment in the pipeline, be it development, staging, training, pre-production, production etc. It all still applies, but for this blog post, let’s just focus on the development environment.

So, what do we want from a development environment?

• We want it to be as similar to production as possible. It should have all the components that production has, but scaled down appropriately. If that means having some mocked components in there, that’s fine. This will really depend on the applications you are building and the scale at which your company operates
• Easy to spin up and tear down, one or two commands
• Fast to build. Doing this shouldn’t be “a thing”, it should just be happening whenever it’s needed
• Useful data, appropriate to the environment
• Automatically built for you via some provisioning system
• The platform to be defined as configuration

Parity with Production

Why is this important? Because “It works on my machine” isn’t acceptable, and it’s the easy way out. If your development environment does not look like production, then you have no idea if your software works in production. If your development environment does not look like production, why not?

Ease of use

Development environments need to be transient… Spin them up, tear them down… It should not feel like you are pushing an elephant up the stairs to use them, so you probably want some kind of provisioning service such as Ansible, Chef, Puppet, or Docker Compose etc. If it’s an overhead to use them, people will do anything to make their lives easier and will stop using them, therefore reverting to type, “It works on my machine”

Fast

You should be able to spin up your development environments in minutes. If it takes longer than getting a cup of coffee, something feels wrong to me. Maybe you are pulling down too many components to provision the Virtual Machine, for example. If you are using Vagrant, maybe you could use Packer to build a better base image of the components that do not change very often: Runtimes, Message Queues, database etc.

The aim with all of this is to be productive as possible, and focus on what matters: Building great software

The data

It shouldn’t be live data, so what then?

This will depend what industry you are in I suspect, so cannot really comment too much. However, I suspect having access to live data may be problematic, so your development environments should have useful anonymised data. If you can seed the data into your applications in a way that also helps test them, that’s a win. For example, you could use newman to pump Postman data into API end points, therefore proving APIs work.

It is the responsibility of the team to generate the appropriate seed data for the applications you are developing. If there is a bug, and it requires certain data, then there should be a script to seed the same data into the development environments moving forward, so you always can test that issue (Automatically, of course)

Automatic for the people

I couldn’t imagine being given a wiki page again, and told to “set up your machine”. If this happened, I would immediately start to automate the build out, and then get it under version control. I do not really mind what automation/tools are used, as long as they are used, maintained, and ingrained into the development teams. This isn’t something you give to one person to own, it’s the team that own it, and they should be empowered to change, adapt, add components where the team sees fits. This then goes up-stream to production, but it started in the right place, in my opinion, the team.

For extra “sharing is caring” points, if you could use the same provisioning scripts for your staging and production systems, you really are in the money. Depending on your organisation, there may be two or more different departments managing the different environments, so this may not always be possible, which is a shame.

I’m currently using Ansible to provision all my host machines, so I have consistency (Great, if you like things just so). My current team at work use Vagrant/VirtualBox to look after the development machine. In the past, we’ve used Chef and Puppet, but Ansible fits better in the team due to experience.

Sharing provisioning scripts for the host machine can also be very beneficial for the team. This will clearly depend on the level of engineer you have in your teams, and their thought process. Some folks will love the fact their host machine can also be provisioned: IDEs, Tools, Languages etc. Other folks will hate it, as the host is their machine, their domain. I am in the middle. My team have Ansible scripts to provision the host, but I also have my own scripts for my personal machines. Luckily, these play well together, so bear that in mind when deciding whether to do this. It’s a great way to get team members up and running.

Configuration in a version control system

So, we have an automatically provisioned development environment, which is fast and easy to use. It now needs to be version controlled with all the other code the team are managing. This is key. Otherwise you will get one engineer in the team who wants to maintain this. This is not scaleable in the long run, nor helpful for the people in the team wanting to learn and progress their skills.

Summary

We are in the business of building software, not setting up our machines. You should be spending more time focusing on your core business model than anything else. Therefore manually configuring your machine seems wasteful at best and negligent at worst.

• Having to setup and maintain all development environments could be the difference between shipping on time, or not
• Adding a new component into your platform, takes one engineer, not all the engineers doing it their way, on their machine. Again, this could be the difference of getting the product to market, or not

With a common development environment both on the host and the guest, you can onboard and support new team members more efficiently. Sure, folks may want to have more control over their host, and I agree with that. But at least there is a standard to deviate away from, when they are more comfortable in doing so.

I came across this slide from a work presentation I did about three 3 years ago. Still love it, and still relevant. We are now actively looking at growing our docker tooling and potentially replacing some of our vagrant machines, but the principles outlined above are the same.

Define, Automate, Version Control.