Introduction
For the longest time I had hesitated on starting the journey of writing a large enough development project outside of my full-time job to keep me busy for months. It was always a mix of “what if no one uses it” or “I don’t know where I might start” or “can I do it all by myself”. To be honest after working in med — large sized organisations for more than a decade; I got used to the much larger than two-pizza teams, layers of politics and relatively slow delivery associated with large organisations. But when it clicked, all I had to do was sit down one evening with intention and start experimenting.
Focus on the MVP
The first version was very simple, all you could do was create a simple toml config file. However that was sufficient and gave me more than enough confidence to continue. However in the journey I learned that I need to focus on a complete set of functional requirements otherwise I will be creating random ideas instead of a functional set of features.
Lean into the Agile
I have worked in plethora of varying environments in #bigdata, #advertising, #insurance and #finance industries with different development methodologies across the board. For me this project meant that I need to be agile while minimising waste, what that means in simple terms is that when I started the project, all I had was a rough sketch of what I wanted and a blank terminal. No product owners, no backlogs, no sophisticated SDLC. This meant that as I started to build the project I had to adapt quickly with my planning, strategy and execution while reducing waste and keeping my focus on a small set of functional features. This enforced me to use some tools that I discuss later but I am glad I employed them as and when I needed them. Eventually towards the end of the beta release, I had a full set of open source / free SDLC tools (in addition to some paid ones) that I use daily for my projects.
X Driven Development
Should I use behaviour driven, test driven or documentation development methodology? At first this thought didn’t even cross my mind. The reason for that is because my main skillset is traversing problems in multi-disciplinary technical space. I don’t have one specific way of doing things. I have to adapt daily and at times multiple times a day depending on the team I am working with. I actually talk about adaptability and empathy in another post. In the end, I used a mix of methodologies I mentioned earlier depending on what I was trying to achieve at the time.
Dogfooding
As many of you are aware, dogfooding (a term which I am not a big fan of) means a product created should be used by the creators. This was the original motivation to build this tool. It’s only after building first few versions myself that I realised that it can be turned into a product that can be used by others and by building this product end-to-end, I’ve distilled my experience over the years to build and curate a system of SDLC tools, mechanisms and methodologies during the project.
Backlog is important
As soon as my first MVP was ready and I could see glaringly obvious bugs to be fixed, fundamental issues and missing functionality that would confuse the users, I realised all of those issues had to be tracked, this forced me to create a backlog (just a todo list) that would turn into part of my SDLC toolset. Don’t get me wrong, you can do a lot without a complicated backlog but if you want to collaborate with others while also building a product and release it on a regular / semi-regular basis, you’re better off creating a backlog. So started my weeks of visualising and backporting all my functional requirements into a sophisticated enough backlog that I could continue to use for myself or other collaborators.
AI is a friend, not a foe
ChatGPT is the norm these days. Even though I started out by myself, I always enjoy collaborating and bouncing ideas off others and getting their feedback. This is where I truly embraced the tool by OpenAI. Having a incredibly technical sidekick that can validate your ideas as well as spout opinionated technical perspectives can really boost your productivity not to mention its ability to help debug, write basic test cases as well as suggesting alternative ways of approaching a problem really came in handy. From my personal experience AI will truly transform the software development space in coming years. It’s a separate conversation on how huge the impact will be in the coming years.
Compliment workflows with the right tooling
If there’s one bias I have for software development and systems engineering i.e. you must use the right tools, especially if you are going to work on a sizable and semi frequently released project. I have been using Jetbrains products for quite a few years and their set of tooling for the entire SDLC is incredible. They understand how to build software and probably are the biggest example of dogfooding in the industry which they talk about here. Additionally the language I chose (Go) specifically for this purpose fits well with the requirements and the user base of the software. It’s a cross-platform command-line utility that will use network access and platform APIs to build backends.
Refactor, refactor, refactor
It goes without saying that you don’t finish a project and throw it over the line even if it delivers all the functions your had in mind. It requires consistent prototyping of ideas, building working functionality and refactoring until the entire application aligns with all the instruments that make up the orchestra to work in harmony. Here’s an excellent talk on refactoring by Martin Fowler on Workflows of Refactoring where he talks about Red, Green, Refactor cycle within TDD. During my journey of building this application I was on a constant cycle of build, test and refactor to the point of the initial versios of the app don’t look anything like the release beta.
Automation is the key
I am an automation enthusiast. That’s one of my main skillsets. Hence I tend to not let any opportunities to automate systems and processes go to awry. So when the time came to the release of the software itself to a central, publicly available repository as well as versioned documentation, I didn’t waste any time leveraging pipelines, building processes and automate instances of repeated tasks so that I can hand off most of the work to some type automated job. Gitlab is my main way of automating the entire workflow.
A good tool will productise itself
I didn’t start on the project visualising it as a product. However as time went by and my daily practise evolved I started to think about how I would release the tool and how many other features can be built. Additionally,s since I plan to open source parts (or all) of it at some point, and collaborate with others, I also wanted to think about how I would go about achieving that. I also didn’t want the users to feel like I am providing a subpar software and that they can integrate it into their existing pipelines / workflows, it forced my had to make it available in a way that it had all the signs of a product, so I thought to myself, why shouldn’t I turn this into a product.
Conclusion
The main reason I built this software is so that I can solve the problem for myself (and others) on how to create terraform remote backends. It’s an issue I had noticed since the day I started using terraform and never found a consistent way to solve this problem. Some suggested to do it manually, others suggested to use APIs and scripts. So I decided to create terraprep.
Terraprep is a cross-platform command-line utility designed to assist users by configuring and deploying terraform remote backends. It supports creation of multiple backend types and uses interactive prompts and TOML based configurations. It is available to download at https://terraprep.xyz
