You may have heard the term “continuous integration” or “continuous deployment” or even “continuous delivery” tossed about in your department as a catch-all phrase for “we need to ship code quickly and constantly”. It’s true that a well-honed continuous integration (CI) program can result in rapid, hyper-agile delivery of software but in order to reap the philosophy’s rewards you have to establish and adhere to a disciplined protocol that is based on a true understanding of what CI actually is.
In order to understand CI, let’s look at the way software used to be shipped in the years leading up to the golden age of Agile, say, the aughts (2000 to 2010-ish). During these years, even the smallest feature change to an in-place application was a major undertaking. Budgets had to be approved, designs made, code written and tested, bugs fixed, user acceptance granted, and then a big monolithic chunk of code was released as a new version of the software. Because making a release was such a big undertaking, the needs of most stakeholders from the business fell to the wayside; there simply wasn’t enough time or budget to cater to all of their needs.
Things began to change, however, as we entered the teens (2013 to present). Web and native apps intended for consumption on smartphones exploded filling more and more niche needs and the typical business stakeholder became ever savvier in all things software as they grew accustomed to having myriad software features to solve problems in their personal life. This created a demand that spilled over into the workplace and became common in just about every conference room around the world: “Amazon sends me updates about the location of my order every step of the way! Why can’t we do that with our replacement part orders??” or “Searching for information on Google is so intuitive–it should be the same when we search our inventory” or how about, “We should make a mini-game like Angry Birds to promote this new ad campaign”. Overnight, software delivery professionals–from developers to quality assurance to analysts–were overwhelmed with requests and outnumbered by throngs of stakeholders with wishlists a mile long. The age of carefully planned, waterfall-like software release delivery schedules and the age of “I want it all and I want it now” Agile methodology had begun.
In the years since that critical inflection point in the art of software delivery Agile, a stream-of-consciousness approach to software delivery has proliferated in response to stakeholder demand/impatience and this has in turn given rise to CI which is the process of streamlining and automating the business of requirement specification, development, quality assurance, testing, user acceptance and, finally, production deployment.
In a CI world, a stakeholder may express a desire for a new feature in the company intranet during the Monday morning meeting. By lunchtime, the business analyst has gathered detailed requirements and placed the requirements into a ticketing system such as Visual Studio Team Services or Jira. This alerts the dev team automatically so that they can step away from the foosball table and get back to their workstations. By Monday afternoon, the developer(s) has accepted the ticket and used it’s automatic integration with the source control repository to create a new “branch” of the code. The developer’s job is done within the hour and her code is checked in which triggers an automatic execution of unit and end-to-end tests and then an automatic build to the QA environment and Slack notification to the QA testers. Once the QA staff has approved the build, the CI pipeline takes over once more and automatically handles the placement of the new branch into the production environment while maintaining the ability to easily roll back to the previous build if necessary. By Tuesday morning, the stakeholder is happily using the feature he requested during the previous day’s morning meeting. This would never be possible without an established CI program in the organization.
A well-developed CI program isn’t just for the benefit of the stakeholders; it has plenty of deep technical advantages as well. For example, most projects have multiple developers working in isolation. Adherence to a CI protocol forces a degree of work atomization which limits the ability for discrete tasks to become too large. This means more frequent code check-ins and integration with the production environment which means fewer nasty merge conflicts and bugs.
By now you probably get what CI does, but you may be asking yourself what exactly it is. Is it a tool? A platform? A philosophy? In reality, it’s a little bit of everything. DevOps professionals create “build definitions” using popular build engines like Visual Studio Team Services, Team City by JetBrains, Jenkins, or Octopus. You can think of these definitions as scripts that have hooks into both the source control repository where your application’s code resides as well as into the environments (servers) that run the working code. In a sense, these build definitions are a collection of IF THIS THEN THAT statements: “If a new ticket is added then create a new branch and email the dev team”, “If a developer checks in their code, then run unit tests”, and then, “If all unit tests pass, then deploy to the QA environment and email the testers”.
Different build engines have different strengths and it is possible that your organization uses more than one of them. In fact, we developed our NoOps/digital developer product called Catapult to help abstract and alleviate the stress of managing multiple build servers and other resources in order to further streamline the continuous integration process.
The other aspect of a solid CI program is to enforce a protocol to be followed by all team members. This is very important because if the team does not use the correct toolchain, the CI program won’t work and its benefits are lost. For example, if the stakeholder from the Monday morning meeting were to have simply emailed his request directly to the developer, that developer–eager to please–may have coded the feature and then checked it directly into the source control repository without following the proper branching protocol. This may have caused merge conflicts that then require a manual review and possibly fail to trigger automatic tests which would then result in the very real possibility that bugs slip through to production environments and create the need for site downtime. The good news is, there are some pretty great tools out there to make adherence to a CI protocol pretty easy. We are, of course, partial to Catapult but regular ol’ VSTS or Jira are pretty good as well.
If you are interested in instituting a CI program at your enterprise but don’t know where to start, please feel free to contact us for help. We are experts in the field of CI and we can either help you design and roll out a custom CI program or implement a licensed instance of Catapult to make CI (and DevOps in general) feel like magic.