The Path to Speed: How to Release Software to Production All Day, Every Day (Intro)

One of the biggest challenges most software companies face nowadays is getting their software to market more quickly, and without sacrificing quality. Users expect more, thanks to the help of modern technology and applications that allow many software businesses to update their products at the click of a button, or even automatically. As a result, expectations are high to deliver more features to customers at a faster rate.

In order to compete, most enterprises must be able to release updated features within days, or, even hours. That means they need to stop using those hit-or-miss, cumbersome and ineffective release procedures that struggle to release an update every few months. So how can they ship high quality software even quicker? To shorten the time between idea creation and the software release date, many companies are turning to continuous delivery using automation.

Delving into Continuous Delivery

As you may know, continuous delivery is about being able to deliver changes like new features, bug fixes and configuration updates quickly and sustainably. The goal is to have the ability to perform any deployment on-demand, from apps to large-scale distributed systems, but in a routine and predictable way.

With continuous delivery, your code is deployable 24/7, no matter how many developers are working on it at the same time. By doing so, there’s no need for code freezes as the entire process is streamlined in such a way that there are enough checks and balances in place to provide high quality releases.

However, issuing software and features on a more frequent basis doesn’t mean users should get a lower quality product. The fact is, continuous delivery is capable of providing fast, consistent results, giving companies a winning edge over their competition.

The Benefits of Continuous Delivery

Continuous delivery offers the following important benefits:

• Faster Market Times: It can take weeks or even months for the integration and test/fix phase of the traditionally-phased software delivery process. But when you can automate certain processes such as building and deployment, and application environment and regression testing, developers can remove integration and regression testing from their daily tasks and focus on other phases. That can also eliminate the burden of frequent re-work that can muddy the phased methodology.
• Lower Risk Releases: The goal of continuous delivery is to ease the pain of delivering software. It lowers the risk of the process and makes it so developers can add features and updates at any time. Leveraging patterns like “blue-green” deployment makes it easier to offer a zero-downtime deployment that users won’t even notice.
• Reduced Costs: When a service or software project is a success, it is sure to evolve over its lifetime. But by automating the build, test, deployment and environment phases, you can greatly lower the expense of making incremental updates. That is because automation removes many of the fixed costs that come with the process of software release.
• Consistently High Quality: Your teams can focus their efforts on high-level testing and other activities when they have automated tools that can detect regressions in minutes. This frees them up to do usability, exploratory, performance and security testing. Developers can build a quality deployment pipeline that allows them to perform these operations continuously during the delivery process. 
• Quicker Onboarding Process: New team members can onboard and are more productive quicker since there’s no need to learn a complicated developing and testing process.
• Better Feedback and Testing: Continuous delivery makes it more financially-viable to work on smaller batches. That allows developers to receive feedback from the working software throughout the delivery cycle. Being able to A/B test potential features with users before building them out allows teams to use a hypothesis-focused approach to developing their products. This reduces building out a number of features that lack value. This ends up saving not only time but also effort and money, too.
• Lowers Team Burnout: Continuous delivery makes releases easier, lowering team burnout. Also, when you release more often, your software delivery department is able to work more actively with users. That means they can quickly and easily determine which concepts work and which don’t. Also, they get to witness the results of their efforts. By lessening those difficult, less-value activities that come with software delivery, your team can focus on more important tasks.

The fact is that continuous delivery is not always easy at first. It’s about perpetual, everyday improvements and striving for a higher quality product. The end result of time, money and effort savings makes the initial investment to implement it well worth it. 

Common Stages of the Delivery Pipeline

Before you can create and automate your delivery pipeline, you need to identify each stage. Remember, every pipeline varies, so these stages aren’t set in stone. But the ones listed below are common for most software projects. Here are the main stages: build, staging and production.

1. Build: The build stage is where you build, package and archive the software (if applicable). It is also where you should run any unit tests. For the build stage, the input is normally a source-code repository and the output is an artifact stored in an artifact repository – oftentimes this is also where you collect output for all your unit tests, which is a critical stage. It is common to configure the build stage to react to changes in the source-code repository, for instance you may want to automatically run your test suite after pushing changes to the repository.
2. Staging: This is when you install or deploy the build artifact to the staging environment, which is a clone of the production environment. This is also when you automate tests to check the new environment, as well as do integration or functional tests on the new capabilities – some teams go as far as adding a layer of security automation/checks in this step.. Most importantly, this is where regression and performance tests take place to ensure the new version doesn’t affect any other capabilities. Staging deployments are usually triggered by simply merging code into a “staging” branch name (or whatever branch name is used to represent the Staging code).
3. Production: The production stage is when you install or deploy the software into the production environment. Additional tests take place to make certain that the new version is running as it should. Certain techniques, which are known as “blue-green” or “red-black” deployments can help prevent downtime during deployment. Similar to Staging, production deployments are usually triggered by simply merging code into a “production” branch.

How to Deploy the Process of Continuous Delivery

Now that you know the benefits and stages, here are three steps to start using continuous delivery in your operations. Here are some tips on setting goals you can measure, as well as leveraging automation for continuous delivery and testing your product to perfection.

Step 1: Ask the Tough Questions and Set Measurable Milestones

The first step is to set milestones or goals that you can measure with ease. To do that, you need to ask yourself some tough questions, including:

• How will you produce high-quality software that doesn’t hinder the user experience while considerably speeding up the development process?
• What are the areas you need to automate the most or the soonest?
• Which method and tools will you use to add automation to your processes?
• What are the financial implications and long-term ROI for implementing automation?

Think about and list the milestones you want to reach to get to your ultimate goal. There’s no need to make your goals perfectly align with what you have heard or read about continuous delivery, DevOps or other practices and methodologies. Tailor it to your specific needs, instead. 

Your focus should be on creating small milestones for building the best infrastructure. It should be an automation process that allows your DevOps team to build out, test and provide small increments of features or updates that will perform in the finished product.

Step 2: Add Automation for Continuous Delivery

The road to the software market is full of problematic areas that you can improve on with automation. They include automating application provisioning, configuration and deployment. You can initiate continuous delivery and DevOps-type projects by incorporating the following tools throughout the production cycle:

• Continuous integration and application release automation.
• On-demand virtualized environment provisioning.
• Measuring test automation.
• Configuration of virtualized environments.

It is possible to use continuous delivery to automate the build, testing, deployment and environment provisioning phases all the way to production. However, if you work for a larger organization with multiple departments and teams, you still may need both manual and automated tools. But no matter the size of the project or company, you have to be flexible enough to adjust your plan in an auditable and moderate way.

You want to produce a continuous delivery production process or pipeline that is fully-automated and streamlined to support your DevOps team. So, instead of just matching each of your specific tasks with your automation technology and tools, be sure to also use it to enhance team collaboration and project management.  

Adding Automation: New Projects Vs. Existing Ones

If you happen to be involved with a “greenfield” project, build and automate your delivery pipeline before you write a lot of the feature code. Create a basic “hello world” application in the project’s programming language. Next, identify each stage of your pipeline, so you can implement automation from one end to the other. This way, you can write your feature code much faster and confidently because you can automatically build and test every bit of code.

If you are involved in a project that has already started, look at what automation is currently in place. Find the places where the biggest “clogs” take place in the delivery pipeline. They are where automation could make the most difference.  

Step 3: And Finally, Test, Test and Retest

When you go from releasing products and updates once a month or once a week to multiple times a day, there just isn’t enough time or manpower to do manual testing. Even if it’s just basic regression testing, it would be impossible to keep up, at least not accurately. Imagine how much harder it would be to check new features while maintaining that level of speed and accuracy, even with more testers.

The answer is automation. One of the most beneficial and important features of an automated continuous delivery pipeline is automated testing. Be sure to plan out your testing strategy for all the stages. Also, include both non-functional and functional testing. A Test Driven Development (TDD) strategy might be a good idea for some teams.

Product testers and developers must closely collaborate to identify each repetitive test, like performance, security and code, and then automate it. But you’ll also want to think about what your users want in terms of features, playability and more.

Why It Takes a Team to Build an Automated Delivery Pipeline

Because creating a continuous delivery pipeline takes a lot of effort and money, it can affect your entire company’s productivity. For that reason, you should manage it as a separate project, just the same as you would when building a software project for a customer. That’s because it requires the same attention, discipline, quality and care as any other software development project. So depending on the number and size of projects your company handles, you should create a team dedicated to setting up, monitoring and maintaining your automation tools. Avoiding some common software development wastes is also a good idea.

Low quality and unreliable automation will waste more money, time and effort than it saves. Your team should be able to trust their testing tools, so they don’t have to spend too much time debugging flaky tests and mistakes. As your delivery pipeline evolves, you’ll be able to use it for more and more projects. A team that focuses on maintaining and optimizing code to keep it working to speed up delivery is priceless.

Remember, automation and tools are merely a means to an end. Every day, newer, upgraded technologies, tools and techniques arrive nearly every day. So, try not to purchase an automation tool just because it’s something new. You want to make sure it works well with the other automation tools in your delivery pipeline. On the other hand, avoid getting stuck in a rut. Stop using old automation tools that no longer work for the productivity and delivery of your product.  

Keeping the Flow Going Between Stages

In the most basic delivery pipelines, the stages are run sequentially. That means an error in one stage will hold the next stage back. In a more complex pipeline, there are multiple instances of some stages. For instance, you could have a production stage for each data center or region. That means you will want to run each production stage in parallel. Most production runs are often secured by a manual approval step to prevent accidental deployments as well as for auditing purposes.

To provide the quickest feedback, the early stages should be as basic and run as fast as possible. The later stages should run progressively more complicated tests in a progressively more production-like environment. Remember, it takes longer for the later stages to run. Also, to run in a production-like environment, they need more dependencies. However, as they pass every successive stage, it helps to raise the confidence to be ready for release to the market.

Choosing the Right Tools to Build Your Delivery Pipeline

Building the most basic delivery pipeline means finding several automation tools and platforms. With so many automation tools, your team needs to have the know-how to automate them to work well in conjunction with each other. You will also need someone who can monitor and maintain the delivery pipeline. Discuss all the available tools with your team to determine the best ones to use. 

The tools you choose also depend on the categories you plan to automate. Here are some common categories in automated delivery pipelines:

• Build Tools: Make, Ant, Gradle, Yarn and Maven.
• Continuous Integration (CI) Server Tools: CircleCI, Travis-CI and Jenkins.
• Source-Code Management Tools: Git.
• On Demand Test Environments: Squash.io
• Configuration Management Tools: SaltStack, Chef, Ansible, Puppet and SaltStack.
• Testing Framework Tools: Selenium, Cypress, TestCafe. Most testing frameworks use a specific programming language.
• Deployment and Provisioning Tools: Terraform, Chef and Ansible. 

Consider combining several open-source tools, such as Gradle and Jenkins to form the foundation for your framework. For storage purposes, many pipelines also have an artifact repository to hold items like install packages and binaries from the build stage and other stages, as well.

Finding Your Framework: Tying All Your Automation Tools Together

One final aspect of creating a delivery pipeline is building a framework to tie all of your automation tools together. An example of that is IBM Cloud Continuous Delivery. It’s a framework that uses sets of integrated tools called “toolchains,” like GitHub and Delivery Pipeline. 

If you want to use an open-source framework or tool or framework, look for one that comes with community support, such as Stack Overflow. These resources work with other tools, support multiple environments, and offer a wide selection of plug-ins or extensions. Also, you can find online sources for bug fixes and other help. You’ll find other people who know how to use your automation tools who can offer some valuable advice and information. 

Conclusion

While it’s advantageous to release software to production multiple times a day, it only works if you can maintain high-quality standards. However, this is possible by setting measurable milestones, using continuous delivery, and adding automation tools for thoroughly testing your software. 

These three steps will help lead you to faster, more successful product releases to market. Don’t let your competition leave you in the dust. Automate your delivery pipeline so they won’t even show up in your rearview window.