Case Study for Android CI -> CD -> CD = Continuous * ( Integration, Delivery, Deployment ) Part 1

Designed by Hafiz Waleed Hussain

WOW, we got one more day so it’s time to make this day awesome by learning something new.

Hello friends, hope you are doing good. So today I am feeling lazy and as we all know if developer feeling lazy its mean some automation is coming. In this post, we will discuss Continuous Integration ( CI ), Continuous Delivery ( CD ), Continuous Deployment ( CD ), and how we can implement in case of Android applications.

Motivation:

This is my favorite part. Why we need Continuous Integration ( CI ), Continuous Delivery ( CD ), Continuous Deployment ( CD )? For me, there are three reasons one as an individual, second as a developer who is working in a team and third is a business perspective.

As an Individual:

1. I want to be a lazy developer.
2. I want to increase productivity by saving time from repetitive tasks.
   As I am writing, my list is growing and I am feeling that will be boring for some of my readers. So, I am going to next reason.

As a Developer:
Here I am going to share with you one of my experience. Once, I am working on a project, which has a single code base for three different countries and we are using four development environments ( stack, staging, preprod, release). So as a whole I have 12 Android apps. Like country1Stack, country1Staging, country1Preprod, country1Release, country2Stack…
I am managing country base code by using flavors in Android Studio and development environment by using Build Variants. Now as my code base is getting large Android Studio performance is really poor. Sometimes that took 10 minutes to change a flavor and build environment. Then APK generation took around 3–5 minutes. A lot of time my Project manager come to me and ask me. Can you give us country3 stack build because API team fixed some bugs so we need to check? There are hundreds of other reasons and I need to give a build to QA. So I need to stop my work, if I am in between of a task, I need to do a git stash and then check out the last branch of that specific country. Then need to change variant and after that generate a build then send to QA team. So that takes some time around one hour. After that, I need to give more 15–30 mins to get my focus back. I really want to stop this but that is growing as we are adding more features. In the end, there is time comes when I feel maybe we need to hire a one person only to generate a build for QA team. So I decided I will do all this process automated. As a conclusion, I implementedContinuous Integration ( CI ), Continuous Delivery ( CD ), Continuous Deployment ( CD ). Now my builds automatically deployed to fabric for QA without me and also we published our apps directly to play store on a beta channel with one click.

As a Business:
I implemented this to save mine and my team time. My team is more productive to implement more feature in less time. There are many other perspectives which I am going to skip now.

In this process, I learn a lot of things which I know theoretically but I am not able to convince my team member’s or management of all these good practices. After this, I figure it out what is agile and why these things are really important. I know this is not a blog on agile practices, due to that I am going to make this shorter. I have a lot of respect for everyone knowledge or skill but In my opinion, Agile is 70–85% is dependent upon development team if your team is not confident to deploy new features in minutes then I don’t think so you are in a good agile environment. Your team should be 100% confident and always ready to deploy which is only possible if you have good code quality with unit tests and proper CI, CD, CD pipeline. Maybe my comment for some developers is useless but that is ok maybe one day you are able to relate all these things together.

Plan:

So, for my readers below is my plan for this case study.

Part 1

• Optional Prerequisites
• What is Continuous Integration ( CI), Continuous Delivery ( CD), Continuous Deployment ( CD )?
• Version control system ( VCS )
• Continuous integration server
• Build System

Part 2

• Application code on Github
• Git structure for the code
• Unit Tests
• Functional Tests
• Gradle build system and it’s commands
• Jenkins integration

Part 3

• Webhook
• Unit Test Report
• Code coverage report
• Lint rules
• Artifacts
• Release notes

Part 4

• Functional / UI / Espresso Unit Test Report
• Functional / UI / Espresso Unit Test Code Coverage Report
• Email
• Notifications may be on slack, hip chat
• Deliver APK using email
• Deliver APK using Fabric
• Deployment on play store beta
• Tag creation after publish
• Merge code with the master branch

Part 5

• Case study ( App has three different flavors )
• Git structure for the code
• Unit Tests
• Functional Tests
• Acceptance Tests
• Gradle build system and it’s commands

Code:

For this whole case study, I will use git repo Android-CI-CD-CD.

Optional Prerequisite:

If you already know Android development with Android studio, git, and Gradle that will be good. Otherwise, you need some extra effort but as a whole, this series will be helpful for everyone who wants to know what is Continuous Integration ( CI), Continuous Delivery ( CD), Continuous Deployment ( CD ) and what type of benefits we can get from this.

What is Continuous Integration, Continous Delivery, Continous Deployment?

First continuous does not mean some process is always running in execution. Here we can say continuous mean automation. Any time any change will occur automation should start the process without human intervention and validate all new developer changes. Now it’s time start our discussion about Continous Integration.

Continous Integration ( CI ) :

Now maybe you have a question. Who will validate developer changes? To validate changes developer need to write unit tests. Tests which validate their program functionality. Some teams are using Test Driven Development to make this first step more powerful and trustable.
( For my team, I have one extra step which you can say before CI start. On developer local machine as he will do a commit and try to push, git hook should check commit message format and git hook run all unit tests on a machine. If both checks pass then code will be pushed on Git and start a Continous Integration. ( Git hook concept we will discuss in this post later. )

Tools for Continous Integration ( CI ):

Source control system such as Git is a really powerful tool. That will take care of conflicts between team member code changes as they try to push code. They need to fix manually if some conflict occurs. As push is done then we need to use some Continuous Integration ( CI ) server like Jenkins, Bamboo, TeamCity etc. In this post, I am going with Jenkins. Jenkins can do polling on a source control system. If figure it out any issue can send notifications. Jenkins can send build and reports. Instead, Jenkins is the main power of our Continuous Integration ( CI), Continuous Delivery ( CD), Continuous Deployment ( CD ). On the other side some source code repository hosting like GitHub, they give us some power to make our Continuous Integration ( CI), Continuous Delivery ( CD), Continuous Deployment ( CD ) more powerful. Like they provide us to notify Jenkins about the change. I think that will be enough for Continuous Integration, remaining things I will show you practically.

Continous Delivery ( CD ) :

Once Continous Integration does a merge of an isolated developer changes into a whole codebase, after that this phase start. This sometimes called CDP continuous delivery pipeline. This phase duty is to take care of the whole code base should be able to provide a final end product in working form.
(For me, I do again unit tests here, because sometimes unintentionally developer do a push without git hook. So to take care of that scenario. I always have unit test check here. )
In this phase mostly we have integration tests, functional tests, and acceptance tests. Also, there are some reporting, for example, unit tests reports, code coverage reports or any other type of report we mostly generate here. If you have any code quality matrix you can implement in this phase. After that we have artifacts. Means a runnable product which is saved in some place. In our case, we will save our APK on Jenkins as an artifact with the proper versioning system. Up to this point if everything is good we can send this deliverable to our QA team for testing by using Email or any service with release notes.

Continous Deployment ( CD ) :

This is the last step of a continuous delivery pipeline. As our CDP proves us everything is good so now we can deploy our application into a product. This last phase is called a Continuous Deployment. Mostly, this phase will take a very long time to mature in any organisation. So most people do manually or sometimes companies deploy their applications as canary or beta up to 10% of their user base. In our case, I will take this step as an immature so I will show you how to deploy APK on to play store beta channel.

Version Control System ( VCS ) :

In this series of posts, we are going to use Git as a version control system for our source code. Git hub as a source code repository host. ( Repo Android-CI-CD-CD. )

Now in Git, we have a really awesome feature which is called git hooks. I already share with you about this in Continous Integration ( CI ). It’s time to show you the power of git hooks and how you guys can use in your daily development.

Git hooks:

“Git hooks are scripts that Git executes before or after events such as commit, push, and receive. Git hooks are a built-in feature — no need to download anything. Git hooks are run locally. ”

For example, I want my team members, every time they do commit they should follow a format:
Ticket Number — Description minimum 20 characters.
It’s time to show you how we can implement that feature in our case study.

In project directory -> .git -> hooks. Here I am going to write my script into commit-msg.sample file to implement commit message format validation.

Once you will save above script into file commit-msg. That will start validation. Remember, you need to remove its .sample extension.

As you can see in above image. Once any team member gives a wrong message. He is not able to commit. Now it’s time to give a proper format message.

Now as I gave a proper message. Git will do a commit for me. So now we can celebrate, we know what is git hook .

I want to show you one more very useful hook. Which will run the unit tests before Git Push. For that, you need to write a script into file pre-push.sample. Remember, you need to follow the same steps which we followed in commit-msg script to run successfully.

After adding this script once I will do a push. My git hook first runs a Gradle clean command. After that, this will run a unit tests. Currently, in our project, we have only one unit test. Which I am going to write to fail. So you guys can see how this hook will work.

Now I tried to push our code on git with this test.

As you can see. Your push will automatically terminate as test failed. Now, its time to fix this issue and push again.

Now, this time our code successfully pushed into our Github repository.

Continuous Integration Server:

As I already discuss with you guys about Continuous Integration Server. It’s time to discuss more Jenkins which we are going to use in this series of posts. So first step is how to install Jenkins. Simple way download its executive file and install on your local machine. This is the recommended way for those who are doing this process the first time. Jenkins website
Those who are doing the second time I will recommend you guys try to use Docker image for the installation. If you never work with docker before this. Do not take tension that is really easy especially in our case. Only follow the steps which I am going to share with you that will be enough. No need to start learning docker.

Docker:
(Note: This part did not have any relationship with CI but I really want everyone should be updated with new technologies which are really helpful for developers. Otherwise_, you can_ skip this and install direct executive from Jenkins website )
(Recommendation: Reader’s try to trust me and give 15mins max to docker if you are not able to use then you can skip this. Thanks)

Install docker on your machine. Donwload Link
After installation check command on your terminal as shown below:

After that, Now I have two options. First, I will share with you Docker Hub link from where you can read about how to install Jenkins or second, I will ignore that and I will share with you simple steps which you need to follow to run docker Jenkins without going anywhere else. I am going with the second approach.
Now run below command that may take some time ( depend upon your internet connection ).

docker pull jenkins/jenkins

This command will start downloading of Jenkins image as shown below:

As download complete. You can run your Jenkins by giving below command.

docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins

Good news Jenkins is up and running now on your machine. Open URL on your browser. http://localhost:8080

You will be asked Administrator Password by Jenkins as shown below:

This password you can get from the terminal where you gave a run command.

After that Jenkins will ask you to install plugins. So for that go with default plugins.

After completion of the plugins. You need to create an account. Once account creation is done. You will be on the Jenkins dashboard as shown below:

BOOM, Jenkins setup is complete from Docker. So, my friends, you have a good news. Now you know how Jenkins work with Docker.
Next important point related to Docker. If someday you try to open again localhost:8080 and you got a message something like shown below:

Only you need to run one command.

docker ps -a

After the execution of the above command. You will get the information related to Docker images as shown below.

If you have more then one docker images, then try to search Image jenkins/jenkins and copy its container id.
Next, run below command with copy container.

docker start Container Id 
docker start 084ea0cdaf66

After the successful execution of above command, refresh your browser with localhost:8080 and Jenkins will start working.

Now, everything related to Docker is done. For the revision purpose, I am going to make a cheat sheet for you guys.

docker pull jenkins/jenkins // Docker installation.
docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins // Jenkins run.
docker ps -a // If localhost not working try this command.
docker start [Container Id] // Docker will start again

Build System:

Build systems are software tools designed to automate the process of program compilation. Build systems are used for a variety of software build tasks. While their primary goal is to efficiently create executables. There are some popular build systems are:

• CMake
• Ant
• Maven
• Make
• Gradle

For our series of posts. We are going to use Gradle which is default Build System for Android. If you want to write any script for this build system you have two choices maybe you can use Groovy language or second is Kotlin. In Android, we mostly add dependencies in a build file. Basically, that is a Gradle file with Groovy language. Here I will share with you some commands related to Gradle which will be useful in next posts. Also, those who are new or they have experience but they are using Android Studio UI for a compilation they can increase there vision by learning these commands.

Now, In Android Studio as I want to clean my project mostly developer go to Menu -> Build and -> Click clean. Basically, once you guys click clean, Android Studio in a background run a Gradle command for us.

If we open Gradle console. We can see easily all commands which we run by Android Studio UI click. LIke as I click on clean in above UI I got some output on Gradle console as shown below:

One more command. As we want to build a debug APK we will click on Menu -> Build -> Build APK (s).

So you can see as in above image. Once we click, behind the scene Gradle run a command assembleDebug and we got an app-debug.apk.
Maybe you are thinking why I am sharing with you this Gradle build command system. So the answer is in Jenkins we will only play with build system commands. There is no Android Studio instead there is no need for Android Studio to implement Continous Integration ( CI ), Continous Delivery ( CD ), and Continous Deployment ( CD ). Instead, we should know commands and how to run these commands. So, it’s time to run one command on our terminal.

./gradlew assembleDebug

After running the above command I got the app-debug.apk and on a terminal, it looks like as shown in below image:

If you want to learn more about how many commands we have in Gradle build system for Android. I will recommend you open Gradle pane in Android Studio and see the commands.

At last, we finished our first post.

Revision:

• We discussed the prerequisites for this series of posts.
• We discussed what is Continuous Integration ( CI ), Continuous Delivery ( CD ), and Continuous Deployment ( CD ).
• We discussed Version Control System ( VCS ), then Git, and then Git hooks an awesome feature.
• We discussed Continous Integration Server ( Jenkins ), Installation of Jenkins by using Docker an awesome tool.
• We discussed Build System ( Gradle ) and we did some example of gradle build system commands.

Conclusion:

Maybe some of my reader’s think. This post is pretty long but I really want to clear all confusions which I faced personally, especially if you are new or maybe experienced developer but you are working without any senior team member’s help.

I hope you enjoyed this post and you learn something new for your carrier growth. OK friends, BYE BYE.

Case Study for Android CI -> CD -> CD = Continuous * ( Integration, Delivery, Deployment ) Part 2