Automated CI Project for Maven using a remote Jenkins Server on AWS EC2 instance and Github Web-hook

Shashwat Kumar
12 min readOct 31, 2020

--

A CI project that automatically triggers a remote Jenkins server when changes are pushed to github repository hosting a maven project.

The project triggers a pipeline on a remote Jenkins server to run the clean, compile, test and install goals as stages in a pipeline for the maven project.

Aim

  • Create an Ubuntu EC2 instance on AWS
  • Connect to the virtual cloud machine via SSH, and install required dependencies
  • Install and run jenkins server on a virtual cloud machine
  • Activate github webhooks for automated build triggering
  • Creating the automated pipeline for Continuous Integration
  • Testing the pipeline via a test push/commit

Step 1

Create an Ubuntu EC2 instance on AWS

Visit the AWS sign in to console at: https://signin.aws.amazon.com/

And sign in using your root user account

Enter your root user username and click next

Enter your password and click Sign In

Upon sign in you’ll land on the Amazon EC2 dashboard.

On the resources section, click on instances

You’ll be directed to a page similar to this.

To create a new instance, click on the Launch Instance tab on the top right

You’ll now scroll down to find the Ubuntu Server 20.04 LTS and click on select.

By default, you’ll have a t2 micro machine selected. No need to change it.

Click the Next: Configure Instance Details tab on the bottom right

Leave this page as default.(Though you may tweak around with some settings if you wish)

Click on the Next: Add Storage tab on the bottom right

Because we are just going to use a jenkins server so we need not add any extra storage. You can leave everything as default again.

Click on Add Tags tab on the bottom right

You can add a tag as a key-value pair.

Once added, click on the Next: Configure Security Groups tab

You can give your security group a name and add a description to it as well.

Also, importantly, Jenkins runs on port 8080, so we have to open that port.

Click on the Add Rule tab

For type: select the custom TCP option

For Protocol, select TCP

For Port Range, type 8080

Now click on Review and Launch tab on the bottom right

Once satisfied with all the settings, click on the Launch tab on the bottom right

On the dialog box that appears, you can select an existing key pair (like done in this example), or create a new one. In either case, make sure the key pair file is downloaded and remember the download location. Click Launch Instance

The instance will take some time to launch.

You can click the View Instance tab to see the instance

If you see a green tick next to your instance state with its state as “Running”, your instance is up and running.

Congratulations, you’ve now launched an EC2 Ubuntu instance on AWS.

Step 2

Connect to the virtual cloud machine via SSH, and install required dependencies

We’ll now connect to the virtual machine using the SSH manager. It would be appropriate to use a Linux machine or Machine on a virtual box. (If not, you may also install a SSH manager for Windows)

Select the running instance by checking the checkbox next to it.

Click on the Actions tab on the top right, and select Connect

On the Connect to Instance page, select the SSH Client Tab

Open the location where you downloaded/saved the private key. Open a terminal at that file location.

Copy the command in the point 3 and run it in the terminal:

The command should be something like “chmod 400 <filename.pem>”

This will make the private key file accessible only to admin.

Now, copy the command under the example and run it in the terminal. Enter “Y” in the terminal when prompted.

This will help you connect remotely to the created instance.

Once done, you’ll be able to see that your terminal shows ubuntu@<ip-address>:~$.

This confirms that you’ve successfully connected to the instance.

Now that you have connected to the instance, we will install the required dependencies for Jenkins.

Run the following commands in the terminal:

  • sudo apt update -y

## to update the existing dependencies, if any

  • sudo apt install openjdk-8-jdk -y

## to install JDK 8. Jenkins uses Java to run

Confirm the java installation using the following commands:

  • java -version
  • javac -version

We’ll now set the JAVA_HOME and export it into the system PATH variable.

Run the following commands:

  • export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 ##your java installation path may be different. Check before you set the JAVA_HOME variable

Now, we’ll install maven

Run:

  • sudo apt install maven

Enter “Y” when prompted to.

We set the MAVEN_HOME variable similar to the JAVA_HOME variable.

Run the following commands:

  • MAVEN_HOME=/usr/share/maven ## your maven installation may exist at a different location. Check before you set this variable
  • PATH=$PATH:$MAVEN_HOME/bin
  • export MAVEN_HOME
  • export PATH

Congratulations. You’ve set all the dependencies now for running Jenkins.

Step 3

Install and run jenkins server on a virtual cloud machine

For installation of jenkins visit the website: https://pkg.jenkins.io/debian-stable/

Here you’ll find the list of all the commands to install Jenkins.

While still connected to the virtual instance, run the first command in the terminal.

The command: “wget -q -O — https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -”

Now run: “sudo nano /etc/apt/sources.list”

This will open an editor to edit the sources.list file

Add the following line to the end of the file:

“deb https://pkg.jenkins.io/debian-stable binary/”

Once done click CTRL+O and Enter. Click CTRL+X to exit the editor.

Run the following command to install jenkins:

  • sudo apt-get update
  • sudo apt-get install jenkins

Enter “Y” when prompted to.

Now that Jenkins is installed. We must run it.

Run the following commands:

  • sudo systemctl start jenkins

## to manually start jenkins

  • sudo systemctl enable jenkins

## to enable jenkins to start at system restart

  • sudo systemctl status jenkins

## to check the status of jenkins- if it is running or not

If you see, an output screen similar to this, then congratulations, Jenkins server is up and running.

Now, it’s time to visit the Jenkins dashboard.

On the AWS instance page, you’ll find the public IP for your instance. Copy it.

Jenkins runs on port 8080. Hence your jenkins dashboard will run on:

http://<public-ip-of instance>:8080

Example: http://35.88.70.01:8080

Once visited, you’ll find Jenkins asking for the administrator password.

You’ll find The dashboard telling you the location of the administrator password file. Copy that location.

Run the following command to see the admin password:

  • sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the output and paste it in the space given on the Jenkins dashboard.

Click Continue tab

On Customize Jenkins page, select the “Install Suggested Plugins” option

The suggested plugins take some time to get installed.

After installation, Jenkins prompts you to create an Admin user.

Fill in the necessary details and remember the username and password you enter.

Click on Save and Continue

Jenkins now shows you the URL of your Jenkins server.

Click Save and Finish tab

You are now ready to use Jenkins.

Click on Start Using Jenkins tab

You’ll now see the Jenkins dashboard.

Congratulations. You’ve now run your very own Jenkins Server. You can now access the Jenkins from any browser and any computer system using URL and the login details.

While still on the Jenkins Dashboard page, click on Manage Jenkins Tab on the left

From this Page click on the Manage Plugins option

You’ll reach the install plugins page. Click on the Available tab

Search for the Maven Integration plugin, Git plugin and Blue Ocean plugin(optional).

Click on the Download and Install after restart option on the bottom of the page

Your plugins will get installed. It will take some time to do so. You can also check the Restart Jenkins when installation is complete option

When your plugins get installed, Jenkins will restart itself.

After Jenkins starts again, Click Manage Jenkins Option and select the Global Tool Configuration option

In the JDK section, name the JDK with a name you wish, and either check the “install automatically” or give the path to the installed jdk on the machine. The JAVA_HOME path you stored in the previous step should be provided here.

In the MAven section, name the MAven with a name you wish, and either check the “install automatically” or give the path to the installed maven on the machine. The MAVEN_HOME path you stored in the previous step should be provided here.

Once done, Click Save option

Step 4

Activate github webhooks for automated build triggering

Go to the Maven project on Github for which you want to set up the automated CI pipeline.

(If you do not have an existing maven project you may push a demo maven project a github repository to follow along)

Click on the Setting tab

Now, click on the Webhooks option on the left panel.

Click the Add webhook button

In the payload url option, paste the jenkins server url and append “/github-webhook/”to it.

Example: http://23.44.55.66:8080/github-webhook/

In the Content Type select the application/JSON option

Check the “Just the push event”

Also check the “Active”

Now click on Add webhook button

Congratulations, you’ve enabled a webhook. Now github will inform Jenkins server whenever a push even occurs.

Additionally, for more automation, we define the “Jenkinsfile” in the github project.

Create a new Jenkinsfile in the github project if you don’t have one already.

(The Jenkinsfile used in this example is as follows:

pipeline{

agent any

stages{

stage(‘Start’){

steps{

echo ‘Start the Pipeline’

}

}

stage(‘Clean phase Starts’){

steps{

echo ‘Start the Clean phase’

}

}

stage(‘Clean’){

steps{

sh ‘mvn clean’

}

}

stage(‘Compile phase Starts’){

steps{

echo ‘Start the Compile phase’

}

}

stage(‘Compile’){

steps{

sh ‘mvn compile’

}

}

stage(‘Test phase Starts’){

steps{

echo ‘Start the Test phase’

}

}

stage(‘Test’){

steps{

sh ‘mvn test’

}

}

stage(‘Install phase Starts’){

steps{

echo ‘Start the Instll phase’

}

}

stage(‘Install’){

steps{

sh ‘mvn install’

}

}

stage(‘End’){

steps{

echo ‘End the pipeline’

}

}

}

}

)

Step 5

Creating the automated pipeline for Continuous Integration

Return to the Jenkins Dashboard.

Click on New Item option to create a new Jenkins pipeline

Here on this page, name the project with a name you find suitable.

Select the project type as “Pipeline”

Click OK tab

In the General Section, check the Github Project option and give the location to the project repository

In the Build Triggers section, select the GitHub hook trigger for GITScm polling option

In the Pipeline section, select Pipeline Script from SCM option in the Definition box

Add SCM as Git

And give the repository link to the github repository

Click the Save button at the bottom

Once done, you’ll reach a page similar to this.

Congratulations. You’ve now created an automated CI pipeline that triggers the Jenkins server whenever you push on github.

Step 6

Testing the pipeline via a test push/commit

And now that an automated pipeline is built, the only thing that’s left to do is test it.

Open a terminal or Command line (depending upon your OS)

Move to the project repository on your local machine

Run the following commands:

  • git pull origin master

## best practice is to pull before you push

  • <Make some changes to the project, however small. Eg: change the ReadMe.md file>
  • git add .
  • git commit -m “Commit message”
  • git push -u origin master

You’ll see your changes pushed to the remote repository on Github.

Additionally see your pipeline on Jenkins server. It starts building too.

Click on your project on the Jenkins Dashboard.

On clicking, you’ll find your project has started building on its own. This was because github notifies Jenkins of the push event that occurred on it.

The status of the build also mentions that

If the project has no errors all the stages complete successfully. And the pipeline runs successfully.

Also the console output shows BUILD SUCCESS

For a more visual pipeline and better representation, you may additionally click on the Blue Ocean option on the left panel (if you installed the plugin before)

Blue Ocean provides a better visual of the rather mundane CI pipeline.

Congratulations. Now you’ve created, enabled and tested an automated CI pipeline.

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

No responses yet

Write a response