In this tutorial, I will implement a DevSecOps CI/CD pipeline using various tools that are used in the industry. DevSecOps, short for Development, Security, and Operations, is a methodology that integrates security practices within the DevOps process. It aims to ensure that security is a shared responsibility throughout the entire IT lifecycle, from initial design to development and production.
Key principles of DevSecOps include:
Shift-Left Security: Incorporating security early in the development process to identify and fix vulnerabilities before they become more costly and difficult to address.
Automation: Using automated tools to perform security checks and enforce security policies, ensuring consistent and repeatable security practices.
Continuous Monitoring: Implementing continuous monitoring to detect and respond to security threats in real time.
Collaboration: Encouraging collaboration between development, security, and operations teams to ensure that security is integrated into every stage of the development lifecycle.
Compliance as Code: Automating compliance checks to ensure that applications and infrastructure meet regulatory and organizational security standards.
By integrating security into the CI/CD pipeline, DevSecOps helps organizations deliver secure software faster and more efficiently.
Now let's get started with the project. Here is the GitHub repository link if you are following along.
Creation of VMs to host Jenkins, SonarQube, and Nexus
Throughout this tutorial, I'll use AWS as the public cloud platform to host the necessary services. Now let's create 3 EC2 Instances with the same configurations.
We will name the instances later.
We will need t2.medium EC2 Instances (which are not under the Free Tier). Keep this in mind if you want to save some money!
We need to configure Security Group rules, but we'll do that later. For simplicity, we can attach a single security group to all three EC2 instances. Make sure to open the SSH/22 port for remote login to the instances. Also, select Ubuntu 24.04 AMI and 20 GiB of disk space for the root volume.
Click on the Launch Instance button.
Give names to the newly launched instances so that we can differentiate them based on their purpose.
If you are using Windows OS, open the Command Prompt for each EC2 Instance and navigate to the directory where your private EC2 instance key pair is downloaded. Then, enter the following command to establish an SSH connection to all three VMs:
ssh -i "Key-Pair-Name.pem" ubuntu@<Public-IP-of-Instances>
We should always execute the below command whenever we establish an SSH connection to the newly created VM. This will update all the packages that are present in the local repository and make the latest version available for installation.
sudo apt update
Now let's install Java on the Jenkins node.
sudo apt install openjdk-17-jre-headless
This will install 17.0.10~6ea-1 version of Java on the Jenkins node. Now Let's install Jenkins using the following snippet from the official Jenkins documentation.
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
Also, execute the following command to enable the Jenkins service to start at boot
sudo systemctl enable jenkins
You can start the Jenkins service with the command:
sudo systemctl start jenkins
You can check the status of the Jenkins service using the command:
sudo systemctl status jenkins
You should able to view the similar output on your command prompt
Now we have to set up or configure Jenkins for that first edit the EC2 Instance Security Group to allow Internet traffic(0.0.0.0 or specific IP range for more granular access) on port 8080 where Jenkins service is exposed.
Click on save changes copy and paste the Jenkins node public IP address on the web browser like <Jenkins-Node-public-IP>:8080 to open the Jenkins.
Run the following command to get the Administrator password for Jenkins, which is stored on the Jenkins node from when we installed Jenkins.
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Click on the Install suggested Plugins button.
Until the plugins are installed on Jenkins VM let's Install Docker on the SonarQube Node as we will use Docker to install SonarQube
Run the following commands to install Docker on the SonarQube VM.
sudo apt install docker.io -y
sudo su
systemctl status docker
systemctl enable docker
sudo su -->Become a root user to execute docker commands
systemctl status docker --> To view the status of Docker service. It should be in active (running) state. Press q to quit.
Now let's install SonarQube as a Docker container. We will use the free community edition image called lts-community. There is also a paid version available called the developer version.
docker run -d -p 9000:9000 sonarqube:lts-community
The first 9000 is the host port (in our case, the SonarQube VM) which is mapped to the second 9000 port, the Docker container port. This is because the SonarQube service is exposed on port 9000. Use docker ps to view the status of the SonarQube container it should be Up and running.
Let's get back on the Jenkins node and provide the details to access the Jenkins dashboard.
On the Getting Started page select Save and continue. Until now we have set up Jenkins but have not configured it yet. For Jenkins master and worker node machine 4GB of RAM and 20 GiB of Storage. For the Jenkins worker node if required, we can allocate more storage.
Now let's try to access the SonarQube service running inside a Docker container on the SonarQube VM on port 9000. Edit the security group rule to open port 9000. Select My-IP as the source because we don't want anyone from the internet to access the service.
Enter your initial username and password as admin for the SonarQube service. Then, update the default password to a strong one.
Until now we have set up the Jenkins and SonarQube VMs now let's set up the Nexus VM/node. Install Docker on Nexus VM using
sudo apt install docker.io -y
sudo su
systemctl status docker
Nexus service runs on default port 8081 just like SonarQube service runs on default port 9000 and Jenkins service runs on default port 8080. Run the following command to run the Nexus service inside a Docker container instead of running it on the Nexus VM/node host.
docker run -d -p 8081:8081 sonatype/nexus3
docker ps
Now edit the Security group of Nexus VM to allow traffic on port 8081 from the source of your IP address.
Copy and paste the Nexus VM IP address on your web browser to access the Nexus service dashboard.
For SonarQube, we know the default username and password is admin. For Nexus, to sign in, the username is also admin by default, but the default password is different for everyone. Just like with Jenkins, where we got the password from a file, we will also get our initial password for Nexus from a file. First, we need to access the Docker container using the following commands.
docker exec -it <Container-ID> /bin/bash
cat /nexus-data/admin.password
Copy and paste the password.
Set the new password for Nexus.
Enable anonymous access
All three VMs are set up now we also need to install Docker on the Jenkins VM. The Jenkins user on the Jenkins node should have access to run docker commands for that we need to add a Jenkins user inside the Docker group. Use the following commands
sudo usermod -aG docker jenkins
sudo reboot
Again run the ssh -i "<private-key-ec2.pem>" ubuntu@public-ip-of-jenkins-node
Another way Jenkins can access other services like Docker without root privileges on the Jenkins VM is by using the command sudo chmod 666 /var/run/docker.sock. However, this command is not secure, so we will not use it.
Now, other users can also execute Docker commands. Let's install some plugins on the Jenkins VM. From the Dashboard, navigate to Manage Jenkins in System Configuration and select Plugins.
Go to the Available Plugins tab on the left side. Search for "SonarQube Scanner" which will start the analysis.
Search for nexus and select "Nexus Artifact Uploader" Search for docker and select "Docker", "Docker Pipeline", "CloudBees Docker Build and Publish", and "docker-build step".
After that search for owasp and select "OWASP Dependency-Check " and search for jdk and select "Eclipse Temurin installer"
This will help us install multiple versions of jdk whichever we want to use we can go with that specific version. Finally, click on the Install button to Install all the selected plugins on the Jenkins VM. Now navigate to the Manage Jenkins page and select the Tools option.
In JDK Installation select Add JDK option give it a name and select Install automatically under which select Install fromadoptium.net and select jdk 17 version. repeat the same steps for jdk 11 version in case jdk 17 fails we should have jdk 11.
Under SonarQube Scanner Installation select Add SonarQube Scanner provide a name like sonarqube-scanner. For Maven Installation select Add Maven and provide a name like maven and select the version from Install from Apache dropdown list.
Under Dependency-Check Installations select Add Dependency-Check provide a name like dependency-check. Select Install Automatically radio button and click on Add Installer dropdown list and select Install from github.com and select a version.
Same for the Docker Installations click on the Add Docker button to provide a name such as docker. Select Install Automatically radio button and click on Add Installer dropdown list and select Download from docker.com
Click on the Apply button and then on the Save button to save the configurations. Now all the tools are configured with Jenkins but not yet connected the SonarQub and Nexus services with Jenkins.
Set up SonarQube with Jenkins. To connect SonarQube with Jenkins we need some authentication like username and password but when tools need to communicate with each other we will be using tokens.
Go to the SonarQube dashboard in the Administration section click on the Security dropdown list and select Users and an admin user will be present that is us we signed In using an admin username and password.
On admin user under Tokens click on the hamburger icon under Tokens for AdministratorGenerate Tokens give a name for that token like admin-token
Click on the Generate button a random hash will be generated.
Go to Jenkins dashboard and navigate to Manage Jenkins In the Security section select Credentials. Select the global and click on Add Credentials
In New Credentials under Kind select Secret text in Scope keep Global selected. In Secret paste, the SonarQube Admin user token and In ID provide a string like sonar-token same for description. Click on Create to create a credential.
Again click on Add Credentials
For Kind keep the Username with password selected and keep Scope Global for the Username enter your dockerhub username and for Password enter your dockerhub password and enter a string for ID like dockerhub-creds
On the Jenkins dashboard navigate to Manage Jenkins and select System Scroll down to the SonarQube servers. Click on the Add SonarQube button. In the Name enter a string like sonar. In the Server URL enter the SnarQube Server IP with a port number like http://<SonarQube-Node-IP:9000> in the Server authentication token dropdown select the token we created in Credentials. Click on apply and save button.
Now let's configure and connect Nexus with Jenkins this configuration needs to be done inside the source configuration files. We need to install a plugin for Managed Files where we create configuration credentials for servers. Go to Jenkins dashboard and select Manage Jenkins and select Plugins click on Available Plugins. Search for Config File Provider plugin. Select it and install it once the installation is complete navigate to the Manage Jenkins and now the Managed files tab will be visible click on that.
Click on Add a new Config button and select Global Maven settings.xml scroll down in ID enter a string like global-maven and click on the Next button. In the Edit Configuration File scroll down in the Content section copy line number 119 to 123 and paste it below line 124.
Go to the Nexus dashboard and click on the Setting icon that is beside the search tab and from the right-hand side select Repository and Repositories. Select maven-releases repository and copy the Name of the repository which is maven-releases paste it inside the <id> </id> block of the <server> </server> block in Content of Jenkins dashboard. For username and password block provide the Nexus admin username and password. For password security, we can use the Server Credentials option above the content tab.
Repeat the same steps for maven-snapshots copy the above <server> </server> block and copy and paste the id from the Nexus dashboard which is maven-snapshots and provide the username and password of the Nexus admin user. Next click on Submit button.
What is the difference between releases and snapshots?
Releases get deployed in the production environment and snapshots are deployed in non-production environments like development.
Now we need to edit the pom.xml file which is present on the GitHub repository. From the Nexus dashboard click on the copy button for maven-releases copy the URL and paste it into the <repository> </repository> block's <url> </url> block. repeat same steps for maven-snapshots.
From the Nexus dashboard click on the copy button for maven-snapshots copy the URL and paste it into the <snapshotRepository> </snapshotRepository> block's <url> </url> block.
If your code is on the local environment make changes and push it on github later. If your code is on GitHub then make changes on GitHub directly.
Creation of Jenkins Pipeline
Click on the New Item button on the Jenkins dashboard enter a name for your pipeline select Pipeline and click on the OK button.
Provide a short description of the pipeline. Select Discard old builds and inside Days to keep builds enter 100 Max # of builds to keep tab enter 2.
Scroll down to the Pipeline section and select Hello World from the dropdown list to get a basic groovy syntax. Remember two different stages can't have the same name. Make use of Pipeline Syntax if you are not aware of any specific syntax of Groovy.
We have not defined Java or Maven so whatever is installed on the Jenkins server that it's going to use. To make use of specific versions of tools like Java and Maven we need to define the versions in the tools{} block of the pipeline script.
Install the Pipeline Maven Integration Plugin from the Plugins tab on the Jenkins dashboard.
Pipeline script till Deploy Artifact to Nexus Stage.
pipeline {
agent any
tools {
maven 'maven'
jdk 'jdk17'
}
environment {
SCANNER_HOME = tool 'sonar-scanner'
}
stages {
stage('Git Checkout') {
steps {
git branch: 'main', url: 'https://github.com/jayeshrajputtech/E-commerce-Java-Full-Stack-Web-Application.git'
}
}
stage('Code Compilation') {
steps {
sh "mvn compile"
}
}
stage('Unit Tests') {
steps {
sh "mvn test -DskipTests=true"
}
}
stage('SonaeQube Analysis') {
steps {
withSonarQubeEnv('sonar') {
sh '''
$SCANNER_HOME/bin/sonar-scanner \
-Dsonar.projectKey=Ecomm-Java-App \
-Dsonar.projectName=Ecomm-Java-App \
-Dsonar.java.binaries=.
'''
}
}
}
stage('OWASP Dependency-Check') {
steps {
dependencyCheck additionalArguments: '--scan ./', odcInstallation: 'dependency-check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
stage('Maven Build') {
steps {
sh "mvn package -DskipTests=true"
}
}
stage('Deploy Artifact to Nexus') {
steps {
withMaven(globalMavenSettingsConfig: 'global-maven', jdk: 'jdk17', maven: 'maven', mavenSettingsConfig: '', traceability: true) {
sh "mvn deploy -DskipTests=true"
}
}
}
}
}
Setup Kubernetes Cluster
Now we need to set up a Kubernetes cluster with one Control Plane node and two worker nodes. Create three EC2 Instances of type t2.medium of Ubuntu 24.04 AMI with 20 GiB disk space and attach them to the same security group as Jenkins, SonarQube, and Nexus VMs.
Connect to each VM through SSH and update them. Run the following commands to set up the Kubernetes cluster. Run the below command on all three k8s nodes.
sudo apt-get update
sudo su
apt install docker.io -y
apt restart docker
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubelet
Now we need to edit the Security group that is attached to all the VMs we have created so far to allow internet traffic on port 6443 to set up the K8s cluster.
On the Control Plane/Master node of k8s run the below command to generate a join token using which K8s worker nodes can join the K8s cluster using port 6443.
kubeadm init
You will get a command as an output from the above command which you need to run on worker nodes of k8s cluster. It'll look like this
kubeadm join 172.31.38.143:6443 --token bdu417.o1nhd7g5hzfut --discovery-token-ca-cert-hash sha256:d792a0e3f2818f44d6946c929af63ecf8b465f6797c478c8f
The below commands are for the setup of calico and Nginx Ingress Controller installation.
kubectl apply -f https://docs.projectcalico.org/v3.20/manifests/calico.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.49.0/deploy/static/provider/baremetal/deploy.yaml
kubectl get nodes
Now to start using our cluster we need to run the following command as a regular or non-root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Once the K8s cluster is set we need to create a service account, and a role assign that role to the service account, create a secret for that service account, and create a token. We also need to create a namespace inside a k8s cluster.
On the control plane or master node of the K8s cluster run the below commands
kubectl create namespace webapps
Create a service account with the name jenkins so that using this service account Jenkins can access the k8s cluster. Create a k8s manifest file named as serviceaccount.yml using Vim editor and copy the below content in that file
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: webapps
Execute the below command to apply the changes
kubectl apply -f serviceaccount.yml
Now create a role to define what kind of access a service account will have that will be attached to this role. Create a k8s manifest file named role.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: app-role
namespace: webapps
rules:
- apiGroups:
- ""
- apps
- autoscaling
- batch
- extensions
- policy
- rbac.authorization.k8s.io
resources:
- pods
- componentstatuses
- configmaps
- daemonstes
- deployments
- events
- endpoints
- horizontalpodautoscalers
- ingress
- jobs
- limitranges
- namespaces
- nodes
- pods
- persistentvolumes
- persistentvolumeclaims
- resourcequotas
- replicasets
- replicationcontrollers
- serviceaccounts
- services
verbs: ["get", "list", "watch", "create", "updadte", "patch", "delete"]
Run the below command to apply the changes
kubectl apply -f role.yml
Next, we need to assign this role to the service account that we created for that Create a k8s manifest file named assign.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: app-rolebinding
namespace: webapps
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: app-role
subjects:
- kind: ServiceAccount
name: jenkins
namespace: webapps
Run the below command to apply the changes
kubectl apply -f assign.yml
Generally, the root account is not used to perform deployments using CI/CD tools. Instead, service accounts are used.
Now we need to generate a token for the service account so that Jenkins can access the service account using the token. Create a k8s manifest file named serviceaccounttoken.yml
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: mysecretname
annotations:
kubernetes.io/service-account.name: jenkins
Run the below command to apply the changes in webapps namespace
kubectl apply -f serviceaccounttoken.yml -n webapps
To view the token that is created inside the secret we need to run the following command. Remember that we need to get/retrieve the decoded token and not the base64 encoded token.
kubectl get secret mysecretname -n webapps -o jsonpath="{.data.token}" | base64 --decode
Now we need to find the config file which is inside the .kube hidden folder that we created earlier. The config file contains complete information about the k8s cluster.
As there is no official trivy plugin available on Jenkins we manually have to install trivy on Jenkins VM. Run the below commands on Jenkins VM to install trivy.
sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
Add Kubernetes Deployment Stages to Jenkins Pipeline Script
Now let's install Kubernetes and Kubernetes CLI plugins from the Jenkins dashboard in the Plugins section under Available Plugins. Click on the Install button.
Create a new credential in Jenkins so that Jenkins can access the Kubernetes cluster as a service account user.
To view the secret service account token that we created earlier execute the following command and paste it into the Secret tab while creating a credential.
kubectl get secret mysecretname -n webapps -o jsonpath="{.data.token}" | base64 --decode
For the Kubernetes API endpoint from the .kube/config file output copy the IP of the server and paste it on Jenkins' Kubernetes API endpoint.
Now update the image name and container port number inside the k8s manifest file named deploymentservice.yml
We also need to install the kubectl on Jenkins VM so that Jenkins can execute the kubectl commands that are declared/defined in the pipeline script on its server. Use the below command to install kubectl on Jenkins node.
sudo snap install kubectl --classic
Complete Jenkins Pipeline script looks like this in which docker, trivy and kubernetes deployment stages are added and it's a final pipeline script.
pipeline {
agent any
tools {
maven 'maven'
jdk 'jdk17'
}
environment {
SCANNER_HOME = tool 'sonar-scanner'
}
stages {
stage('Git Checkout') {
steps {
git branch: 'main', url: 'https://github.com/jayeshrajputtech/E-commerce-Java-Full-Stack-Web-Application.git'
}
}
stage('Code Compilation') {
steps {
sh "mvn compile"
}
}
stage('Unit Tests') {
steps {
sh "mvn test -DskipTests=true"
}
}
stage('SonaeQube Analysis') {
steps {
withSonarQubeEnv('sonar') {
sh '''
$SCANNER_HOME/bin/sonar-scanner \
-Dsonar.projectKey=Ecomm-Java-App \
-Dsonar.projectName=Ecomm-Java-App \
-Dsonar.java.binaries=.
'''
}
}
}
stage('OWASP Dependency-Check') {
steps {
dependencyCheck additionalArguments: '--scan ./', odcInstallation: 'dependency-check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
stage('Maven Build') {
steps {
sh "mvn package -DskipTests=true"
}
}
stage('Deploy Artifact to Nexus') {
steps {
withMaven(globalMavenSettingsConfig: 'global-maven', jdk: 'jdk17', maven: 'maven', mavenSettingsConfig: '', traceability: true) {
sh "mvn deploy -DskipTests=true"
}
}
}
stage('Docker Build and Tag'){
steps{
script{
withDockerRegistry(credentialsId: 'dockerhub-credentials', toolName: 'docker') {
sh "docker build -t jayeshrajput/java-ecommerce-app:latest -f docker/Dockerfile ."
}
}
}
}
stage('Trivy Image Scan'){
steps{
sh "trivy image jayeshrajput/java-ecommerce-app:latest > trivy-report.txt"
}
}
stage('Push Image to Dockerhub'){
steps{
script{
withDockerRegistry(credentialsId: 'dockerhub-credentials', toolName: 'docker') {
sh "docker push jayeshrajput/java-ecommerce-app:latest"
}
}
}
}
stage('Deploy to Kubernetes'){
steps{
script{
withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s-service-account-token', namespace: 'webapps', serverUrl: 'https://172.31.38.143:6443']]) {
sh "kubectl apply -f deploymentservice.yml -n webapps --validate=false"
sh "kubectl get svc -n webapps"
}
}
}
}
}
}
Remember one more change to reduce complexity edit the security group source and allow all internet traffic (0.0.0.0) on the required ports except for SSH traffic on all the VMs.
We have kept the ports 30000-32768 open because we are using NodePort type of service using which our application can get any port number within this range and we can access our application using the K8s cluster worker nodes IP and the port number.
These are the results from Jenkins, SonarQube, and Nexus
Our application is also visible and running.
Now we must terminate all the EC2 Instances that we have created.
Conclusion
In this tutorial, we have successfully implemented a DevSecOps CI/CD pipeline for deploying Java web applications using industry-standard tools. By integrating security practices early in the development process, automating security checks, and fostering collaboration between development, security, and operations teams, we have ensured that security is a shared responsibility throughout the entire IT lifecycle.
We started by setting up the necessary infrastructure on AWS, including Jenkins, SonarQube, and Nexus, and configured them to work together seamlessly. We then created a Kubernetes cluster to deploy our application, ensuring scalability and reliability. Throughout the process, we emphasized the importance of continuous monitoring and compliance as code to maintain high-security standards.
By following this comprehensive guide, you can now deploy secure Java web applications efficiently, leveraging the power of DevSecOps to deliver robust and secure software solutions.
About the Author
Jayesh is a senior college student passionate about guiding newcomers in Cloud and DevOps. He is determined to help others learn new cloud-native tools and technologies while advocating for inclusivity in the tech industry. Connect with Jayesh on LinkedIn