Understanding Kubernetes Maven Plugin resource fragments configuration

Eclipse JKube

In my previous blogpost, I talked about Eclipse JKube Kubernetes Maven Plugin XML configuration and how you can use it to configure Eclipse JKube’s output image/manifests. Today in this blog, we’ll be looking at Eclipse JKube Resource Fragments: a configuration option specific to Kubernetes manifests(JSON/YAML).

What are Resource fragments?

In Eclipse JKube terminology, resource fragments are complete/partial JSON/YAML snippets which user places in src/main/jkube directory of this project in order to tweak JKube’s opinionated manifest generation mechanism according to his needs. This use is usually for users who have decent knowledge of Kubernetes primitive types.

Let’s say a user wants to add an additional environment variable to his deployment. She can do it via XML configuration as well, but let’s say she adds a deployment.yml file in src/main/jkube directory of her project:

spec:
  template:
    spec:
      containers:
        - env:
            - name: FOO
              value: BAR

After this user runs mvn k8s:resource goal as usual and can see generated manifests in target/classes/META-INF/jkube/kubernetes folder. Finally generated manifests would be merged output of Eclipse JKube’s opinionated defaults and User’s YAML fragment:

Difference between user provided fragment and final generated Deployment manifest by JKube

As you can see in the screenshot, Eclipse JKube added it’s own opinionated labels, annotations, name(defaults to project.artifactId ), containers by inspecting project dependencies. You’ll also notice that additional environment variable specified by the user is preserved in final generated YAML manifest too. So by specifying just a fragment of complete YAML manifest we can tune JKube, that’s why these are called resource fragments.

Using Resource Fragments:

In order to use resource fragment configuration option, you need to provide a partial YAML file in src/main/jkube with content you want to override in JKube’s opinionated manifests. Name of the file depends on the Kubernetes resource you are trying to configure. For example, if it’s a Deployment , you’ll add a deployment.yml file; If Service , you’ll add a service.yaml file. You can find complete kind to filename mapping here in Kubernetes Maven Plugin’s documentation:

org.eclipse.jkube/kubernetes-maven-plugin

Here is an example of specifying fragments in src/main/jkube directory:

eclipse-jkube-sample-with-resource-fragments : $ tree src/main/jkube/
src/main/jkube/
├── configmap.yml
├── deployment.yml
├── sa.yml
├── second-configmap.yaml
└── service.yml
0 directories, 6 files

As you can see here I have provided files to defineConfigMap, Deployment , ServiceAccount and Service . You can see contents of all these files in below snippet:

$ cat src/main/jkube/configmap.yml 
apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: 2017-12-27T18:36:28Z
  name: game-config-env-file
data:
  allowed: '"true"'
  enemies: aliens
  lives: "3"
$ cat src/main/jkube/deployment.yml 
spec:
  template:
    spec:
      containers:
        - env:
            - name: FOO
              value: BAR
$ cat src/main/jkube/sa.yml 
metadata:
  name: ribbon
$ cat src/main/jkube/second-configmap.yaml  
application.properties: |
   spring.profiles.active=dev
$ cat src/main/jkube/service.yml 
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 8080

Note that in one case of configmap.yml , I have provided complete YAML manifest. In other cases, I have just specified minor YAML manifests specific to what I wish to override with respect to JKube defaults. You can go ahead and use Kubernetes Maven Plugin to generate manifests:

$ mvn k8s:resource
[INFO] Scanning for projects...
[INFO] 
[INFO] ---org.eclipse.jkube.quickstarts.maven:external-resources>--
[INFO] Building Eclipse JKube :: Quickstarts :: Maven :: External Resources 1.4.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------
[INFO] 
[INFO] --- kubernetes-maven-plugin:1.4.0-SNAPSHOT:resource (default-cli) @ external-resources ---
[INFO] k8s: Running generator spring-boot
[INFO] k8s: spring-boot: Using Docker image quay.io/jkube/jkube-java-binary-s2i:0.0.9 as base / builder
[INFO] k8s: Using resource templates from /home/rokumar/work/repos/jkube-testing/eclipse-jkube-sample-with-resource-fragments/src/main/jkube
[INFO] k8s: jkube-healthcheck-spring-boot: Adding readiness probe on port 8080, path='/health', scheme='HTTP', with initial delay 10 seconds
[INFO] k8s: jkube-healthcheck-spring-boot: Adding liveness probe on port 8080, path='/health', scheme='HTTP', with initial delay 180 seconds
[INFO] k8s: jkube-service-discovery: Using first mentioned service port '8080' 
[INFO] k8s: jkube-revision-history: Adding revision history limit to 2
...
[INFO] --------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ---------------------------------------------------
[INFO] Total time:  3.705 s
[INFO] Finished at: 2021-06-12T23:12:38+05:30
[INFO] ---------------------------------------------------

You can inspect the generated manifests in JKube’s output directory to see how JKube merged your fragments with it’s opinionated defaults. You’ll notice that the generated Kubernetes resources follow ${RESOURCE-NAME}-${KIND} as file naming strategy:

$ ls target/classes/META-INF/jkube/kubernetes -l
total 32
-rw-rw-r--. 1 rokumar rokumar 2438 Jun 12 23:12 external-resources-deployment.yml
-rw-rw-r--. 1 rokumar rokumar  294 Jun 12 21:21 external-resources-ingress.yml
-rw-rw-r--. 1 rokumar rokumar  860 Jun 12 23:12 external-resources-service.yml
-rw-rw-r--. 1 rokumar rokumar  306 Jun 12 23:12 game-config-env-file-configmap.yml
-rw-rw-r--. 1 rokumar rokumar  198 Jun 11 21:44 ribbon-1-serviceaccount.yml
-rw-rw-r--. 1 rokumar rokumar  198 Jun 12 23:12 ribbon-serviceaccount.yml
-rw-rw-r--. 1 rokumar rokumar  258 Jun 12 23:12 second-configmap.yml
$ cat target/classes/META-INF/jkube/kubernetes/external-resources-service.yml 
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/path: /metrics
    jkube.io/git-branch: master
    prometheus.io/port: "9779"
    jkube.io/scm-url: https://github.com/spring-projects/spring-boot/spring-boot-starter-parent/external-resources
    prometheus.io/scrape: "true"
    jkube.io/git-url: git@github.com:r0haaaan/eclipse-jkube-sample-with-resource-fragments.git
    jkube.io/scm-tag: HEAD
    jkube.io/git-commit: 1baf9f6f08ae8a1397a6e00fea62e75bfb659718
  labels:
    expose: "true"
    app: external-resources
    provider: jkube
    version: 1.4.0-SNAPSHOT
    group: org.eclipse.jkube.quickstarts.maven
  name: external-resources
spec:
  ports:
  - name: http
    port: 8080
    protocol: TCP
  selector:
    app: external-resources
    provider: jkube
    group: org.eclipse.jkube.quickstarts.maven
  type: LoadBalancer

Using Raw Resource fragments:

The YAML fragments provided in src/main/jkube are processed by Eclipse JKube, which means Eclipse JKube modifies them and adds some opinionated enhancements to your input fragment. Eclipse JKube has this concept of enrichers(a set of classes containing small configuration logic for your Kuberntes manifests) which make use of Visitor pattern in order to “enrich” your Kubernetes manifests one by one.

But what about the use case when user doesn’t want to JKube to interfere with it’s resource YAMLs and just want to deploy it along with application related manifests, just like doing kubectl apply -f /path/to/manifest.yml ? Well, JKube handles this as well. You can place these kind of YAML manifests in src/main/jkube/raw directory:

$ tree src/main/jkube/
src/main/jkube/
├── deployment.yml
└── raw
    ├── postgres-configmap.yml
    ├── postgres-deployment.yml
    └── postgres-service.yml
1 directory, 4 files

In above example, all YAML manifests provided in raw/ directory would be applied as it is by Eclipse JKube during apply phase. Eclipse JKube would not add any additional labels/annotations like it does with YAML manifests placed in src/main/jkube directory.

Example: Deploying Spring Boot on Postgres to Kubernetes using Eclipse JKube

In order to better understand fragments, let’s take a very simple example of a spring-boot application which use Postgres database. For deploying it to Kubernetes, we also need to make sure Postgres is running as a pod in Kubernetes Cluster. In this example, we will set up postgres using raw resource fragments and place a Deployment, Service and ConfigMap for Postgres in src/main/jkube/raw directory:

spring-boot-hibernate-crud-demo : $ ls src/main/jkube/raw/ -l
total 12
rokumar 192 Jun 19 19:37 postgres-configmap.yml
rokumar 479 Jun 19 19:26 postgres-deployment.yml
rokumar 161 Jun 19 19:26 postgres-service.yml

postgres-deployment.yml would contain Deployment for Postgres which would control Postgres pods. postgres-service.yml would expose this Deployment. postgres-configmap.yml would be containing credentials like username, password etc. Note that this Deployment is not using any volumes for persisting data since it’s only for demo purposes.

Apart from these three YAML manifests in raw/ directory. We have an additional deployment.yml in src/main/jkube where we customize Deployment generated by Eclipse JKube to include some additional environment variables read from postgres-config ConfigMap:

spec:
  template:
    spec:
      containers:
        - env:
            - name: POSTGRESQL_USER
              valueFrom:
                configMapKeyRef:
                  name: postgres-config
                  key: POSTGRESQL_USER
            - name: POSTGRESQL_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: postgres-config
                  key: POSTGRESQL_PASSWORD
            - name: POSTGRESQL_DATABASE
              valueFrom:
                configMapKeyRef:
                  name: postgres-config
                  key: POSTGRESQL_DATABASE

We will modify our Spring Boot application.yml to read database name, username, password from these environment variables:

spring.datasource.url=jdbc:postgresql://${POSTGRES_SERVICE_HOST}:5432/${POSTGRESQL_DATABASE}?createDatabaseIfNotExist=true
spring.datasource.username=${POSTGRESQL_USER}
spring.datasource.password=${POSTGRESQL_PASSWORD}

Once everything is configured, you can go ahead and deploy your application to Kubernetes. I tested this locally with minikube cluster and I followed these commands:

# Start minikube
$ minikube start

# To point your shell to minikube's docker-daemon
$ eval $(minikube -p minikube docker-env)

# Run Eclipse JKube build, resource, apply goals
$ mvn package k8s:build k8s:resource k8s:apply

Deploying Spring Boot on Postgres application to Kubernetes using Eclipse JKube

Check application pods after build finishes:

spring-boot-hibernate-crud-demo : $ kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
postgres-cf987cb96-svvz4          1/1     Running   0          3m7s
spring-boot-hibernate-crud-demo-5fc5697bf5-vvzwc   1/1     Running   1          3m7s

Once pods are in Running state try checking application services, I had configured Eclipse JKube to generate NodePort service. I was able to access it like this:

minikube service list

minikube service list

Check application logs in one window and try accessing application via some api testing tool:

kubectl logs pod/spring-boot-hibernate-crud-demo-5fc5697bf5-vvzwc -f

Checking whether application is running okay in kubernetes

Once everything is working as per expectations, you can undeploy everything using undeploy goal:

mvn k8s:undeploy

Conclusion:

In this blog, you learned how we can use raw YAML manifests(either complete or partial) with Eclipse JKube in order to customize Kubernetes manifests. This can be useful in various scenarios where XML configuration might not provide complete configuration support. You can find code used in today’s blog at this Github repository:

r0haaaan/jkube-spring-boot-hibernate-crud-demo

You can always read more about it in Kubernetes Maven Plugin documentation. If you have any questions, please feel free to reach out to us on our Gitter channel or via Github.