tekton - OCP pipeline 3 steps

환경은 openshift이다. on-prem k8s 구성은 추후 이어서 기록하자.

argocd를 사용하지 않고 하는 예제고, clone - build - dockerize and deploy의 3step이다.

 

먼저 pvc

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-pipeline-workspace-sample-app-boot
  namespace: sample
  annotations:
    pv.kubernetes.io/bind-completed: 'yes'
    pv.kubernetes.io/bound-by-controller: 'yes'
    volume.beta.kubernetes.io/storage-provisioner: csi.trident.netapp.io
    volume.kubernetes.io/storage-provisioner: csi.trident.netapp.io
  finalizers:
    - kubernetes.io/pvc-protection
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: ontap-sc-delete
  volumeMode: Filesystem
  
  ---
  kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-pipeline-mvn-sample-boot-app
  namespace: sample
  annotations:
    pv.kubernetes.io/bind-completed: 'yes'
    pv.kubernetes.io/bound-by-controller: 'yes'
    volume.beta.kubernetes.io/storage-provisioner: csi.trident.netapp.io
    volume.kubernetes.io/storage-provisioner: csi.trident.netapp.io
  finalizers:
    - kubernetes.io/pvc-protection
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: ontap-sc-delete
  volumeMode: Filesystem

 

pom.xml 하단에 아래 내용 추가. 넥서스는 사전에 구성되어 있어야한다.

<build>
    <finalName>${project.name}-${supermoon.app.version}</finalName>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>
                        <groupId>org.projectlombok</groupId>
                        <artifactId>lombok</artifactId>
                    </exclude>
                </excludes>
            </configuration>
        </plugin>
        <plugin>
            <groupId>com.google.cloud.tools</groupId>
            <artifactId>jib-maven-plugin</artifactId>
            <version>3.4.0</version>
            <configuration>
              <to>
                <image>docker.io/mooneyred/supermoon:${version}</image>
              </to>
            </configuration>
          </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-source-plugin</artifactId>
            <configuration>
                <skipSource>true</skipSource>
                <excludes>
                    <exclude>
                        <groupId>org.projectlombok</groupId>
                        <artifactId>lombok</artifactId>
                    </exclude>
                </excludes>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>${java.version}</source>
                <target>${java.version}</target>
                <compilerVersion>${java.version}</compilerVersion>
                <encoding>${encoding}</encoding>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <configuration>
                <archive>
                    <compress>true</compress>
                </archive>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <skipTests>true</skipTests>
            </configuration>
        </plugin>
    </plugins>
</build>


	<distributionManagement>
        <repository>
            <id>native-maven-release</id> 
            <url>http://nexus.supermoon.it/repository/native-maven-release</url> 
        </repository>
        <snapshotRepository>
            <id>native-maven-snapshot</id>
            <url>http://nexus.supermoon.it/repository/native-maven-snapshot</url>
        </snapshotRepository>
    </distributionManagement>

 

settings.xml은 configmap 이나 secret으로 설정한다. 나는 우선 configmap으로 했다.

kind: ConfigMap
apiVersion: v1
metadata:
  name: mvn-settings-sample
  namespace: sample
data:
  settings.xml: >
    <?xml version="1.0" encoding="UTF-8"?>
 
 
    <settings xmlns="http://maven.apache.org/SETTINGS/1.2.0"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">
 
      <servers> <!-- 계정별로 repository 접근권한을 다르게 줘야할 경우 server 접근 정보를 입력한다 -->
        <server>
            <id>native-maven-release</id>
            <username>cloud-native-nexus</username>
            <password>cloud-native-nexus</password>
          </server>
          <server>
          <id>native-maven-snapshot</id>
          <username>cloud-native-nexus</username>
          <password>cloud-native-nexus</password>
          </server>
          <server>
              <id>native-maven-public</id>
              <username>cloud-native-nexus</username>
              <password>cloud-native-nexus</password>
          </server>
          <server>
              <id>native-maven-central</id>
              <username>cloud-native-nexus</username>
              <password>cloud-native-nexus</password>
          </server>
      </servers>
 
    </settings>

 

빌드 컨피그를 작성한다. 파이프라인에서 호출하는데 빌드된 바이너리(jar)로 도커라이즈할 것이므로 source type은 binary로 한다.

kind: BuildConfig
apiVersion: build.openshift.io/v1
metadata:
  name: supermoon-sample-boot
  namespace: sample
  labels:
    app: supermoon-sample-boot
spec:
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      namespace: sample
      name: 'supermoon-sample-boot:latest'
  resources: {}
  successfulBuildsHistoryLimit: 2
  failedBuildsHistoryLimit: 5
  strategy:
    type: Docker
    dockerStrategy:
      dockerfilePath: Dockerfile
  postCommit: {}
  source:
    type: Binary
    binary: {}
  runPolicy: Serial

 

이제 pipeline에서 사용할 task를 정리한다.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  labels:
    app.kubernetes.io/instance: cloud
  name: init-pipeline-task
  namespace: sample
spec:
  params:
    - default: hello
      name: GREETING
      type: string
  steps:
    - command:
        - /bin/bash
        - '-c'
        - echo
        - $(inputs.params.GREETING)
      image: registry.redhat.io/ubi7/ubi-minimal
      name: pipeline-init
      resources: {}
      
  ---
  apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  annotations:
    tekton.dev/categories: Git
    tekton.dev/displayName: git clone
    tekton.dev/pipelines.minVersion: 0.21.0
    tekton.dev/platforms: 'linux/amd64,linux/s390x,linux/ppc64le,linux/arm64'
    tekton.dev/tags: git
  name: git-clone-sample
  namespace: sample
  labels:
    app.kubernetes.io/instance: devops
    app.kubernetes.io/version: '0.4'
    operator.tekton.dev/provider-type: redhat
spec:
  description: For Sample
  params:
    - description: Repository URL to clone from.
      name: url
      type: string
    - default: ''
      description: 'Revision to checkout. (branch, tag, sha, ref, etc...)'
      name: revision
      type: string
    - default: ''
      description: Refspec to fetch before checking out revision.
      name: refspec
      type: string
    - default: 'true'
      description: Initialize and fetch git submodules.
      name: submodules
      type: string
    - default: '1'
      description: 'Perform a shallow clone, fetching only the most recent N commits.'
      name: depth
      type: string
    - default: 'true'
      description: >-
        Set the `http.sslVerify` global git config. Setting this to `false` is
        not advised unless you are sure that you trust your git remote.
      name: sslVerify
      type: string
    - default: ''
      description: Subdirectory inside the `output` Workspace to clone the repo into.
      name: subdirectory
      type: string
    - default: ''
      description: >-
        Define the directory patterns to match or exclude when performing a
        sparse checkout.
      name: sparseCheckoutDirectories
      type: string
    - default: 'true'
      description: >-
        Clean out the contents of the destination directory if it already exists
        before cloning.
      name: deleteExisting
      type: string
    - default: ''
      description: HTTP proxy server for non-SSL requests.
      name: httpProxy
      type: string
    - default: ''
      description: HTTPS proxy server for SSL requests.
      name: httpsProxy
      type: string
    - default: ''
      description: Opt out of proxying HTTP/HTTPS requests.
      name: noProxy
      type: string
    - default: 'true'
      description: Log the commands that are executed during `git-clone`'s operation.
      name: verbose
      type: string
    - default: >-
        registry.redhat.io/openshift-pipelines/pipelines-git-init-rhel8@sha256:dde6d6d4b40f87ccc6737f1e317c13f6ff153155da4ebc48a2a5ebf31582f727
      description: The image providing the git-init binary that this Task runs.
      name: gitInitImage
      type: string
    - default: /tekton/home
      description: >
        Absolute path to the user's home directory. Set this explicitly if you
        are running the image as a non-root user or have overridden
 
        the gitInitImage param with an image containing custom user
        configuration.
      name: userHome
      type: string
  results:
    - description: The precise commit SHA that was fetched by this Task.
      name: commit
      type: string
    - description: The precise URL that was fetched by this Task.
      name: url
      type: string
  steps:
    - env:
        - name: HOME
          value: $(params.userHome)
        - name: PARAM_URL
          value: $(params.url)
        - name: PARAM_REVISION
          value: $(params.revision)
        - name: PARAM_REFSPEC
          value: $(params.refspec)
        - name: PARAM_SUBMODULES
          value: $(params.submodules)
        - name: PARAM_DEPTH
          value: $(params.depth)
        - name: PARAM_SSL_VERIFY
          value: $(params.sslVerify)
        - name: PARAM_SUBDIRECTORY
          value: $(params.subdirectory)
        - name: PARAM_DELETE_EXISTING
          value: $(params.deleteExisting)
        - name: PARAM_HTTP_PROXY
          value: $(params.httpProxy)
        - name: PARAM_HTTPS_PROXY
          value: $(params.httpsProxy)
        - name: PARAM_NO_PROXY
          value: $(params.noProxy)
        - name: PARAM_VERBOSE
          value: $(params.verbose)
        - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
          value: $(params.sparseCheckoutDirectories)
        - name: PARAM_USER_HOME
          value: $(params.userHome)
        - name: WORKSPACE_OUTPUT_PATH
          value: $(workspaces.output.path)
        - name: WORKSPACE_SSH_DIRECTORY_BOUND
          value: $(workspaces.ssh-directory.bound)
        - name: WORKSPACE_SSH_DIRECTORY_PATH
          value: $(workspaces.ssh-directory.path)
        - name: GIT_CREDENTIALS
          valueFrom:
            secretKeyRef:
              key: .git-credentials
              name: gitlab-auth-secret
        - name: GITCONFIG
          valueFrom:
            secretKeyRef:
              key: .gitconfig
              name: gitlab-auth-secret
        - name: GITLAB_URL
          valueFrom:
            secretKeyRef:
              key: .gitlab-url
              name: gitlab-auth-secret
      image: $(params.gitInitImage)
      name: clone
      resources: {}
      script: >
        #!/usr/bin/env sh
 
        set -eu
 
 
        if [ "${PARAM_VERBOSE}" = "true" ] ; then
          set -x
        fi
 
        if [ "true" = "true" ] ; then
          echo $GIT_CREDENTIALS > ${PARAM_USER_HOME}/.git-credentials
          echo $GITCONFIG > ${PARAM_USER_HOME}/.gitconfig
 
          chmod 400 "${PARAM_USER_HOME}/.git-credentials"
          chmod 400 "${PARAM_USER_HOME}/.gitconfig"
 
          git config --global pack.windowMemory "200m"
          git config --global pack.packSizeLimit "200m"
          git config --global pack.threads "1"
          git config --global pack.window "0"
          git config --global http.postBuffer 3194304000
        fi
 
 
        if [ "${WORKSPACE_SSH_DIRECTORY_BOUND}" = "true" ] ; then
          cp -R "${WORKSPACE_SSH_DIRECTORY_PATH}" "${PARAM_USER_HOME}"/.ssh
          chmod 700 "${PARAM_USER_HOME}"/.ssh
          chmod -R 400 "${PARAM_USER_HOME}"/.ssh/*
        fi
 
 
        CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}"
 
 
        cleandir() {
          # Delete any existing contents of the repo directory if it exists.
          #
          # We don't just "rm -rf ${CHECKOUT_DIR}" because ${CHECKOUT_DIR} might be "/"
          # or the root of a mounted volume.
          if [ -d "${CHECKOUT_DIR}" ] ; then
            # Delete non-hidden files and directories
            rm -rf "${CHECKOUT_DIR:?}"/*
            # Delete files and directories starting with . but excluding ..
            rm -rf "${CHECKOUT_DIR}"/.[!.]*
            # Delete files and directories starting with .. plus any other character
            rm -rf "${CHECKOUT_DIR}"/..?*
          fi
        }
 
 
        if [ "${PARAM_DELETE_EXISTING}" = "true" ] ; then
          cleandir
        fi
 
 
        test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}"
 
        test -z "${PARAM_HTTPS_PROXY}" || export
        HTTPS_PROXY="${PARAM_HTTPS_PROXY}"
 
        test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}"
 
        /ko-app/git-init \
          -url="${GITLAB_URL}${PARAM_URL}" \
          -revision="${PARAM_REVISION}" \
          -refspec="${PARAM_REFSPEC}" \
          -path="${CHECKOUT_DIR}" \
          -sslVerify="${PARAM_SSL_VERIFY}" \
          -submodules="${PARAM_SUBMODULES}" \
          -depth="${PARAM_DEPTH}" \
          -sparseCheckoutDirectories="${PARAM_SPARSE_CHECKOUT_DIRECTORIES}"
        cd "${CHECKOUT_DIR}"
 
        RESULT_SHA="$(git rev-parse HEAD)"
 
        EXIT_CODE="$?"
 
        if [ "${EXIT_CODE}" != 0 ] ; then
          exit "${EXIT_CODE}"
        fi
 
        printf "%s" "${RESULT_SHA}" > "$(results.commit.path)"
 
        printf "%s" "${GITLAB_URL}${PARAM_URL}" > "$(results.url.path)"
  workspaces:
    - description: The git repo will be cloned onto the volume backing this Workspace.
      name: output
    - description: |
        A .ssh directory with private key, known_hosts, config, etc. Copied to
        the user's home before git commands are executed. Used to authenticate
        with the git remote when performing the clone. Binding a Secret to this
        Workspace is strongly recommended over other volume types.
      name: ssh-directory
      optional: true
 ---
 apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  labels:
    app.kubernetes.io/instance: sample
    app.kubernetes.io/part-of: sample
  name: maven-build-jdk11
  namespace: sample
spec:
  steps:
    - args:
        - >-
          cp /root/settings.xml /root/.m2/settings.xml && cd /tmp/src && mvn
          package -U -Dmaven.test.skip=true
      command:
        - /bin/bash
        - '-c'
      image: 'mooneyred/maven:3.8.4-jdk-11' # 적합한 jdk11로
      name: patch
      resources: {}
  workspaces:
    - mountPath: /tmp/src
      name: source
    - mountPath: /root/.m2
      name: m2
    - mountPath: /root
      name: setting
 ---
 
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  labels:
    app.kubernetes.io/instance: sample
    app.kubernetes.io/part-of: sample
  name: openshift-cli-workspaces
  namespace: sample
spec:
  params:
    - default:
        - cd /tmp/src && pwd && ls
      description: The OpenShift CLI arguments to run
      name: ARGS
      type: array
  steps:
    - args:
        - $(params.ARGS)
      command:
        - /bin/bash
        - '-c'
      image: 'image-registry.openshift-image-registry.svc:5000/openshift/cli:latest'
      name: oc
      resources: {}
  workspaces:
    - mountPath: /tmp/src
      name: source

 

이제 pipeline을 생성해본다.

IMAGE_NAME, GIT_REPO, GIT_REVISION은 parameter로 콘솔에서 주입한다.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: sample-pipe
  namespace: sample
spec:
  params:
    - default: supermoon-sample-boot
      name: IMAGE_NAME
      type: string
    - default: >-
        http://git.supermoon.it/sample/supermoon-sample-boot.git
      name: GIT_REPO
      type: string
    - default: dev
      name: GIT_REVISION
      type: string
  tasks:
    - name: fetch-repository
      params:
        - name: url
          value: $(params.GIT_REPO)
        - name: revision
          value: $(params.GIT_REVISION)
        - name: deleteExisting
          value: 'true'
      taskRef:
        kind: ClusterTask
        name: git-clone
      workspaces:
        - name: output
          workspace: workspace
    - name: maven-build
      runAfter:
        - fetch-repository
      taskRef:
        kind: Task
        name: maven-build-jdk11
      workspaces:
        - name: source
          workspace: workspace
        - name: m2
          workspace: maven-cache
        - name: setting
          workspace: setting
    - name: deploy
      params:
        - name: ARGS
          value:
            - >-
              mkdir -p /tmp/binary/target &&  find /tmp/src/target -maxdepth 1
              -name "*.jar" -o -name "*.war" | xargs -I{} cp {}
              /tmp/binary/target/ &&  cp -rf /tmp/src/scouter
              /tmp/binary/scouter &&  cp /tmp/src/Dockerfile /tmp/binary/ &&  ls
              -rl /tmp/binary/ &&  oc start-build supermoon-sample-boot
              --from-dir /tmp/binary --follow --wait
      runAfter:
        - maven-build
      taskRef:
        kind: Task
        name: openshift-cli-workspaces
      workspaces:
        - name: source
          workspace: workspace
  workspaces:
    - name: workspace
    - name: maven-cache
    - name: setting

 

이미지스트림을 생성하자.

kind: ImageStream
apiVersion: image.openshift.io/v1
metadata:
  name: supermoon-sample-boot
  namespace: sample
  labels:
    app: supermoon-sample-boot
spec:
  lookupPolicy:
    local: false

 

이제 실행시, workspace, maven-cache, setting 을 생성한 pv와 configmap으로 잘 세팅하여 실행시켜본다.