K3S 结合 coding 实现持续集成、持续部署

折腾k3s,最初的想法就是可以配合coding实现持续集成、持续部署。这两天把整个流程都跑了一下,记录如下:

基础环境

这里首先创建一个名为coding的Namespace,之后所有的操作都限制在这个命名空间内。这里主要参考coding官网的文档,但是完全按照那个文档来是不行的,文档中,角色创建的是ClusterRole,但是绑定的时候用的又是Role。我这里统一用的Role。

apiVersion: v1
kind: Namespace
metadata:
  name: coding
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 name: coding-cd-role
 namespace: coding
rules:
- apiGroups: [""]
  resources: ["namespaces", "configmaps", "events", "replicationcontrollers", "serviceaccounts", "pods/logs"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["pods", "pods/portforward", "services", "services/proxy", "secrets"]
  verbs: ["*"]
- apiGroups: ["autoscaling"]
  resources: ["horizontalpodautoscalers"]
  verbs: ["list", "get"]
- apiGroups: ["apps"]
  resources: ["controllerrevisions", "statefulsets"]
  verbs: ["list"]
- apiGroups: ["extensions", "app", "apps"]
  resources: ["deployments", "replicasets", "ingresses", "daemonsets"]
  verbs: ["*"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: coding-cd-service-account
 namespace: coding
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: coding-cd-role-binding
 namespace: coding
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: Role
 name: coding-cd-role
subjects:
- kind: ServiceAccount
  namespace: coding
  name: coding-cd-service-account

接着在k3s管理节点上运行命令获取Secret:

# Copy the secret name from the output of the get secret command
kubectl get serviceaccounts coding-cd-service-account -n coding -o yaml
kubectl get secret coding-cd-service-account-token-t9nh5 -n coding -o yaml # 注意这里的名字需要根据上一个命令的输出确定

这里还需要给服务器防火墙设置一下规则,允许CODING 持续部署的公网 IP 段访问。

接着在coding中“部署控制台”》云账号》绑定云账号 中,选择“Kubernetes"类型,认证方式选择“Service Account”,勾选“允许持续部署管理集群已有资源”。

开始测试

整体的思路是:

  1. git push 一个简单的静态网页;
  2. 自动打包成一个docker镜像;
  3. 在k3s中拉取镜像,运行起来。

第2步中,打包的docker镜像,推送到coding的制品仓库中,需要事先创建好docker仓库。

第3步中,在k3s中拉取镜像,需要解决一个问题,就是k3s访问coding非公开镜像的问题。

这一部分可以参考官网文档,相关命令如下:

kubectl create secret docker-registry coding-docker --docker-server=仓库域名 --docker-username=项目令牌用户名 --docker-password=项目令牌密码 -n coding

你可以在 CODING 制品仓库中的配置凭据中找到仓库域名:

先在k3s中配置好环境,

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webtest
  namespace: coding
  labels:
    app: webtest
spec:
  selector:
    matchLabels:
      app: webtest
  replicas: 1
  template:
    metadata:
      labels:
        app: webtest
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
              protocol: TCP
      imagePullSecrets:
        - name: coding-docker
---
apiVersion: v1
kind: Service
metadata:
  name: webtest-service
  namespace: coding
spec:
  selector:
    app: webtest
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: webtest-ingress
  namespace: coding
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: webtest.trycatch.xyz
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: webtest-service
                port:
                  number: 80

在coding构建计划中,选择这个模板,填写必要配置,设置好触发规则,就可以开始测试构建了。

这是一个基础模板,以后别的构建,都可以在这个模板上扩充,这里把Jenkinsfile贴出来:

pipeline {
  agent any
  stages {
    stage('检出') {
      steps {
        checkout([$class: 'GitSCM',
        branches: [[name: GIT_BUILD_REF]],
        userRemoteConfigs: [[
          url: GIT_REPO_URL,
          credentialsId: CREDENTIALS_ID
        ]]])
      }
    }
    stage('构建镜像并推送到 CODING Docker 制品库') {
      steps {
        sh "docker build -t ${CODING_DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_VERSION} -f ${DOCKERFILE_PATH} ${DOCKER_BUILD_CONTEXT}"
        useCustomStepPlugin(key: 'coding-public:artifact_docker_push', version: 'latest', params: [image:"${CODING_DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_VERSION}",repo:'docker',properties:'[]'])
      }
    }
    stage('部署到远端 Kubernetes 集群') {
      steps {
        cdDeploy(deployType: 'PATCH_IMAGE', application: "${CCI_CURRENT_TEAM}", pipelineName: '${PROJECT_NAME}-${CCI_JOB_NAME}-2211063', image: "${CODING_DOCKER_REG_HOST}/${CODING_DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_VERSION}", cloudAccountName: "${CD_ACCOUNT_NAME}", namespace: "${CD_NAMESPACE_NAME}", manifestType: "${CD_MANIFEST_TYPE}", manifestName: "${CD_MANIFEST_NAME}", containerName: "${CD_CONTAINER_NAME}", credentialId: '6ce8d2c8d5f44e81a975660a6e25e394', personalAccessToken: "${CD_PERSONAL_ACCESS_TOKEN}")
      }
    }
  }
  environment {
    CODING_DOCKER_REG_HOST = "${CCI_CURRENT_TEAM}-docker.pkg.${CCI_CURRENT_DOMAIN}"
    CODING_DOCKER_IMAGE_NAME = "${PROJECT_NAME.toLowerCase()}/${DOCKER_REPO_NAME}/${DOCKER_IMAGE_NAME}"
  }
}

用于测试的项目:

Dockfile文件

FROM nginx
ADD index.html /www/index.html
ADD web.conf /etc/nginx/conf.d/web.conf
EXPOSE 80

web.conf文件

server {
    listen       80;
    server_name     webtest.trycatch.xyz;
    location / {
        root   /www;
        index  index.html index.htm;
    }
}

index.html文件

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Document</title>
  </head>
  <body>
    hello world
  </body>
</html>

到这里,应该就可以撒花了,看似很顺了,但是我搞了一个晚上+一个早上才搞通

Leave a Comment

Back to Top