使用 GitHub Actions 编译 kubernetes 组件

在使用 kubernetes 过程中由于某些需求往往要修改一下 k8s 官方的源码，然后重新编译才行。本文就以修改 kubeadm 生成证书为默认 10 年为例，来讲解如何使用 GitHub Actions 来编译和发布生成的二进制文件。

构建

clone repo

将 kubernetes 官方源码 fork 到自己的 repo 中

$ git clone https://github.com/rainingwalk/kubernetes.git 
$ cd kubernetes 
$ git remote add upstream https://github.com/kubernetes/kubernetes.git 
$ git fetch --all 
$ git checkout upstream/release-1.21 
$ git checkout -B kubeadm-1.21

$ git clone https://github.com/rainingwalk/kubernetes.git

$ cd kubernetes

$ git remote add upstream https://github.com/kubernetes/kubernetes.git

$ git fetch --all

$ git checkout upstream/release-1.21

$ git checkout -B kubeadm-1.21

workflow

.github/workflows/kubeadm.yaml

---
name: Build kubeadm binary

on:
  push:
    tag:
      - 'v*'
jobs:
  build:
    runs-on: ubuntu-20.04
    # 使此git action在release时有写权限，不然会报GitHub release failed with status: 403 undefined
    permissions:
      contents: write
    # 这里我们选择以 tag 的方式触发 job 的运行
    if: startsWith(github.ref, 'refs/tags/')
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      # 运行 build/run.sh 构建脚本来编译相应平台上的二进制文件
      - name: Build kubeadm binary
        shell: bash
        run: |
          bash -x build/run.sh make kubeadm KUBE_BUILD_PLATFORMS=linux/amd64
          bash -x build/run.sh make kubeadm KUBE_BUILD_PLATFORMS=linux/arm64

      # 构建好的二进制文件存放在 _output/dockerized/bin/ 中
      # 我们根据二进制目标文件的系统名称+CPU体系架构名称进行命名
      - name: Prepare for upload
        shell: bash
        run: |
          mv _output/dockerized/bin/linux/amd64/kubeadm kubeadm-linux-amd64
          mv _output/dockerized/bin/linux/arm64/kubeadm kubeadm-linux-arm64
          sha256sum kubeadm-linux-{amd64,arm64} > sha256sum.txt
          
      # 使用 softprops/action-gh-release 来将构建产物上传到 GitHub release 当中
      - name: Release and upload packages
        uses: softprops/action-gh-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          files: |
            sha256sum.txt
            kubeadm-linux-amd64
            kubeadm-linux-arm64

---

name: Build kubeadm binary

on:

push:

tag:

- 'v*'

jobs:

build:

runs-on: ubuntu-20.04

# 使此git action在release时有写权限，不然会报GitHub release failed with status: 403 undefined

permissions:

contents: write

# 这里我们选择以 tag 的方式触发 job 的运行

if: startsWith(github.ref, 'refs/tags/')

steps:

- name: Checkout

uses: actions/checkout@v2

# 运行 build/run.sh 构建脚本来编译相应平台上的二进制文件

- name: Build kubeadm binary

shell: bash

run: |

bash -x build/run.sh make kubeadm KUBE_BUILD_PLATFORMS=linux/amd64

bash -x build/run.sh make kubeadm KUBE_BUILD_PLATFORMS=linux/arm64

# 构建好的二进制文件存放在 _output/dockerized/bin/ 中

# 我们根据二进制目标文件的系统名称+CPU体系架构名称进行命名

- name: Prepare for upload

shell: bash

run: |

mv _output/dockerized/bin/linux/amd64/kubeadm kubeadm-linux-amd64

mv _output/dockerized/bin/linux/arm64/kubeadm kubeadm-linux-arm64

sha256sum kubeadm-linux-{amd64,arm64} > sha256sum.txt

# 使用 softprops/action-gh-release 来将构建产物上传到 GitHub release 当中

- name: Release and upload packages

uses: softprops/action-gh-release@v1

env:

GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

with:

files: |

sha256sum.txt

kubeadm-linux-amd64

kubeadm-linux-arm64

build/run.sh

: Run a command in a build docker container. Common invocations:

build/run.sh make: Build just linux binaries in the container. Pass options and packages as necessary.

build/run.sh make cross: Build all binaries for all platforms. To build only a specific platform, add KUBE_BUILD_PLATFORMS=<os>/<arch>

build/run.sh make kubectl KUBE_BUILD_PLATFORMS=darwin/amd64: Build the specific binary for the specific platform (kubectl and darwin/amd64 respectively in this example)

build/run.sh make test: Run all unit tests

build/run.sh make test-integration: Run integration test

build/run.sh make test-cmd: Run CLI tests

修改源码

cmd/kubeadm/app/constants/constants.go

找到 CertificateValidity 变量将它在 365 天后面加两个 0，就将证书续命为 100 年了。

        // CertificateValidity defines the validity for all the signed certificates generated by kubeadm
        CertificateValidity = time.Hour * 24 * 36500

        // CACertAndKeyBaseName defines certificate authority base name
        CACertAndKeyBaseName = "ca"

// CertificateValidity defines the validity for all the signed certificates generated by kubeadm

CertificateValidity = time.Hour * 24 * 36500

// CACertAndKeyBaseName defines certificate authority base name

CACertAndKeyBaseName = "ca"

cherry-pick

在分支上完成修改之后，我们将这个修改 cherry-pick 到其他的 tag 上面去，下面以 v1.21.4 为例子：在 v1.21.4 tag 的基础之上将上述的修改 cherry-pick 过来，重新打上新的 tag。

获取上述修改的 commit id

$ COMMIT_ID=$(git rev-parse HEAD)

1

$ COMMIT_ID=$(git rev-parse HEAD)

checkout 到 v1.21.4 这个 tag 上

$ git checkout v1.21.4 Note: checking out 'v1.21.4'. 

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. 

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: 

HEAD is now at 3cce4a82b44 Release commit for Kubernetes v1.21.4

$ git checkout v1.21.4 Note: checking out 'v1.21.4'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example:

HEAD is now at 3cce4a82b44 Release commit for Kubernetes v1.21.4

将修改 cherry-pick 到当前 tag 上

$ git cherry-pick $COMMIT_ID 
[detached HEAD baadbe03458] Update kubeadm CertificateValidity time to ten years Date:
 Tue Aug 24 16:32:49 2021 +0800 
2 files changed, 38 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/kubeadm.yaml

$ git cherry-pick $COMMIT_ID

[detached HEAD baadbe03458] Update kubeadm CertificateValidity time to ten years Date:

Tue Aug 24 16:32:49 2021 +0800

2 files changed, 38 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/kubeadm.yaml

重新打上新的 tag，如 v1.21.4-patch-1.0

$ git tag v1.21.4-patch-1.0 -f Updated tag 'v1.21.4-patch-1.0' (was 70bcbd6de6c)

1
2

$ git tag v1.21.4-patch-1.0 -f
Updated tag 'v1.21.4-patch-1.0' (was 70bcbd6de6c)

将 tag push 到 repo 中触发 workflow

$ git push origin --tags -f 
Enumerating objects: 17, done. Counting objects: 100% (17/17), done. 
Delta compression using up to 4 threads Compressing objects: 100% (9/9), done. 
Writing objects: 100% (10/10), 1.13 KiB | 192.00 KiB/s, done.
Total 10 (delta 7), reused 0 (delta 0) remote: Resolving deltas: 100% (7/7), 
completed with 7 local objects. To github.com:k8sli/kubernetes.git 
+ c2a633e07ec...baadbe03458 v1.21.4-patch-1.0 -> v1.21.4-patch-1.0 (forced update)

$ git push origin --tags -f

Enumerating objects: 17, done. Counting objects: 100% (17/17), done.

Delta compression using up to 4 threads Compressing objects: 100% (9/9), done.

Writing objects: 100% (10/10), 1.13 KiB | 192.00 KiB/s, done.

Total 10 (delta 7), reused 0 (delta 0) remote: Resolving deltas: 100% (7/7),

completed with 7 local objects. To github.com:k8sli/kubernetes.git

+ c2a633e07ec...baadbe03458 v1.21.4-patch-1.0 -> v1.21.4-patch-1.0 (forced update)

注：此处如果用windows系统的话，会发现所有文件都有变更。变更的内容是文件的权限，都变成了 755。是因为git 在提交项目时会保留文件权限位，但是 windows 的 ntfs 等文件系统不支持文件权限的设置。

解决方案：

忽略文件权限，执行下面的配置命令即可。git 通过 filemode 处理文件权限

# 全局设置

$ git config –global core.fileMode false

只为本项目设置：

$ git config core.filemode false
风险：

安全起见，文件最好都没有执行权限。具体见 https://stackoverflow.com/questions/1580596/how-do-i-make-git-ignore-file-mode-chmod-chang

下面是我修改1.22.17版本的执行记录：

[root@sean kubernetes]# git checkout v1.22.17
Note: checking out 'v1.22.17'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD 目前位于 a7736ea... Release commit for Kubernetes v1.22.17
[root@sean kubernetes]# git cherry-pick $COMMIT_ID
[分离头指针 d1c350b] extend cert to 100 years
 2 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 .github/workflows/kubeadm.yaml
[root@sean kubernetes]# git tag v1.22.17-patch-1.0 -f
已更新tag 'v1.22.17-patch-1.0'（曾为 78c5908）
[root@sean kubernetes]# git push origin --tags -f
Username for 'https://github.com': rainingwalk
Password for 'https://rainingwalk@github.com': 
Counting objects: 18, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (11/11), 1.30 KiB | 0 bytes/s, done.
Total 11 (delta 7), reused 1 (delta 0)
remote: Resolving deltas: 100% (7/7), completed with 7 local objects.
To https://github.com/rainingwalk/kubernetes.git
 + 78c5908...d1c350b v1.22.17-patch-1.0 -> v1.22.17-patch-1.0 (forced update)
 * [new tag]         v1.22.17 -> v1.22.17

[root@sean kubernetes]# git checkout v1.22.17

Note: checking out 'v1.22.17'.

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD 目前位于 a7736ea... Release commit for Kubernetes v1.22.17

[root@sean kubernetes]# git cherry-pick $COMMIT_ID

[分离头指针 d1c350b] extend cert to 100 years

2 files changed, 38 insertions(+), 1 deletion(-)

create mode 100644 .github/workflows/kubeadm.yaml

[root@sean kubernetes]# git tag v1.22.17-patch-1.0 -f

已更新tag 'v1.22.17-patch-1.0'（曾为 78c5908）

[root@sean kubernetes]# git push origin --tags -f

Username for 'https://github.com': rainingwalk

Password for 'https://rainingwalk@github.com':

Counting objects: 18, done.

Delta compression using up to 2 threads.

Compressing objects: 100% (10/10), done.

Writing objects: 100% (11/11), 1.30 KiB | 0 bytes/s, done.

Total 11 (delta 7), reused 1 (delta 0)

remote: Resolving deltas: 100% (7/7), completed with 7 local objects.

To https://github.com/rainingwalk/kubernetes.git

+ 78c5908...d1c350b v1.22.17-patch-1.0 -> v1.22.17-patch-1.0 (forced update)

* [new tag] v1.22.17 -> v1.22.17

整个构建过程大概需要5分钟左右，效率还是挺高的。

总结

上面只展示了以一个 tag 为单位进行构建的流程，想要构建其他版本的 kubeadm ，可以按照同样的流程和方法来完成。其实写一个 shell 脚本来处理也是十分简单，如下：

#!/bin/bash

set -o errexit
set -o nounset

# 定义 commit ID
: ${COMMIT:="8b832eba48ee40ff50e51ca99989cbff727dd2ee"}

# 定义需要重新编译的版本号
: ${TAGS:="v1.20.15"}
#: ${TAGS:="v1.22.1 v1.22.0 v1.21.4 v1.21.3 v1.20.10 v1.19.14 v1.18.10"}

for tag in ${TAGS}; do
    git reset --hard ${tag}
    git cherry-pick ${COMMIT}
    git tag ${tag}-patch-1.0
    git push origin ${tag}-patch-1.0
done

#!/bin/bash

set -o errexit

set -o nounset

# 定义 commit ID

: ${COMMIT:="8b832eba48ee40ff50e51ca99989cbff727dd2ee"}

# 定义需要重新编译的版本号

: ${TAGS:="v1.20.15"}

#: ${TAGS:="v1.22.1 v1.22.0 v1.21.4 v1.21.3 v1.20.10 v1.19.14 v1.18.10"}

for tag in ${TAGS}; do

git reset --hard ${tag}

git cherry-pick ${COMMIT}

git tag ${tag}-patch-1.0

git push origin ${tag}-patch-1.0

done

使用 GitHub Actions 的好处就是能够为我们解决代码管理和产物管理，构建好的二进制文件存放在 GitHub release 当中，下载和使用起来十分方便，不用在自己搞一台单独的机器或者存储服务器，节省很多人力维护成本

使用 GitHub Actions 编译 kubernetes 组件

Published by sean on 02/06/202302/06/2023

构建

clone repo

workflow

修改源码

cherry-pick

总结

0 Comments

Leave a Reply Cancel reply

KUBERNETES

kubernetes使用本地磁盘进行动态管理pv

KUBERNETES

kubectl 的奇技淫巧

KUBERNETES

构建多种系统架构支持的 Docker 镜像

使用 GitHub Actions 编译 kubernetes 组件

Published by sean on 02/06/202302/06/2023

构建

clone repo

workflow

修改源码

cherry-pick

总结

0 Comments

Leave a Reply Cancel reply

Related Posts

KUBERNETES

kubernetes使用本地磁盘进行动态管理pv

KUBERNETES

kubectl 的奇技淫巧

KUBERNETES

构建多种系统架构支持的 Docker 镜像