sean – Page 3 – 一江春水向东流

KUBERNETES

构建多种系统架构支持的 Docker 镜像

最新在信创项目中，经常需要构建支持amd64和arm64架构的镜像，而有的场景在同一个 Kubernetes 集群中的节点是混合架构的，也就是说，其中某些节点的 CPU 架构是 x86 的，而另一些节点是 ARM 的。为了让我们的镜像在这样的环境下运行，一种最简单的做法是根据节点类型为其打上相应的标签，然后针对不同的架构构建不同的镜像，比如 demo:v1-amd64 和 demo:v1-arm64，然后还需要写两套 YAML：一套使用 demo:v1-amd64 镜像，并通过 nodeSelector 选择 x86 的节点，另一套使用 demo:v1-arm64 镜像，并通过 nodeSelector 选择 ARM 的节点。很显然，这种做法不仅非常繁琐，而且管理起来也相当麻烦，如果集群中还有其他架构的节点，那么维护成本将成倍增加。你可能知道，每个 Docker 镜像都是通过一个 manifest 来描述的，manifest 中包含了这个镜像的基本信息，包括它的 mediaType、大小、摘要以及每一层的分层信息等。可以使用 docker manifest inspect 查看某个镜像的 manifest 信息：

$ docker manifest inspect aneasystone/hello-actuator:v1
{
        "schemaVersion": 2,
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "config": {
                "mediaType": "application/vnd.docker.container.image.v1+json",
                "size": 3061,
                "digest": "sha256:d6d5f18d524ce43346098c5d5775de4572773146ce9c0c65485d60b8755c0014"
        },
        "layers": [
                {
                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                        "size": 2811478,
                        "digest": "sha256:5843afab387455b37944e709ee8c78d7520df80f8d01cf7f861aae63beeddb6b"
                },
                {
                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                        "size": 928436,
                        "digest": "sha256:53c9466125e464fed5626bde7b7a0f91aab09905f0a07e9ad4e930ae72e0fc63"
                },
                {
                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                        "size": 186798299,
                        "digest": "sha256:d8d715783b80cab158f5bf9726bcada5265c1624b64ca2bb46f42f94998d4662"
                },
                {
                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                        "size": 19609795,
                        "digest": "sha256:112ce4ba7a4e8c2b5bcf3f898ae40a61b416101eba468397bb426186ee435281"
                }
        ]
}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

$ docker manifest inspect aneasystone/hello-actuator:v1

{

"schemaVersion": 2,

"mediaType": "application/vnd.docker.distribution.manifest.v2+json",

"config": {

"mediaType": "application/vnd.docker.container.image.v1+json",

"size": 3061,

"digest": "sha256:d6d5f18d524ce43346098c5d5775de4572773146ce9c0c65485d60b8755c0014"

},

"layers": [

{

"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",

"size": 2811478,

"digest": "sha256:5843afab387455b37944e709ee8c78d7520df80f8d01cf7f861aae63beeddb6b"

},

{

"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",

"size": 928436,

"digest": "sha256:53c9466125e464fed5626bde7b7a0f91aab09905f0a07e9ad4e930ae72e0fc63"

},

{

"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",

"size": 186798299,

"digest": "sha256:d8d715783b80cab158f5bf9726bcada5265c1624b64ca2bb46f42f94998d4662"

},

{

"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",

"size": 19609795,

"digest": "sha256:112ce4ba7a4e8c2b5bcf3f898ae40a61b416101eba468397bb426186ee435281"

}

]

}

可以加上 –verbose 查看更详细的信息，包括该 manifest 引用的镜像标签和架构信息：

$ docker manifest inspect --verbose aneasystone/hello-actuator:v1
{
        "Ref": "docker.io/aneasystone/hello-actuator:v1",
        "Descriptor": {
                "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                "digest": "sha256:f16a1fcd331a6d196574a0c0721688360bf53906ce0569bda529ba09335316a2",
                "size": 1163,
                "platform": {
                        "architecture": "amd64",
                        "os": "linux"
                }
        },
        "SchemaV2Manifest": {
                ...
        }
}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

$ docker manifest inspect --verbose aneasystone/hello-actuator:v1

{

"Ref": "docker.io/aneasystone/hello-actuator:v1",

"Descriptor": {

"mediaType": "application/vnd.docker.distribution.manifest.v2+json",

"digest": "sha256:f16a1fcd331a6d196574a0c0721688360bf53906ce0569bda529ba09335316a2",

"size": 1163,

"platform": {

"architecture": "amd64",

"os": "linux"

}

},

"SchemaV2Manifest": {

...

}

我们一般不会直接使用 manifest，而是通过标签来关联它，方便人们使用。从上面的输出结果可以看出，该 manifest 通过 docker.io/aneasystone/hello-actuator:v1 这个镜像标签来关联，支持的平台是 linux/amd64，该镜像有四个分层，另外注意这里的 mediaType 字段，它的值是 application/vnd.docker.distribution.manifest.v2+json，表示这是 Docker 镜像格式（如果是 application/vnd.oci.image.manifest.v1+json 表示 OCI 镜像）。可以看出这个镜像标签只关联了一个 manifest ，而一个 manifest 只对应一种架构；如果同一个镜像标签能关联多个 manifest ，不同的 manifest 对应不同的架构，那么当我们通过这个镜像标签启动容器时，容器引擎就可以自动根据当前系统的架构找到对应的 manifest 并下载对应的镜像。实际上这就是多架构镜像（ multi-arch Read more…

By sean, 2 years06/09/2023 ago

MIDDLEWARE

nginx反向代理Harbor/配置https

使用docker-compose安装harbor，配置ssl证书后使用nginx反向代理到harbor.配置后安装docker可以直接用域名登录harbor，无需配置私有仓库 nginx配置 1.修改harbor配置文件

hostname: ycr.yyiuap.com
http:
  port: 81
external_url: https://ycr.yyiuap.com

1

2

3

4

hostname: ycr.yyiuap.com

http:

port: 81

external_url: https://ycr.yyiuap.com

2.nginx配置

upstream harbor {
    server x.x.x.246:81;
}

server {
    listen 81;
    server_name ycr.yyiuap.com;
    location / {
        proxy_pass http://harbor;
        client_max_body_size 0;
        proxy_connect_timeout 90;
        proxy_read_timeout 90;
        proxy_buffer_size 4k;
        proxy_buffers 6 32k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
    }
}


server {
    listen       443 ssl;
    listen       [::]:443;
    server_name  ycr.yyiuap.com;

    ssl_certificate /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.cert;
    ssl_certificate_key /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    client_max_body_size 0;

    chunked_transfer_encoding on;

    location /v2/ {
      proxy_pass http://harbor/v2/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_buffering off;
      proxy_request_buffering off;
    }

    location / {
        proxy_pass http://harbor;
        client_max_body_size 0;
        proxy_connect_timeout 90;
        proxy_read_timeout 90;
        proxy_buffer_size 4k;
        proxy_buffers 6 32k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect http:// $scheme://;
    }
}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

upstream harbor {

server x.x.x.246:81;

}

server {

listen 81;

server_name ycr.yyiuap.com;

location / {

proxy_pass http://harbor;

client_max_body_size 0;

proxy_connect_timeout 90;

proxy_read_timeout 90;

proxy_buffer_size 4k;

proxy_buffers 6 32k;

proxy_busy_buffers_size 64k;

proxy_temp_file_write_size 64k;

}

server {

listen 443 ssl;

listen [::]:443;

server_name ycr.yyiuap.com;

ssl_certificate /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.cert;

ssl_certificate_key /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';

ssl_prefer_server_ciphers on;

ssl_session_cache shared:SSL:10m;

client_max_body_size 0;

chunked_transfer_encoding on;

location /v2/ {

proxy_pass http://harbor/v2/;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_buffering off;

proxy_request_buffering off;

}

location / {

proxy_pass http://harbor;

client_max_body_size 0;

proxy_connect_timeout 90;

proxy_read_timeout 90;

proxy_buffer_size 4k;

proxy_buffers 6 32k;

proxy_busy_buffers_size 64k;

proxy_temp_file_write_size 64k;

proxy_set_header Host $host:$server_port;

proxy_set_header X-Forwarded-For $remote_addr;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_redirect http:// $scheme://;

}

另：创建证书脚本：

#!/bin/bash

registry_url="ycr.yyiuap.com"


# 生成使用的相关证书
openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \
    -key ca.key \
    -out ca.crt

openssl genrsa -out $registry_url.key 4096

openssl req -sha512 -new \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \
    -key $registry_url.key \
    -out $registry_url.csr

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1=$registry_url
DNS.2=reg.yyuap
DNS.3=`hostname`
EOF

openssl x509 -req -sha512 -days 3650 \
    -extfile v3.ext \
    -CA ca.crt -CAkey ca.key -CAcreateserial \
    -in $registry_url.csr \
    -out $registry_url.crt

openssl x509 -inform PEM -in $registry_url.crt -out $registry_url.cert

mkdir -p /etc/docker/certs.d/$registry_url/

cp $registry_url.cert /etc/docker/certs.d/$registry_url/
cp $registry_url.key /etc/docker/certs.d/$registry_url/
cp ca.crt /etc/docker/certs.d/$registry_url/

#将证书加入系统级别信任
cp /etc/docker/certs.d/ca.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust extract
systemctl daemon-reload && systemctl restart docker

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

#!/bin/bash

registry_url="ycr.yyiuap.com"

# 生成使用的相关证书

openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 \

-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \

-key ca.key \

-out ca.crt

openssl genrsa -out $registry_url.key 4096

openssl req -sha512 -new \

-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \

-key $registry_url.key \

-out $registry_url.csr

cat > v3.ext <<-EOF

authorityKeyIdentifier=keyid,issuer

basicConstraints=CA:FALSE

keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment

extendedKeyUsage = serverAuth

subjectAltName = @alt_names

[alt_names]

DNS.1=$registry_url

DNS.2=reg.yyuap

DNS.3=`hostname`

EOF

openssl x509 -req -sha512 -days 3650 \

-extfile v3.ext \

-CA ca.crt -CAkey ca.key -CAcreateserial \

-in $registry_url.csr \

-out $registry_url.crt

openssl x509 -inform PEM -in $registry_url.crt -out $registry_url.cert

mkdir -p /etc/docker/certs.d/$registry_url/

cp $registry_url.cert /etc/docker/certs.d/$registry_url/

cp $registry_url.key /etc/docker/certs.d/$registry_url/

cp ca.crt /etc/docker/certs.d/$registry_url/

#将证书加入系统级别信任

cp /etc/docker/certs.d/ca.crt /etc/pki/ca-trust/source/anchors/

update-ca-trust extract

systemctl daemon-reload && systemctl restart docker

问题： docker pull ycr.yyiuap.com/base/golang:alphine-node-3 Error response from daemon: received unexpected HTTP status: 503 Service Unavailable 解决：去掉 http_proxy代理即可参考：https://blog.csdn.net/oscarun/article/details/121395218 1、request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)错误： [root@iZrj9j76z8dlull9vqa4tqZ ~]# docker pull harbor.xxx.cn:443/1-1/mytomcat:8.5.52 Error response from Read more…

By sean, 2 years06/09/2023 ago

KUBERNETES

docker下载镜像报错unknown method AddResource for service containerd.services.leases.v1.Leases: not implemented

导语：下载镜像到最后会提示unknown method AddResource: not implemented

<span class="pipeline-node-17">+ docker buildx build --output=type=local,name=openeuler,dest=./resources --platform=linux/amd64,linux/arm64 -f build/Dockerfile.os.openeuler .
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 11.0s done
#1 creating container buildx_buildkit_multi-platform0 0.0s done
#1 ERROR: Error response from daemon: No such image: moby/buildkit:buildx-stable-1</span>

1

2

3

4

5

6

<span class="pipeline-node-17">+ docker buildx build --output=type=local,name=openeuler,dest=./resources --platform=linux/amd64,linux/arm64 -f build/Dockerfile.os.openeuler .

#1 [internal] booting buildkit

#1 pulling image moby/buildkit:buildx-stable-1

#1 pulling image moby/buildkit:buildx-stable-1 11.0s done

#1 creating container buildx_buildkit_multi-platform0 0.0s done

#1 ERROR: Error response from daemon: No such image: moby/buildkit:buildx-stable-1</span>

原因，安装完docker启动后，还需要重启一下containerd服务 sudo systemctl restart containerd.service sudo systemctl restart docker 参考https://blog.csdn.net/sinat_14840559/article/details/114399166

By sean, 2 years06/07/2023 ago

MIDDLEWARE

Jenkins迁移

以前的Jenkins是通过docker容器跑的：

/usr/bin/docker run \
  --network host \
  --restart=on-failure:5 \
  --name=jenkins \
  -u=root \
  -v /ali-nas/data/jenkins:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /usr/bin/docker:/usr/bin/docker \
  -v /usr/libexec/docker/cli-plugins:/usr/libexec/docker/cli-plugins \
  -v /root/.docker/buildx:/root/.docker/buildx \
  -v /usr/local/maven:/usr/local/maven \
  -v /lib64/:/lib64/ \
  -e PATH=/usr/local/maven/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH \
  jenkins/jenkins:alpine

1

2

3

4

5

6

7

8

9

10

11

12

13

14

/usr/bin/docker run \

--network host \

--restart=on-failure:5 \

--name=jenkins \

-u=root \

-v /ali-nas/data/jenkins:/var/jenkins_home \

-v /var/run/docker.sock:/var/run/docker.sock \

-v /usr/bin/docker:/usr/bin/docker \

-v /usr/libexec/docker/cli-plugins:/usr/libexec/docker/cli-plugins \

-v /root/.docker/buildx:/root/.docker/buildx \

-v /usr/local/maven:/usr/local/maven \

-v /lib64/:/lib64/ \

-e PATH=/usr/local/maven/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH \

jenkins/jenkins:alpine

此方法有一些不便，想迁移到tomcat原生方式运行首先下载安装包：

wget https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.75/bin/apache-tomcat-9.0.75.tar.gz
wget https://get.jenkins.io/war-stable/2.401.1/jenkins.war

1 2	wget https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.75/bin/apache-tomcat-9.0.75.tar.gz wget https://get.jenkins.io/war-stable/2.401.1/jenkins.war

将tomcat解压后，将jenkins.war放到webapps目录启动tomcat，打开页面报错404，查看localhost.log日志： java.lang.UnsupportedClassVersionError 原因是2.361.1以上版本的jenkins要求java11或java17，而本机为java8，而jenkins的2.401要求java11以后下载JDK:

wget https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz

1	wget https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz

创建启动文件：

cat /usr/lib/systemd/system/jenkins.service
[Unit]
Discription=tomcat server
After=network.target

[Service]
#User=tomcat
#Group=tomcat
WorkingDirectory=/data/jenkins/tomcat9
Environment="JAVA_HOME=/data/jenkins/jdk-11.0.2"
Environment="JENKINS_HOME=/data/jenkins/tomcat9/webapps"
Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom"
Environment="CATALINA_BASE=/data/jenkins/tomcat9"
Environment="CATALINA_HOME=/data/jenkins/tomcat9"
Environment="CATALINA_OPTS=-Xms1024M -Xmx10240M -server -XX:+UseParallelGC"
Type=forking
ExecStart=/data/jenkins/tomcat9/bin/startup.sh
ExecStop=/data/jenkins/tomcat9/bin/shutdown.sh
Restart=always

[Install]
WantedBy=multi-user.target

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

cat /usr/lib/systemd/system/jenkins.service

[Unit]

Discription=tomcat server

After=network.target

[Service]

#User=tomcat

#Group=tomcat

WorkingDirectory=/data/jenkins/tomcat9

Environment="JAVA_HOME=/data/jenkins/jdk-11.0.2"

Environment="JENKINS_HOME=/data/jenkins/tomcat9/webapps"

Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom"

Environment="CATALINA_BASE=/data/jenkins/tomcat9"

Environment="CATALINA_HOME=/data/jenkins/tomcat9"

Environment="CATALINA_OPTS=-Xms1024M -Xmx10240M -server -XX:+UseParallelGC"

Type=forking

ExecStart=/data/jenkins/tomcat9/bin/startup.sh

ExecStop=/data/jenkins/tomcat9/bin/shutdown.sh

Restart=always

[Install]

WantedBy=multi-user.target

在准备好必要的环境后，jenkins的迁移需要将jenkins主目录原有数据拷贝到新的机器或者新目录下，主要包含config.xml文件以及jobs、users、workspace、plugins四个目录，然后重启jenkins即可

cd /data/jenkins/tomcat9/webapps
\cp -r /ali-nas/data/jenkins/jobs/* .
\cp -r /ali-nas/data/jenkins/config.xml .
\cp -r /ali-nas/data/jenkins/users/* users/
\cp -r /ali-nas/data/jenkins/workspace .
\cp -r /ali-nas/data/jenkins/plugins/* plugins/

1

2

3

4

5

6

cd /data/jenkins/tomcat9/webapps

\cp -r /ali-nas/data/jenkins/jobs/* .

\cp -r /ali-nas/data/jenkins/config.xml .

\cp -r /ali-nas/data/jenkins/users/* users/

\cp -r /ali-nas/data/jenkins/workspace .

\cp -r /ali-nas/data/jenkins/plugins/* plugins/

By sean, 2 years06/06/2023 ago

HANDBOOK

ansible 安装 kubespray – 优化安装速度

同样的配置(8c16G), ansible在x86_64机器上安装，以前需要半个多小时，优化后大概花费10分钟时间，在arm64架构上安装大概需要15分钟首先优化ansible.cfg配置文件：相比之前修改的地方：测量任务执行速度：profile_tasks 插件 Ansible 官方提供了几个可用于计时的回调插件： (1).profile_tasks：该回调插件用于计时每个任务的执行时长 (2).profile_roles 插件用于计时每个 Role 的执行时长 (3).timer 插件用于计时每个 play 执行时长要使用这些插件，需要在 ansible.cfg 配置文件中的 callback_whitelist 中加入各插件使用 forks、serial、strategy 等指令可以改变 Ansible 的执行策略。默认情况下 forks=5，这表明在某一时刻最多只有 5 个执行任务的工作进程 (还有一个主进程)，也即最多只能挑选 5 个节点同时执行任务。 serail 是 play 级别的指令，用于指定几个节点作为一批去执行该 play，该 play 执行完后才让下一批节点执行该 play 中的任务。如果不指定 serial，则默认的行为等价于将所有节点当作一批。 strategy 指令用于指定节点执行任务时的策略，其侧重点在于节点而在于任务，默认情况下其策略为 linear，表示某个节点先执行完一个任务后等待其余所有节点都执行完该任务，才统一进入下一个任务。另一种策略是 free Read more…

By sean, 2 years04/14/2023 ago

MIDDLEWARE

nginx问题解决

nginx转发post请求静态页面405 背景:用户支付成功后的回调是个静态页面。由于from表单连续提交是post方式，所以会报405 not allowed 错误。常识:使用post方式请求js、html这样的静态文件一般的web服务器都会返回405 Method Not Allowed。因为默认情况下，nginx、apache、IIs等web服务无法响应静态页面的post请求，后端用来处理post请求，生产环境中不会有此问题（一般都不允许配置静态页面的post请求）问题:为什么默认不支持静态页面post请求呢？首先了解一下post请求方法，post请求一般用于提交表单或上传文件，post请求会导致新资源的建立或旧资源的更改。就安全方面来说（排除url地址的透明性），它对比get请求会有更改资源的情况，有些静态资源是不允许更改的，所以默认情况下web服务器上的静态资源都不允许发起post请求。网上说的有很多种，重新编译nginx，设置正则匹配可以访问html文件类啊什么的，都不好用，而且基本都是你抄我我抄你，没有实际应用过。乱糟糟，最后本人用下面这种方式解决了问题。

upstream domain_uau {
  server 192.168.6.202:5555 weight=5;
}
server {
    listen              80;
    server_name         dev-app-server.sinoif.com;
    access_log  /var/www/logs/nginx/dev-app-server.sinoif-access.log;
    location / {
        proxy_set_header Host $host;
        proxy_set_header  X-Real-IP        $remote_addr;
        proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://domain_uau/;
        add_header Cache-Control "no-cache, no-store";#不生成缓存
        error_page 405 =200 @fullfuck;
        proxy_intercept_errors on;
    }
    location @fullfuck {
        proxy_method GET;
        proxy_pass http://192.168.6.202:5555;
    }
 
}
#error_page 405 =200 @fullfuck 或者 error_page 405 = @fullfuck; 2种写法都可以 
#proxy_intercept_errors当上游服务器响应头回来后，可以根据响应状态码的值进行拦截错误处理，与error_page 指令相互结合。用在访问上游服务器出现错误的情况下，这个参数一定要带上并且开启on，否则下边不生效
@:定义命名location区段，这些区段客户段不能访问，只可以由内部产生的请
求来访问，如try_files或error_page等
逻辑就是如果post请求返回了405错误，那么定义405区段请求为GET方式，注意 error_page 要写在location里，而不是server里否则不生效

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

upstream domain_uau {

server 192.168.6.202:5555 weight=5;

}

server {

listen 80;

server_name dev-app-server.sinoif.com;

access_log /var/www/logs/nginx/dev-app-server.sinoif-access.log;

location / {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-NginX-Proxy true;

proxy_pass http://domain_uau/;

add_header Cache-Control "no-cache, no-store";#不生成缓存

error_page 405 =200 @fullfuck;

proxy_intercept_errors on;

}

location @fullfuck {

proxy_method GET;

proxy_pass http://192.168.6.202:5555;

}

#error_page 405 =200 @fullfuck 或者 error_page 405 = @fullfuck; 2种写法都可以

#proxy_intercept_errors当上游服务器响应头回来后，可以根据响应状态码的值进行拦截错误处理，与error_page 指令相互结合。用在访问上游服务器出现错误的情况下，这个参数一定要带上并且开启on，否则下边不生效

@:定义命名location区段，这些区段客户段不能访问，只可以由内部产生的请

求来访问，如try_files或error_page等

逻辑就是如果post请求返回了405错误，那么定义405区段请求为GET方式，注意 error_page 要写在location里，而不是server里否则不生效

Nginx配置跨域请求 Access-Control-Allow-Origin * 当出现403跨域错误的时候 No ‘Access-Control-Allow-Origin’ header is present on the requested resource，需要给Nginx服务器配置响应的header参数：一、解决方案只需要在Nginx的配置文件中配置以下参数：

location / {  
    add_header Access-Control-Allow-Origin *;
    add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';
    add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization';

    if ($request_method = 'OPTIONS') {
        return 204;
    }
}

1

2

3

4

5

6

7

8

9

location / {

add_header Access-Control-Allow-Origin *;

add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';

add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization';

if ($request_method = 'OPTIONS') {

return 204;

}

二、解释 1. Access-Control-Allow-Origin

服务器默认是不被允许跨域的。给Nginx服务器配置`Access-Control-Allow-Origin *`后，表示服务器可以接受所有的请求源（Origin）,即接受所有跨域的请求。

1	服务器默认是不被允许跨域的。给Nginx服务器配置`Access-Control-Allow-Origin *`后，表示服务器可以接受所有的请求源（Origin）,即接受所有跨域的请求。

2. Access-Control-Allow-Headers 是为了防止出现以下错误： Request header field Content-Type is not allowed by Access-Control-Allow-Headers in preflight response. 这个错误表示当前请求Content-Type的值不被支持。其实是我们发起了”application/json”的类型请求导致的。这里涉及到一个概念：预检请求（preflight request）,请看下面”预检请求”的介绍。 3. Access-Control-Allow-Methods 是为了防止出现以下错误： Content-Type Read more…

By sean, 2 years03/23/2023 ago

SECURITY

给节点套上CF的warp，彻底解决access denied error code 1020问题

前言最近的网络最热门的，应该当ChatGPT所属了，在IT开发者界更是：有担忧的，有兴奋的，有求知的，一时兴起万层浪，但是，也有很多小白连最基本的大门都没办法进来，因为流量太大了，已远远超过了ChatGPT的服务器所承受能力，这个访问量已大大超出了你的想像，所以请了一台流量高防大将:CloudFlare，先是真人human验证，再是IP验证，很多新人都对Access deniedAccess denied Error code 1020深有体会吧，今天就聊一下如何彻底地解决这个问题。思路由于CF验证IP，咱们大陆的IP会access denied拒绝，很多海外节点也被拒绝了，因为咱们国人多，热度高，流量大，所以第一排除的就是咱们中国及使用代理最多的中国IP。思路就是套用CloudFlare官方自己的Warp，它对自己人还是友好放行的原理。思路跟解锁Netfilx原生IP一样。条件自己能配置的节点套的IP未被CF自己封 Warp 什么是Warp呢，请查看官方介绍吧官方地址：https://1.1.1.1 用一句话介绍就是，私人的私人代理，如果你也想使用，或者升级到Warp+,推荐连接，我可能获得1G流量安装安装前

curl chat.openai.com

1	curl chat.openai.com

直接显示经典的error code:1020错误，说明我能服务器本身就无法访问ChatGPT，所以你当节点，那也肯定是跑不了1020的了 web

Access denied
You do not have access to chat.openai.com.

The site owner may have set restrictions that prevent you from accessing the site.

Ray ID: 795a3ced8bfa934a
Timestamp: 2023-02-07 06:58:35 UTC
Your IP address: 146.5X.x.x.
Requested URL: chat.openai.com/auth/login
Error reference number: 1020
Server ID: FL_443F37
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36

1

2

3

4

5

6

7

8

9

10

11

12

Access denied

You do not have access to chat.openai.com.

The site owner may have set restrictions that prevent you from accessing the site.

Ray ID: 795a3ced8bfa934a

Timestamp: 2023-02-07 06:58:35 UTC

Your IP address: 146.5X.x.x.

Requested URL: chat.openai.com/auth/login

Error reference number: 1020

Server ID: FL_443F37

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36

我的证实是无法使用的了都是1020错误代码。估计是封IP段安装中第一，先给服务上个快照，折腾坏了也能恢复折腾前的状态第二，安装warp 官网教程：https://pkg.cloudflareclient.com/install 或者通过warp一键配置脚本（项目地址：https://github.com/P3TERX/warp.sh） bash <(curl -fsSL git.io/warp.sh) d 配置xray 登录x-ui控制后台——面板设置——xray相关设置——xray配置模板——复制出来，备份好在高亮文体编辑器里修改（可选） 1，在”outbounds”的代码块添加出站规则 2，在”routing”的代码块添加路由规则，使用域名chat.openai.com及ip138.com(为了测试是否生效)的规则我的xray配置文件

{
  "api": {
    "services": [
      "HandlerService",
      "LoggerService",
      "StatsService"
    ],
    "tag": "api"
  },
  "inbounds": [
    {
      "listen": "127.0.0.1",
      "port": 62789,
      "protocol": "dokodemo-door",
      "settings": {
        "address": "127.0.0.1"
      },
      "tag": "api"
    },
    {
      "listen": "127.0.0.1",
      "port": 44223,
      "protocol": "vmess",
      "settings": {
        "clients": [
          {
            "alterId": 64,
            "id": "xx-51fa-xxx-a26d-xxxx"
          }
        ],
        "disableInsecureEncryption": true
      },
      "sniffing": {
        "destOverride": [
          "http",
          "tls"
        ],
        "enabled": true
      },
      "streamSettings": {
        "network": "ws",
        "security": "none",
        "wsSettings": {
          "headers": {},
          "path": "/ray"
        }
      },
      "tag": "inbound-44223"
    }
  ],
  "outbounds": [
    {
      "protocol": "freedom",
      "settings": {}
    },
    {
      "tag": "chatGPT_proxy",
      "protocol": "socks",
      "settings": {
        "servers": [
          {
            "address": "127.0.0.1",
            "port": 40000
          }
        ]
      }
    },
    {
      "protocol": "blackhole",
      "settings": {},
      "tag": "blocked"
    }
  ],
  "policy": {
    "system": {
      "statsInboundDownlink": true,
      "statsInboundUplink": true
    }
  },
  "log" : {
    "access": "/data/logs/v2ray/access.log",
    "error": "/data/logs/v2ray/error.log",
    "loglevel": "warning"
  },
  "routing": {
    "rules": [
          {
        "type": "field",
        "outboundTag": "chatGPT_proxy",
        "domain": [
          "chat.openai.com",
          "ip138.com"
        ]
      },
      {
        "inboundTag": [
          "api"
        ],
        "outboundTag": "api",
        "type": "field"
      },
      {
        "ip": [
          "geoip:private"
        ],
        "outboundTag": "blocked",
        "type": "field"
      },
      {
        "outboundTag": "blocked",
        "protocol": [
          "bittorrent"
        ],
        "type": "field"
      }
    ]
  },
  "stats": {}
}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

{

"api": {

"services": [

"HandlerService",

"LoggerService",

"StatsService"

],

"tag": "api"

},

"inbounds": [

{

"listen": "127.0.0.1",

"port": 62789,

"protocol": "dokodemo-door",

"settings": {

"address": "127.0.0.1"

},

"tag": "api"

},

{

"listen": "127.0.0.1",

"port": 44223,

"protocol": "vmess",

"settings": {

"clients": [

{

"alterId": 64,

"id": "xx-51fa-xxx-a26d-xxxx"

}

],

"disableInsecureEncryption": true

},

"sniffing": {

"destOverride": [

"http",

"tls"

],

"enabled": true

},

"streamSettings": {

"network": "ws",

"security": "none",

"wsSettings": {

"headers": {},

"path": "/ray"

}

},

"tag": "inbound-44223"

}

],

"outbounds": [

{

"protocol": "freedom",

"settings": {}

},

{

"tag": "chatGPT_proxy",

"protocol": "socks",

"settings": {

"servers": [

{

"address": "127.0.0.1",

"port": 40000

}

]

}

},

{

"protocol": "blackhole",

"settings": {},

"tag": "blocked"

}

],

"policy": {

"system": {

"statsInboundDownlink": true,

"statsInboundUplink": true

}

},

"log" : {

"access": "/data/logs/v2ray/access.log",

"error": "/data/logs/v2ray/error.log",

"loglevel": "warning"

},

"routing": {

"rules": [

{

"type": "field",

"outboundTag": "chatGPT_proxy",

"domain": [

"chat.openai.com",

"ip138.com"

]

},

{

"inboundTag": [

"api"

],

"outboundTag": "api",

"type": "field"

},

{

"ip": [

"geoip:private"

],

"outboundTag": "blocked",

"type": "field"

},

{

"outboundTag": "blocked",

"protocol": [

"bittorrent"

],

"type": "field"

}

]

},

"stats": {}

}

总结方法是从解锁Netflix得到的启示，套了Warp就能解锁CloudFlare对你的节点的封锁了，而且现在CloudFlare拒绝访问的IP限制还不是很严格的，从而这个方法可以拯救你节点。总结思路：套用warp——设置域名分流规则

By sean, 2 years03/18/2023 ago

OS

一键自动创建磁盘并挂载

#! /bin/bash
 
#注:只有硬盘没有做过系统才能这么搞，如果有做过文件系统会直接返回异常信息。
 
diskEmpty=""
nameOfLv="lv_data"
nameOfVg="vg_data"
pathOfLog="/tmp/mkfs.log"
pathOfMount="/data"
 
echo -e "\n\n`date '+%Y-%m-%d %H:%M:%S'` ###" >> $pathOfLog
#验证磁盘是否为空，如果为空则把磁盘名称放入diskEmpty中，空格区分
for i in `lsblk -l | grep disk |awk '{print $1}'`； do
    isEmpty=1
    for j in `hexdump -n 170000 -C /dev/$i | head -1000 | awk -F'|' '{print $2}' | grep -v '^$' |awk -F'.' '{print NF-1}'`； do
        if [ $j -ne 16 ];then
            echo "Warnning: disk /dev/$i have data。please check the data of the disk 。" >> $pathOfLog
            isEmpty=0
            break
        fi
    done
    if [ $isEmpty -eq 1 ];then
        diskEmpty="$diskEmpty"" /dev/$i"
    fi
done
echo $diskEmpty
 
#创建逻辑卷和逻辑盘
vgcreate $nameOfVg $diskEmpty &>> $pathOfLog
lvcreate -l 100%FREE -n $nameOfLv $nameOfVg &>> $pathOfLog
 
#验证是否创建成功，如果失败则退出
lvscan |awk -F"'" '{print $2}' | awk -F '/' '{print $4}' | grep "^$nameOfLv$" &>> $pathOfLog
 
if [ $? -ne 0 ];then
echo "ERROR: no exists device of $nameOfLv named !" &>> $pathOfLog
exit 1
fi
 
#获取用于格式化文件系统的真实绝对路径
pathOfDeviceFormkfs=`lvscan |awk -F"'" '{print $2}'|grep "$nameOfLv$"`
echo 'pathOfDeviceFormkfs:' `lvscan |awk -F"'" '{print $2}'|grep "$nameOfLv$"`
 
#在该逻辑卷上制作文件系统
mkfs.xfs $pathOfDeviceFormkfs &>> $pathOfLog
 
#获取新创建的逻辑卷的真实绝对路径
pathOfDevice=`blkid |awk -F":" '{print $1}'|grep "$nameOfLv$"`
#检查挂载点是否存在，不存在则创建
[ ! -d $pathOfMount ] && mkdir $pathOfMount &>> $pathOfLog
 
#获取该逻辑卷的uuid和文件系统类型
infoOfDevice=`blkid |grep "^$pathOfDevice" | awk '{print $2" "$3}' | awk -F'"' '{print $2" "$4}'`
 
uuid=`echo "$infoOfDevice" | awk '{print $1}'`
typeOfFs=`echo "$infoOfDevice" | awk '{print $2}'`
 
#保证两个变量值存在，如果不存在就不做任何临时和永久的挂载操作。
[ -z "$uuid" ] && echo "uuid is null!" &>> $pathOfLog && exit
[ -z "$pathOfDevice" ] && echo "pathOfDevice is null!" &>> $pathOfLog && exit
#将挂载信息写入fstab并挂载
for i in `cat -n /etc/fstab | awk '{print $1" "$2}' | grep -E "$uuid|$pathOfDevice" | awk '{print $1}'`； do
    sed -n "${i}p" /etc/fstab >> $pathOfLog
    echo "warnning: 该数据将删除！"
done
 
echo "UUID=$uuid $pathOfMount $typeOfFs defaults 0 0" >> /etc/fstab
 
mount -a &>> $pathOfLog

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

#! /bin/bash

#注:只有硬盘没有做过系统才能这么搞，如果有做过文件系统会直接返回异常信息。

diskEmpty=""

nameOfLv="lv_data"

nameOfVg="vg_data"

pathOfLog="/tmp/mkfs.log"

pathOfMount="/data"

echo -e "\n\n`date '+%Y-%m-%d %H:%M:%S'` ###" >> $pathOfLog

#验证磁盘是否为空，如果为空则把磁盘名称放入diskEmpty中，空格区分

for i in `lsblk -l | grep disk |awk '{print $1}'`； do

isEmpty=1

for j in `hexdump -n 170000 -C /dev/$i | head -1000 | awk -F'|' '{print $2}' | grep -v '^$' |awk -F'.' '{print NF-1}'`； do

if [ $j -ne 16 ];then

echo "Warnning: disk /dev/$i have data。please check the data of the disk 。" >> $pathOfLog

isEmpty=0

break

fi

done

if [ $isEmpty -eq 1 ];then

diskEmpty="$diskEmpty"" /dev/$i"

fi

done

echo $diskEmpty

#创建逻辑卷和逻辑盘

vgcreate $nameOfVg $diskEmpty &>> $pathOfLog

lvcreate -l 100%FREE -n $nameOfLv $nameOfVg &>> $pathOfLog

#验证是否创建成功，如果失败则退出

lvscan |awk -F"'" '{print $2}' | awk -F '/' '{print $4}' | grep "^$nameOfLv$" &>> $pathOfLog

if [ $? -ne 0 ];then

echo "ERROR: no exists device of $nameOfLv named !" &>> $pathOfLog

exit 1

fi

#获取用于格式化文件系统的真实绝对路径

pathOfDeviceFormkfs=`lvscan |awk -F"'" '{print $2}'|grep "$nameOfLv$"`

echo 'pathOfDeviceFormkfs:' `lvscan |awk -F"'" '{print $2}'|grep "$nameOfLv$"`

#在该逻辑卷上制作文件系统

mkfs.xfs $pathOfDeviceFormkfs &>> $pathOfLog

#获取新创建的逻辑卷的真实绝对路径

pathOfDevice=`blkid |awk -F":" '{print $1}'|grep "$nameOfLv$"`

#检查挂载点是否存在，不存在则创建

[ ! -d $pathOfMount ] && mkdir $pathOfMount &>> $pathOfLog

#获取该逻辑卷的uuid和文件系统类型

infoOfDevice=`blkid |grep "^$pathOfDevice" | awk '{print $2" "$3}' | awk -F'"' '{print $2" "$4}'`

uuid=`echo "$infoOfDevice" | awk '{print $1}'`

typeOfFs=`echo "$infoOfDevice" | awk '{print $2}'`

#保证两个变量值存在，如果不存在就不做任何临时和永久的挂载操作。

[ -z "$uuid" ] && echo "uuid is null!" &>> $pathOfLog && exit

[ -z "$pathOfDevice" ] && echo "pathOfDevice is null!" &>> $pathOfLog && exit

#将挂载信息写入fstab并挂载

for i in `cat -n /etc/fstab | awk '{print $1" "$2}' | grep -E "$uuid|$pathOfDevice" | awk '{print $1}'`； do

sed -n "${i}p" /etc/fstab >> $pathOfLog

echo "warnning: 该数据将删除！"

done

echo "UUID=$uuid $pathOfMount $typeOfFs defaults 0 0" >> /etc/fstab

mount -a &>> $pathOfLog

By sean, 2 years03/18/2023 ago

KUBERNETES

Kubernetes 网络

Overview 本文将探讨 Kubernetes 中的网络模型，以及对各种网络模型进行分析。 Underlay Network Model 什么是 Underlay Network 底层网络 Underlay Network 顾名思义是指网络设备基础设施，如交换机，路由器, DWDM 使用网络介质将其链接成的物理网络拓扑，负责网络之间的数据包传输。 Underlay network topology underlay network 可以是二层，也可以是三层；二层的典型例子是以太网 Ethernet，三层是的典型例子是互联网 Internet。而工作于二层的技术是 vlan，工作在三层的技术是由 OSPF, BGP 等协议组成。 k8s 中的 underlay network 在 kubernetes 中，underlay network 中比较典型的例子是通过将宿主机作为路由器设备，Pod 的网络则通过学习路由条目从而实现跨节点通讯。 underlay network topology in kubernetes 这种模型下典型的有 flannel 的 host-gw 模式与 calico Read more…

By sean, 2 years03/08/2023 ago

KUBERNETES

信创系统安装docker-ce

在统信系统上安装docker-ce, 一般官方源只有docker-engine，对于基于yum源管理的系统，需要配置使用centos源来安装docker-ce 访问https://mirrors.aliyun.com/centos/，发现在阿里源当中，只有centos8提供了aarch64架构源，所以需要配置8版本阿里repo源地址： https://mirrors.aliyun.com/repo/ 一。首先配置阿里镜像源

curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-8.repo
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

1 2	curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-8.repo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

其中Centos-8.repo也可以换成Centos-vault-8.5.2111的源：

curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-vault-8.5.2111.repo

1	curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-vault-8.5.2111.repo

定义 yum 变量&修改 repo 修改 centos 和 docker repo文件中的 $releasever 为 centos_version ，原因是在统信服务器操作系统中 $releasever被修改为了 10，而我们需要使用 centos 8的镜像源，如果你不替换，基本上仓库的每一个地址都是404。

sed -i 's/$releasever/8/g' /etc/yum.repos.d/docker-ce.repo
sed -i 's/$releasever/8/g' /etc/yum.repos.d/CentOS-Base.repo

1 2	sed -i 's/$releasever/8/g' /etc/yum.repos.d/docker-ce.repo sed -i 's/$releasever/8/g' /etc/yum.repos.d/CentOS-Base.repo

yum makecache 安装docker: yum install docker-ce-20.10.17 docker-ce-cli-20.10.17 containerd.io-1.6.7

By sean, 2 years03/02/2023 ago