未分类

HANDBOOK

解决ansible mitogen 0.3.0+ 插件未渲染ansible_ssh_common_args模板变量问题

问题使用ansible mitogen 0.3.4插件进行kubespray安装时，报错： EOF on stream; last 100 lines received:\nssh: Could not resolve hostname {%: Name or service not known 分析经过debug分析，kubespray-default默认定义了如下变量模板：

ansible_ssh_common_args: |
"{% if 'bastion' in groups['all'] %} -o ProxyCommand='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p 
-p {{ hostvars['bastion']['ansible_port'] | default(22) }} {{ hostvars['bastion']['ansible_user'] }}@{{ hostvars['bastion']['ansible_host'] }} {% if ansible_ssh_private_key_file is defined %}
-i {{ ansible_ssh_private_key_file }}{% endif %} ' {% endif %}"

ansible_ssh_common_args: |

"{% if 'bastion' in groups['all'] %} -o ProxyCommand='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p

-p {{ hostvars['bastion']['ansible_port'] | default(22) }} {{ hostvars['bastion']['ansible_user'] }}@{{ hostvars['bastion']['ansible_host'] }} {% if ansible_ssh_private_key_file is defined %}

-i {{ ansible_ssh_private_key_file }}{% endif %} ' {% endif %}"

但是通过ansible role去执行后，通过mitogen进行ssh并没有渲染出来变量：

mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l root -p 22 -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" 
-o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -o "GlobalKnownHostsFile /dev/null" -C -o ControlMaster=auto 
-o ControlPersist=60s {% if bastion in groups[all] %} -o "ProxyCommand=ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p 
-p {{ hostvars[bastion][ansible_port] | default(22) }} {{ hostvars[bastion][ansible_user] }}@{{ hostvars[bastion][ansible_host] }} {% if ansible_ssh_private_key_file is defined %}
-i {{ ansible_ssh_private_key_file }}{% endif %} " {% endif %} 172.20.59.209 /usr/bin/python -c

mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l root -p 22 -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10"

-o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -o "GlobalKnownHostsFile /dev/null" -C -o ControlMaster=auto

-o ControlPersist=60s {% if bastion in groups[all] %} -o "ProxyCommand=ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p

-p {{ hostvars[bastion][ansible_port] | default(22) }} {{ hostvars[bastion][ansible_user] }}@{{ hostvars[bastion][ansible_host] }} {% if ansible_ssh_private_key_file is defined %}

-i {{ ansible_ssh_private_key_file }}{% endif %} " {% endif %} 172.20.59.209 /usr/bin/python -c

可以看到，mitogen的ssh将ansible_ssh_common_args原封不动地输出来了修改源码验证：修改transport_config.py，增加debug信息：

476     def ssh_args(self):
478         print('------------>')
479         print('ssh_args: ',C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))
480         print('ssh_common_args: ',C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))
481         print('ssh_extra_args: ', C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))
482         print('----------->')
483         return [
484             mitogen.core.to_text(term)
485             for s in (
486                 C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})),
487                 C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})),
488                 C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {}))
489                 #C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=local_vars),
490                 #C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=local_vars),
491                 #C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=local_vars)
492             )
493             for term in ansible.utils.shlex.shlex_split(s or '')
494         ]

476 def ssh_args(self):

478 print('------------>')

479 print('ssh_args: ',C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))

480 print('ssh_common_args: ',C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))

481 print('ssh_extra_args: ', C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})))

482 print('----------->')

483 return [

484 mitogen.core.to_text(term)

485 for s in (

486 C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})),

487 C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {})),

488 C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=self._task_vars.get("vars", {}))

489 #C.config.get_config_value("ssh_args", plugin_type="connection", plugin_name="ssh", variables=local_vars),

490 #C.config.get_config_value("ssh_common_args", plugin_type="connection", plugin_name="ssh", variables=local_vars),

491 #C.config.get_config_value("ssh_extra_args", plugin_type="connection", plugin_name="ssh", variables=local_vars)

492 )

493 for term in ansible.utils.shlex.shlex_split(s or '')

494 ]

下面是输出：

------------>
ssh_args:  -o ControlMaster=auto -o ControlPersist=1d -o UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -o Compression=yes -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o AddKeysToAgent=yes -o ControlPath=~/.ssh/%r@%h-%p
ssh_common_args:  {% if 'bastion' in groups['all'] %} -o ProxyCommand='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p -p {{ hostvars['bastion']['ansible_port'] | default(22) }} {{ hostvars['bastion']['ansible_user'] }}@{{ hostvars['bastion']['ansible_host'] }} {% if ansible_ssh_private_key_file is defined %}-i {{ ansible_ssh_private_key_file }}{% endif %} ' {% endif %}
ssh_extra_args:  
----------->

------------>

ssh_args: -o ControlMaster=auto -o ControlPersist=1d -o UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -o Compression=yes -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o AddKeysToAgent=yes -o ControlPath=~/.ssh/%r@%h-%p

ssh_common_args: {% if 'bastion' in groups['all'] %} -o ProxyCommand='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -W %h:%p -p {{ hostvars['bastion']['ansible_port'] | default(22) }} {{ hostvars['bastion']['ansible_user'] }}@{{ hostvars['bastion']['ansible_host'] }} {% if ansible_ssh_private_key_file is defined %}-i {{ ansible_ssh_private_key_file }}{% endif %} ' {% endif %}

ssh_extra_args:

----------->

可以看到ssh_common_args变量没有渲染解决将_task_vars.get(“vars”, {}) 改为 _task_vars.get(“hostvars”, {})，从hostvars取值搜索官方issures: https://github.com/mitogen-hq/mitogen/pull/956 commit ac34252bcccb60b50e6a8ed3a3b2f42d256d62e0 https://github.com/mitogen-hq/mitogen/pull/956/commits/ac34252bcccb60b50e6a8ed3a3b2f42d256d62e0 总结 Read more…

By sean, 5 months12/07/2023 ago

MIDDLEWARE

nginx反向代理Harbor/配置https

使用docker-compose安装harbor，配置ssl证书后使用nginx反向代理到harbor.配置后安装docker可以直接用域名登录harbor，无需配置私有仓库 nginx配置 1.修改harbor配置文件

hostname: ycr.yyiuap.com
http:
  port: 81
external_url: https://ycr.yyiuap.com

hostname: ycr.yyiuap.com

http:

port: 81

external_url: https://ycr.yyiuap.com

2.nginx配置

upstream harbor {
    server x.x.x.246:81;
}

server {
    listen 81;
    server_name ycr.yyiuap.com;
    location / {
        proxy_pass http://harbor;
        client_max_body_size 0;
        proxy_connect_timeout 90;
        proxy_read_timeout 90;
        proxy_buffer_size 4k;
        proxy_buffers 6 32k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
    }
}


server {
    listen       443 ssl;
    listen       [::]:443;
    server_name  ycr.yyiuap.com;

    ssl_certificate /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.cert;
    ssl_certificate_key /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    client_max_body_size 0;

    chunked_transfer_encoding on;

    location /v2/ {
      proxy_pass http://harbor/v2/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_buffering off;
      proxy_request_buffering off;
    }

    location / {
        proxy_pass http://harbor;
        client_max_body_size 0;
        proxy_connect_timeout 90;
        proxy_read_timeout 90;
        proxy_buffer_size 4k;
        proxy_buffers 6 32k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect http:// $scheme://;
    }
}

upstream harbor {

server x.x.x.246:81;

}

server {

listen 81;

server_name ycr.yyiuap.com;

location / {

proxy_pass http://harbor;

client_max_body_size 0;

proxy_connect_timeout 90;

proxy_read_timeout 90;

proxy_buffer_size 4k;

proxy_buffers 6 32k;

proxy_busy_buffers_size 64k;

proxy_temp_file_write_size 64k;

}

server {

listen 443 ssl;

listen [::]:443;

server_name ycr.yyiuap.com;

ssl_certificate /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.cert;

ssl_certificate_key /usr/local/nginx/conf/certs/ycr.yyiuap.com/ycr.yyiuap.com.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';

ssl_prefer_server_ciphers on;

ssl_session_cache shared:SSL:10m;

client_max_body_size 0;

chunked_transfer_encoding on;

location /v2/ {

proxy_pass http://harbor/v2/;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_buffering off;

proxy_request_buffering off;

}

location / {

proxy_pass http://harbor;

client_max_body_size 0;

proxy_connect_timeout 90;

proxy_read_timeout 90;

proxy_buffer_size 4k;

proxy_buffers 6 32k;

proxy_busy_buffers_size 64k;

proxy_temp_file_write_size 64k;

proxy_set_header Host $host:$server_port;

proxy_set_header X-Forwarded-For $remote_addr;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_redirect http:// $scheme://;

}

另：创建证书脚本：

#!/bin/bash

registry_url="ycr.yyiuap.com"


# 生成使用的相关证书
openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \
    -key ca.key \
    -out ca.crt

openssl genrsa -out $registry_url.key 4096

openssl req -sha512 -new \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \
    -key $registry_url.key \
    -out $registry_url.csr

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1=$registry_url
DNS.2=reg.yyuap
DNS.3=`hostname`
EOF

openssl x509 -req -sha512 -days 3650 \
    -extfile v3.ext \
    -CA ca.crt -CAkey ca.key -CAcreateserial \
    -in $registry_url.csr \
    -out $registry_url.crt

openssl x509 -inform PEM -in $registry_url.crt -out $registry_url.cert

mkdir -p /etc/docker/certs.d/$registry_url/

cp $registry_url.cert /etc/docker/certs.d/$registry_url/
cp $registry_url.key /etc/docker/certs.d/$registry_url/
cp ca.crt /etc/docker/certs.d/$registry_url/

#将证书加入系统级别信任
cp /etc/docker/certs.d/ca.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust extract
systemctl daemon-reload && systemctl restart docker

#!/bin/bash

registry_url="ycr.yyiuap.com"

# 生成使用的相关证书

openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 \

-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \

-key ca.key \

-out ca.crt

openssl genrsa -out $registry_url.key 4096

openssl req -sha512 -new \

-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$registry_url" \

-key $registry_url.key \

-out $registry_url.csr

cat > v3.ext <<-EOF

authorityKeyIdentifier=keyid,issuer

basicConstraints=CA:FALSE

keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment

extendedKeyUsage = serverAuth

subjectAltName = @alt_names

[alt_names]

DNS.1=$registry_url

DNS.2=reg.yyuap

DNS.3=`hostname`

EOF

openssl x509 -req -sha512 -days 3650 \

-extfile v3.ext \

-CA ca.crt -CAkey ca.key -CAcreateserial \

-in $registry_url.csr \

-out $registry_url.crt

openssl x509 -inform PEM -in $registry_url.crt -out $registry_url.cert

mkdir -p /etc/docker/certs.d/$registry_url/

cp $registry_url.cert /etc/docker/certs.d/$registry_url/

cp $registry_url.key /etc/docker/certs.d/$registry_url/

cp ca.crt /etc/docker/certs.d/$registry_url/

#将证书加入系统级别信任

cp /etc/docker/certs.d/ca.crt /etc/pki/ca-trust/source/anchors/

update-ca-trust extract

systemctl daemon-reload && systemctl restart docker

问题： docker pull ycr.yyiuap.com/base/golang:alphine-node-3 Error response from daemon: received unexpected HTTP status: 503 Service Unavailable 解决：去掉 http_proxy代理即可参考：https://blog.csdn.net/oscarun/article/details/121395218 1、request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)错误： [root@iZrj9j76z8dlull9vqa4tqZ ~]# docker pull harbor.xxx.cn:443/1-1/mytomcat:8.5.52 Error response from Read more…

By sean, 11 months06/09/2023 ago

MIDDLEWARE

nginx问题解决

nginx转发post请求静态页面405 背景:用户支付成功后的回调是个静态页面。由于from表单连续提交是post方式，所以会报405 not allowed 错误。常识:使用post方式请求js、html这样的静态文件一般的web服务器都会返回405 Method Not Allowed。因为默认情况下，nginx、apache、IIs等web服务无法响应静态页面的post请求，后端用来处理post请求，生产环境中不会有此问题（一般都不允许配置静态页面的post请求）问题:为什么默认不支持静态页面post请求呢？首先了解一下post请求方法，post请求一般用于提交表单或上传文件，post请求会导致新资源的建立或旧资源的更改。就安全方面来说（排除url地址的透明性），它对比get请求会有更改资源的情况，有些静态资源是不允许更改的，所以默认情况下web服务器上的静态资源都不允许发起post请求。网上说的有很多种，重新编译nginx，设置正则匹配可以访问html文件类啊什么的，都不好用，而且基本都是你抄我我抄你，没有实际应用过。乱糟糟，最后本人用下面这种方式解决了问题。

upstream domain_uau {
  server 192.168.6.202:5555 weight=5;
}
server {
    listen              80;
    server_name         dev-app-server.sinoif.com;
    access_log  /var/www/logs/nginx/dev-app-server.sinoif-access.log;
    location / {
        proxy_set_header Host $host;
        proxy_set_header  X-Real-IP        $remote_addr;
        proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://domain_uau/;
        add_header Cache-Control "no-cache, no-store";#不生成缓存
        error_page 405 =200 @fullfuck;
        proxy_intercept_errors on;
    }
    location @fullfuck {
        proxy_method GET;
        proxy_pass http://192.168.6.202:5555;
    }
 
}
#error_page 405 =200 @fullfuck 或者 error_page 405 = @fullfuck; 2种写法都可以 
#proxy_intercept_errors当上游服务器响应头回来后，可以根据响应状态码的值进行拦截错误处理，与error_page 指令相互结合。用在访问上游服务器出现错误的情况下，这个参数一定要带上并且开启on，否则下边不生效
@:定义命名location区段，这些区段客户段不能访问，只可以由内部产生的请
求来访问，如try_files或error_page等
逻辑就是如果post请求返回了405错误，那么定义405区段请求为GET方式，注意 error_page 要写在location里，而不是server里否则不生效

upstream domain_uau {

server 192.168.6.202:5555 weight=5;

}

server {

listen 80;

server_name dev-app-server.sinoif.com;

access_log /var/www/logs/nginx/dev-app-server.sinoif-access.log;

location / {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-NginX-Proxy true;

proxy_pass http://domain_uau/;

add_header Cache-Control "no-cache, no-store";#不生成缓存

error_page 405 =200 @fullfuck;

proxy_intercept_errors on;

}

location @fullfuck {

proxy_method GET;

proxy_pass http://192.168.6.202:5555;

}

#error_page 405 =200 @fullfuck 或者 error_page 405 = @fullfuck; 2种写法都可以

#proxy_intercept_errors当上游服务器响应头回来后，可以根据响应状态码的值进行拦截错误处理，与error_page 指令相互结合。用在访问上游服务器出现错误的情况下，这个参数一定要带上并且开启on，否则下边不生效

@:定义命名location区段，这些区段客户段不能访问，只可以由内部产生的请

求来访问，如try_files或error_page等

逻辑就是如果post请求返回了405错误，那么定义405区段请求为GET方式，注意 error_page 要写在location里，而不是server里否则不生效

Nginx配置跨域请求 Access-Control-Allow-Origin * 当出现403跨域错误的时候 No ‘Access-Control-Allow-Origin’ header is present on the requested resource，需要给Nginx服务器配置响应的header参数：一、解决方案只需要在Nginx的配置文件中配置以下参数：

location / {  
    add_header Access-Control-Allow-Origin *;
    add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';
    add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization';

    if ($request_method = 'OPTIONS') {
        return 204;
    }
}

location / {

add_header Access-Control-Allow-Origin *;

add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS';

add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization';

if ($request_method = 'OPTIONS') {

return 204;

}

二、解释 1. Access-Control-Allow-Origin

服务器默认是不被允许跨域的。给Nginx服务器配置`Access-Control-Allow-Origin *`后，表示服务器可以接受所有的请求源（Origin）,即接受所有跨域的请求。

1	服务器默认是不被允许跨域的。给Nginx服务器配置`Access-Control-Allow-Origin *`后，表示服务器可以接受所有的请求源（Origin）,即接受所有跨域的请求。

2. Access-Control-Allow-Headers 是为了防止出现以下错误： Request header field Content-Type is not allowed by Access-Control-Allow-Headers in preflight response. 这个错误表示当前请求Content-Type的值不被支持。其实是我们发起了”application/json”的类型请求导致的。这里涉及到一个概念：预检请求（preflight request）,请看下面”预检请求”的介绍。 3. Access-Control-Allow-Methods 是为了防止出现以下错误： Content-Type Read more…

By sean, 1 year03/23/2023 ago

如何精简Docker镜像

精简Docker镜像的五种通用方法精简 Docker 镜像的好处很多，不仅可以节省存储空间和带宽，还能减少安全隐患。优化镜像大小的手段多种多样，因服务所使用的基础开发语言不同而有差异。本文将介绍精简 Docker 镜像的几种通用方法。精简 Docker 镜像大小的必要性 Docker 镜像由很多镜像层（Layers）组成（最多 127 层），镜像层依赖于一系列的底层技术，比如文件系统（filesystems）、写时复制（copy-on-write）、联合挂载（union mounts）等技术，你可以查看Docker 社区文档以了解更多有关 Docker 存储驱动的内容，这里就不再赘述技术细节。总的来说，Dockerfile 中的每条指令都会创建一个镜像层，继而会增加整体镜像的尺寸。下面是精简 Docker 镜像尺寸的好处：减少构建时间减少磁盘使用量减少下载时间因为包含文件少，攻击面减小，提高了安全性提高部署速度五点建议减小Docker镜像尺寸一、优化基础镜像优化基础镜像的方法就是选用合适的更小的基础镜像，常用的 Linux 系统镜像一般有 Ubuntu、CentOs、Alpine，其中 Alpine 更推荐使用。大小对比如下： Alpine 是一个高度精简又包含了基本工具的轻量级 Linux 发行版，基础镜像只有 4.41M，各开发语言和框架都有基于 Alpine 制作的基础镜像，强烈推荐使用它。查看上面的镜像尺寸对比结果，你会发现最小的镜像也有 4.41M，那么有办法构建更小的镜像吗？答案是肯定的，例如 http://gcr.io/google_containers/pause-amd64:3.1 镜像仅有 742KB。为什么这个镜像能这么小？在为大家解密之前，再推荐两个基础镜像： 1、scratch 镜像 scratch 是一个空镜像，只能用于构建其他镜像，比如你要运行一个包含所有依赖的二进制文件，如Golang 程序，可以直接使用 scratch Read more…

By sean, 2 years ago

未分类

gitlab pipeline 自动构建

使用gitlab pipeline自动构建，需要为项目配置runner, 安装方法为：

curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"
curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm64"

chmod +x /usr/local/bin/gitlab-runner
useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
gitlab-runner start
gitlab-runner register  # 按照提示输入相关信息

curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"

curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm64"

chmod +x /usr/local/bin/gitlab-runner

useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner

gitlab-runner start

gitlab-runner register # 按照提示输入相关信息

构建一个docker:buildx镜像做测试 .gitlab-ci.yml：

variables:
  IMAGE_REPO: $REGISTRY/library/docker
  IMAGE_TAG: buildx

before_script:
  - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY

build:
  image: docker:latest
  stage: build
  services:
    - name: docker:dind
  script:
    - docker build -t $IMAGE_REPO:$IMAGE_TAG -t $IMAGE_REPO .
    - docker push $IMAGE_REPO
    - docker push $IMAGE_REPO:$IMAGE_TAG
  only:
    - master

variables:

IMAGE_REPO: $REGISTRY/library/docker

IMAGE_TAG: buildx

before_script:

- docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY

build:

image: docker:latest

stage: build

services:

- name: docker:dind

script:

- docker build -t $IMAGE_REPO:$IMAGE_TAG -t $IMAGE_REPO .

- docker push $IMAGE_REPO

- docker push $IMAGE_REPO:$IMAGE_TAG

only:

- master

Dockerfile:

FROM docker:latest

ARG ALPINE_REPO=mirrors.aliyun.com

RUN mkdir -p /root/.docker/cli-plugins \
    && wget -qO ~/.docker/cli-plugins/docker-buildx $BASE_URL/extra/buildx-v0.8.2.linux-amd64 \
    && wget -qO /usr/local/bin/skopeo $BASE_URL/extra/skopeo-linux-amd64 \
    && ln -s /usr/local/bin/skopeo /usr/bin/skopeo \
    && wget -qO /usr/local/bin/yq $BASE_URL/extra/yq-linux-amd64 \
    && chmod +x /root/.docker/cli-plugins/docker-buildx /usr/local/bin/* \
    && [[ ! -z $ALPINE_REPO ]] && sed -i "s/dl-cdn.alpinelinux.org/$ALPINE_REPO/" /etc/apk/repositories || true \
    && apk --no-cache add wget \
    && /bin/rm -rf /tmp/* /var/cache/apk/*

FROM docker:latest

ARG ALPINE_REPO=mirrors.aliyun.com

RUN mkdir -p /root/.docker/cli-plugins \

&& wget -qO ~/.docker/cli-plugins/docker-buildx $BASE_URL/extra/buildx-v0.8.2.linux-amd64 \

&& wget -qO /usr/local/bin/skopeo $BASE_URL/extra/skopeo-linux-amd64 \

&& ln -s /usr/local/bin/skopeo /usr/bin/skopeo \

&& wget -qO /usr/local/bin/yq $BASE_URL/extra/yq-linux-amd64 \

&& chmod +x /root/.docker/cli-plugins/docker-buildx /usr/local/bin/* \

&& [[ ! -z $ALPINE_REPO ]] && sed -i "s/dl-cdn.alpinelinux.org/$ALPINE_REPO/" /etc/apk/repositories || true \

&& apk --no-cache add wget \

&& /bin/rm -rf /tmp/* /var/cache/apk/*

运行时提示Got permission denied while trying to connect to the Docker daemon socket，解决方法：在runner服务器上修改gitlab-runner用户加入docker组：

usermod -a -G docker gitlab-runner

1	usermod -a -G docker gitlab-runner

By sean, 2 years07/07/2022 ago

未分类

harbor v2.5.1安装

使用docker-compose来安装harbor v2.5.1，安装过程中有一些问题，下面记录一下：下载离线安装包：

wget https://github.com/goharbor/harbor/releases/download/v2.5.1/harbor-offline-installer-v2.5.1.tgz

1	wget https://github.com/goharbor/harbor/releases/download/v2.5.1/harbor-offline-installer-v2.5.1.tgz

2. 准备docker-compose.yml和harbor.yml: docker-compose.yml:

version: '2.3'
services:
  log:
    image: goharbor/harbor-log:v2.5.1
    container_name: harbor-log
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - DAC_OVERRIDE
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/log/:/var/log/docker/:z
      - type: bind
        source: ./common/config/log/logrotate.conf
        target: /etc/logrotate.d/logrotate.conf
      - type: bind
        source: ./common/config/log/rsyslog_docker.conf
        target: /etc/rsyslog.d/rsyslog_docker.conf
    ports:
      - 127.0.0.1:1514:10514
    networks:
      - harbor
  registry:
    image: goharbor/registry-photon:v2.5.1
    container_name: registry
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/registry:/storage:z
      - ./common/config/registry/:/etc/registry/:z
      - type: bind
        source: /ali-nas/harbor/secret/registry/root.crt
        target: /etc/registry/root.crt
      - type: bind
        source: ./common/config/shared/trust-certificates
        target: /harbor_cust_cert
    networks:
      - harbor
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "registry"
  registryctl:
    image: goharbor/harbor-registryctl:v2.5.1
    container_name: registryctl
    env_file:
      - ./common/config/registryctl/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/registry:/storage:z
      - ./common/config/registry/:/etc/registry/:z
      - type: bind
        source: ./common/config/registryctl/config.yml
        target: /etc/registryctl/config.yml
      - type: bind
        source: ./common/config/shared/trust-certificates
        target: /harbor_cust_cert
    networks:
      - harbor
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "registryctl"
  postgresql:
    image: goharbor/harbor-db:v2.5.1
    container_name: harbor-db
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - DAC_OVERRIDE
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/database:/var/lib/postgresql/data:z
    networks:
      harbor:
    env_file:
      - ./common/config/db/env
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "postgresql"
    shm_size: '1gb'
  core:
    image: goharbor/harbor-core:v2.5.1
    container_name: harbor-core
    env_file:
      - ./common/config/core/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/ca_download/:/etc/core/ca/:z
      - /ali-nas/harbor/:/data/:z
      - ./common/config/core/certificates/:/etc/core/certificates/:z
      - type: bind
        source: ./common/config/core/app.conf
        target: /etc/core/app.conf
      - type: bind
        source: /ali-nas/harbor/secret/core/private_key.pem
        target: /etc/core/private_key.pem
      - type: bind
        source: /ali-nas/harbor/secret/keys/secretkey
        target: /etc/core/key
      - type: bind
        source: ./common/config/shared/trust-certificates
        target: /harbor_cust_cert
    networks:
      harbor:
    depends_on:
      - log
      - registry
      - redis
      - postgresql
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "core"
  portal:
    image: goharbor/harbor-portal:v2.5.1
    container_name: harbor-portal
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - NET_BIND_SERVICE
    volumes:
      - type: bind
        source: ./common/config/portal/nginx.conf
        target: /etc/nginx/nginx.conf
    networks:
      - harbor
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "portal"

  jobservice:
    image: goharbor/harbor-jobservice:v2.5.1
    container_name: harbor-jobservice
    env_file:
      - ./common/config/jobservice/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/job_logs:/var/log/jobs:z
      - type: bind
        source: ./common/config/jobservice/config.yml
        target: /etc/jobservice/config.yml
      - type: bind
        source: ./common/config/shared/trust-certificates
        target: /harbor_cust_cert
    networks:
      - harbor
    depends_on:
      - core
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "jobservice"
  redis:
    image: goharbor/redis-photon:v2.5.1
    container_name: redis
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /ali-nas/harbor/redis:/var/lib/redis
    networks:
      harbor:
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "redis"
  proxy:
    image: goharbor/nginx-photon:v2.5.1
    container_name: nginx
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - NET_BIND_SERVICE
    volumes:
      - ./common/config/nginx:/etc/nginx:z
      - type: bind
        source: ./common/config/shared/trust-certificates
        target: /harbor_cust_cert
    networks:
      - harbor
    ports:
      - 81:8080
    depends_on:
      - registry
      - core
      - portal
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://localhost:1514"
        tag: "proxy"
networks:
  harbor:
    external: false

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

version: '2.3'

services:

log:

image: goharbor/harbor-log:v2.5.1

container_name: harbor-log

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- DAC_OVERRIDE

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/log/:/var/log/docker/:z

- type: bind

source: ./common/config/log/logrotate.conf

target: /etc/logrotate.d/logrotate.conf

- type: bind

source: ./common/config/log/rsyslog_docker.conf

target: /etc/rsyslog.d/rsyslog_docker.conf

ports:

- 127.0.0.1:1514:10514

networks:

- harbor

registry:

image: goharbor/registry-photon:v2.5.1

container_name: registry

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/registry:/storage:z

- ./common/config/registry/:/etc/registry/:z

- type: bind

source: /ali-nas/harbor/secret/registry/root.crt

target: /etc/registry/root.crt

- type: bind

source: ./common/config/shared/trust-certificates

target: /harbor_cust_cert

networks:

- harbor

depends_on:

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "registry"

registryctl:

image: goharbor/harbor-registryctl:v2.5.1

container_name: registryctl

env_file:

- ./common/config/registryctl/env

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/registry:/storage:z

- ./common/config/registry/:/etc/registry/:z

- type: bind

source: ./common/config/registryctl/config.yml

target: /etc/registryctl/config.yml

- type: bind

source: ./common/config/shared/trust-certificates

target: /harbor_cust_cert

networks:

- harbor

depends_on:

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "registryctl"

postgresql:

image: goharbor/harbor-db:v2.5.1

container_name: harbor-db

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- DAC_OVERRIDE

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/database:/var/lib/postgresql/data:z

networks:

harbor:

env_file:

- ./common/config/db/env

depends_on:

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "postgresql"

shm_size: '1gb'

core:

image: goharbor/harbor-core:v2.5.1

container_name: harbor-core

env_file:

- ./common/config/core/env

restart: always

cap_drop:

- ALL

cap_add:

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/ca_download/:/etc/core/ca/:z

- /ali-nas/harbor/:/data/:z

- ./common/config/core/certificates/:/etc/core/certificates/:z

- type: bind

source: ./common/config/core/app.conf

target: /etc/core/app.conf

- type: bind

source: /ali-nas/harbor/secret/core/private_key.pem

target: /etc/core/private_key.pem

- type: bind

source: /ali-nas/harbor/secret/keys/secretkey

target: /etc/core/key

- type: bind

source: ./common/config/shared/trust-certificates

target: /harbor_cust_cert

networks:

harbor:

depends_on:

- log

- registry

- redis

- postgresql

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "core"

portal:

image: goharbor/harbor-portal:v2.5.1

container_name: harbor-portal

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

- NET_BIND_SERVICE

volumes:

- type: bind

source: ./common/config/portal/nginx.conf

target: /etc/nginx/nginx.conf

networks:

- harbor

depends_on:

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "portal"

jobservice:

image: goharbor/harbor-jobservice:v2.5.1

container_name: harbor-jobservice

env_file:

- ./common/config/jobservice/env

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/job_logs:/var/log/jobs:z

- type: bind

source: ./common/config/jobservice/config.yml

target: /etc/jobservice/config.yml

- type: bind

source: ./common/config/shared/trust-certificates

target: /harbor_cust_cert

networks:

- harbor

depends_on:

- core

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "jobservice"

redis:

image: goharbor/redis-photon:v2.5.1

container_name: redis

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

volumes:

- /ali-nas/harbor/redis:/var/lib/redis

networks:

harbor:

depends_on:

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "redis"

proxy:

image: goharbor/nginx-photon:v2.5.1

container_name: nginx

restart: always

cap_drop:

- ALL

cap_add:

- CHOWN

- SETGID

- SETUID

- NET_BIND_SERVICE

volumes:

- ./common/config/nginx:/etc/nginx:z

- type: bind

source: ./common/config/shared/trust-certificates

target: /harbor_cust_cert

networks:

- harbor

ports:

- 81:8080

depends_on:

- registry

- core

- portal

- log

logging:

driver: "syslog"

options:

syslog-address: "tcp://localhost:1514"

tag: "proxy"

networks:

harbor:

external: false

harbor.yml文件需要更改hostname, 监听port, 如果不开启ssl, 则注释掉https, 更改harbor_admin_password，和data_volume

hostname: 172.20.47.2

http:
  port: 81

harbor_admin_password: xxxxx

database:
  password: root123
  max_idle_conns: 100
  max_open_conns: 900

data_volume: /ali-nas/harbor



trivy:
  ignore_unfixed: false
  skip_update: false
  insecure: false

jobservice:
  max_job_workers: 10

notification:
  webhook_job_max_retry: 10

chart:
  absolute_url: disabled

log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /ali-nas/harbor/log


_version: 2.5.0


proxy:
  http_proxy:
  https_proxy:
  no_proxy:
  components:
    - core
    - jobservice
    - trivy

hostname: 172.20.47.2

http:

port: 81

harbor_admin_password: xxxxx

database:

password: root123

max_idle_conns: 100

max_open_conns: 900

data_volume: /ali-nas/harbor

trivy:

ignore_unfixed: false

skip_update: false

insecure: false

jobservice:

max_job_workers: 10

notification:

webhook_job_max_retry: 10

chart:

absolute_url: disabled

log:

level: info

local:

rotate_count: 50

rotate_size: 200M

location: /ali-nas/harbor/log

_version: 2.5.0

proxy:

http_proxy:

https_proxy:

no_proxy:

components:

- core

- jobservice

- trivy

执行完docker-compose up -d后，容器全部启来了，但是再执行docker-compose down后，再执行install.sh，或up启动，发现有几个问题： docker-compose up启动报错： Creating harbor-log … error ERROR: for harbor-log Cannot start service log: driver failed programming external connectivity on endpoint harbor-log (cb1338bb88ec65aba9727cb1e7656e6c44b3705095121e27ab9240a66cda1ef1): (iptables failed: iptables –wait -t filter Read more…

By sean, 2 years06/30/2022 ago

未分类

vscode配置代理

配置代理，让vscode和git飞起来对于 git, 只需要用下面的明亮就可以了

git config --global https.proxy http://127.0.0.1:1080
git config --global https.proxy https://127.0.0.1:1080
git config --global http.proxy 'socks5://127.0.0.1:1080'
git config --global https.proxy 'socks5://127.0.0.1:1080'

git config --global https.proxy http://127.0.0.1:1080

git config --global https.proxy https://127.0.0.1:1080

git config --global http.proxy 'socks5://127.0.0.1:1080'

git config --global https.proxy 'socks5://127.0.0.1:1080'

如果想要取消：

git config --global --unset http.proxy
git config --global --unset https.proxy

1 2	git config --global --unset http.proxy git config --global --unset https.proxy

但是如果是vscode，同样也是通过首选项->设置->搜索proxy，然后可以输入上面的proxy。速度就飞了。。或者可以这样：

export https_proxy=http://127.0.0.1:4780 http_proxy=http://127.0.0.1:4780 all_proxy=socks5://127.0.0.1:4780

1	export https_proxy=http://127.0.0.1:4780 http_proxy=http://127.0.0.1:4780 all_proxy=socks5://127.0.0.1:4780

当然，自备梯子

By sean, 2 years06/21/2022 ago

未分类

「Docker Buildx」- 构建“跨平台”镜像

使用Buildx 在x86架构centos上构建支持多架构镜像，首先，要确保内核大于4.0 升级内核版本

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
# 查询目前可升级的内核版本
yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
# 安装kernel-ml 最新版
yum --enablerepo=elrepo-kernel install kernel-ml -y
# 查询所有启动候选项
awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
# 设置grub引导默认启动项
grub2-set-default 0
# 重启
reboot
# 检测内核版本
uname -a

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm

# 查询目前可升级的内核版本

yum --disablerepo="*" --enablerepo="elrepo-kernel" list available

# 安装kernel-ml 最新版

yum --enablerepo=elrepo-kernel install kernel-ml -y

# 查询所有启动候选项

awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg

# 设置grub引导默认启动项

grub2-set-default 0

# 重启

reboot

# 检测内核版本

uname -a

卸载旧内核

[root@localhost ~]# rpm -qa | grep kernel
kernel-3.10.0-327.22.2.el7.x86_64
kernel-devel-3.10.0-327.22.2.el7.x86_64
kernel-tools-libs-3.10.0-327.28.2.el7.x86_64
kernel-headers-3.10.0-327.28.2.el7.x86_64
kernel-3.10.0-327.28.2.el7.x86_64
kernel-devel-3.10.0-327.13.1.el7.x86_64
php-symfony-http-kernel-2.8.7-1.el7.noarch
kernel-tools-3.10.0-327.28.2.el7.x86_64
kernel-devel-3.10.0-327.28.2.el7.x86_64
kernel-devel-3.10.0-327.18.2.el7.x86_64

[root@localhost ~]# yum remove kernel-3.10.0*
[root@localhost ~]# reboot

[root@localhost ~]# rpm -qa | grep kernel

kernel-3.10.0-327.22.2.el7.x86_64

kernel-devel-3.10.0-327.22.2.el7.x86_64

kernel-tools-libs-3.10.0-327.28.2.el7.x86_64

kernel-headers-3.10.0-327.28.2.el7.x86_64

kernel-3.10.0-327.28.2.el7.x86_64

kernel-devel-3.10.0-327.13.1.el7.x86_64

php-symfony-http-kernel-2.8.7-1.el7.noarch

kernel-tools-3.10.0-327.28.2.el7.x86_64

kernel-devel-3.10.0-327.28.2.el7.x86_64

kernel-devel-3.10.0-327.18.2.el7.x86_64

[root@localhost ~]# yum remove kernel-3.10.0*

[root@localhost ~]# reboot

安装docker:

# 最新版
curl -sSl https://qiao.dev/bash/install_docker_latest_centos.sh | sh
# k8s最新支持版本 18.09.8
curl -sSl https://qiao.dev/bash/install_docker_18.09.8_centos.sh | sh

# 最新版

curl -sSl https://qiao.dev/bash/install_docker_latest_centos.sh | sh

# k8s最新支持版本 18.09.8

curl -sSl https://qiao.dev/bash/install_docker_18.09.8_centos.sh | sh

开启特性： /etc/systemd/system/docker.service.d/docker-options.conf 中加入： –experimental=true \ 可以添加私有仓库的ssl证书到buildx容器内

BUILDER=$(docker ps | grep buildkitd | cut -f1 -d' ')
docker cp /etc/docker/certs.d/harbor.offline/harbor.offline.crt $BUILDER:/usr/local/share/ca-certificates/
docker exec $BUILDER sh -c "cat /usr/local/share/ca-certificates/harbor.offline.crt >> /etc/ssl/certs/ca-certificates.crt"
docker restart $BUILDER

BUILDER=$(docker ps | grep buildkitd | cut -f1 -d' ')

docker cp /etc/docker/certs.d/harbor.offline/harbor.offline.crt $BUILDER:/usr/local/share/ca-certificates/

docker exec $BUILDER sh -c "cat /usr/local/share/ca-certificates/harbor.offline.crt >> /etc/ssl/certs/ca-certificates.crt"

docker restart $BUILDER

启动交叉编译: 方式1：

docker run -it --rm --privileged tonistiigi/binfmt --install all

docker buildx create --name multi-platform --use --platform linux/amd64,linux/arm64 --driver docker-container

docker run -it --rm --privileged tonistiigi/binfmt --install all

docker buildx create --name multi-platform --use --platform linux/amd64,linux/arm64 --driver docker-container

方式2 ：也可以安装qemu:

# 安装epel yum库
wget -P /etc/yum.repos.d/ http://mirrors.aliyun.com/repo/epel-7.repo
yum clean all
yum install qemu -y

# 安装epel yum库

wget -P /etc/yum.repos.d/ http://mirrors.aliyun.com/repo/epel-7.repo

yum clean all

yum install qemu -y

如构建ntp构建： docker buildx build –platform linux/amd64,linux/arm64 -t chrony . 如果想将构建好的镜像保存在本地，可以将 type 指定为 docker，但必须分别为不同的 CPU 架构构建不同的镜像，不能合并成一个镜像，即： docker buildx build –platform linux/amd64,linux/amd64-t chrony -o type=docker . docker Read more…

By sean, 2 years02/14/2022 ago

未分类

通过ansible获取文件的几种方式

通过ansible获取文件的几种方式: 通过lookup读取ansible管理节点上的的内容，并将其内容赋值给变量。如: 获取管理节点上的ssh key:

- name: Autoops | set ssh key
  set_fact:
    autoops_public_key: "{{ autoops_public_key | default(lookup('file', '/root/.ssh/id_rsa.pub')) }}"
    autoops_private_key: "{{ autoops_private_key | default(lookup('file', '/root/.ssh/id_rsa')) }}"
  delegate_to: localhost
  connection: local
  run_once: yes

- name: Autoops | set ssh key

set_fact:

autoops_public_key: "{{ autoops_public_key | default(lookup('file', '/root/.ssh/id_rsa.pub')) }}"

autoops_private_key: "{{ autoops_private_key | default(lookup('file', '/root/.ssh/id_rsa')) }}"

delegate_to: localhost

connection: local

run_once: yes

fetch 模块的作用就是”Fetches a file from remote nodes，fetch为”拿来”之意，当我们需要将受管主机中的文件拉取到ansible主机时，则可以使用此模块, 可以从远端将文件拉取到本地, 如获取k8s master节点上的ca证书到本地:

- hosts: kube_control_plane:localhost
  gather_facts: False
  pre_tasks:
  - setup:
      gather_subset: 'min'
    tags: 
      - kubeconfig_localhost
      - user_init
  tasks:
  - import_role: 
      name: kubespray-defaults
  - name: "set_fact for current_user"
    set_fact:
      current_user: "{{ hostvars[inventory_hostname]['ansible_env'].USER }}"
      current_user_home: "{{ hostvars[inventory_hostname]['ansible_env'].HOME }}"
    tags: 
      - kubeconfig_localhost
      - user_init

  - block:
    # edit admin.conf
    - name: modify admin.conf to api server domain
      lineinfile:
        dest: "{{ current_user_home }}/.kube/config"
        state: present
        regexp: '(^\s*.server: .*://).*(:.*$)'
        line: '\1{{ apiserver_loadbalancer_domain_name }}\2'
        backrefs: yes
      when: loadbalancer_apiserver is not defined
    delegate_to: localhost
    connection: local
    delegate_facts: yes
    become: no
    run_once: yes
    when:
      - kubeconfig_localhost
    tags: kubeconfig_localhost

  - name: Fetch ca.crt from master to ansible host
    fetch:
      src: "{{ kube_cert_compat_dir }}/ca.crt"
      dest: "{{ kube_cert_compat_dir }}/ca.crt"
      flat: yes
      validate_checksum: no
    register: copy_ca_result
    until: copy_ca_result is not failed
    retries: 20
    become: no
    run_once: yes
    when:
      - kubeconfig_localhost
      - "'HOSTNAME' in hostvars['localhost']['ansible_env']"
      - "hostvars['localhost']['ansible_env'].get('HOSTNAME') not in groups['kube_control_plane']"
    tags: kubeconfig_localhost

- hosts: kube_control_plane:localhost

gather_facts: False

pre_tasks:

- setup:

gather_subset: 'min'

tags:

- kubeconfig_localhost

- user_init

tasks:

- import_role:

name: kubespray-defaults

- name: "set_fact for current_user"

set_fact:

current_user: "{{ hostvars[inventory_hostname]['ansible_env'].USER }}"

current_user_home: "{{ hostvars[inventory_hostname]['ansible_env'].HOME }}"

tags:

- kubeconfig_localhost

- user_init

- block:

# edit admin.conf

- name: modify admin.conf to api server domain

lineinfile:

dest: "{{ current_user_home }}/.kube/config"

state: present

regexp: '(^\s*.server: .*://).*(:.*$)'

line: '\1{{ apiserver_loadbalancer_domain_name }}\2'

backrefs: yes

when: loadbalancer_apiserver is not defined

delegate_to: localhost

connection: local

delegate_facts: yes

become: no

run_once: yes

when:

- kubeconfig_localhost

tags: kubeconfig_localhost

- name: Fetch ca.crt from master to ansible host

fetch:

src: "{{ kube_cert_compat_dir }}/ca.crt"

dest: "{{ kube_cert_compat_dir }}/ca.crt"

flat: yes

validate_checksum: no

until: copy_ca_result is not failed

retries: 20

become: no

run_once: yes

when:

- kubeconfig_localhost

- "'HOSTNAME' in hostvars['localhost']['ansible_env']"

- "hostvars['localhost']['ansible_env'].get('HOSTNAME') not in groups['kube_control_plane']"

tags: kubeconfig_localhost

使用cat测试到变量 slurp 模块用于拉取远端文件的 base64 码, 可通过b64decode进行解码任务: 获取etcd主机的ssl文件信息注册到变量, 用来生成etcdkeeper配置文件, 首先用cat来获取: tasks/main.yml:

- block:
  - name: get remote file contents
    command: "cat {{ etcdkeeper_cacert_file }}"
    register: etcdkeeper_cacert_file

  - name: get remote file contents
    command: "cat {{ etcdkeeper_cert_file }}"
    register: etcdkeeper_cert_file

  - name: get remote file contents
    command: "cat {{ etcdkeeper_key_file }}"
    register: etcdkeeper_key_file
  when:
    - inventory_hostname == groups['etcd'][0]

- name: show key contents
  debug:
    var: hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.stdout

- name: Autoops | Create addon dir
  file:
    path: "{{ item }}"
    state: directory
    owner: root
    group: root
    mode: 0755
    recurse: yes
  with_items:
    - "{{ kube_config_dir }}/addons/etcdkeeper"
  when:
    - inventory_hostname == groups['kube_control_plane'][0]

- name: Autoops | Create manifests
  template: 
    src: "{{ item }}.yml.j2" 
    dest: "{{ kube_config_dir }}/addons/etcdkeeper/{{ item }}.yml"
  vars:
    etcdkeeper_cacert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.stdout }}"
    etcdkeeper_cert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cert_file.stdout }}"
    etcdkeeper_key: "{{ hostvars[groups['etcd'][0]].etcdkeeper_key_file.stdout }}"
  with_items:
    - etcdkeeper_ns
    - etcdkeeper_cm
    - etcdkeeper_deploy
    - etcdkeeper_svc
  register: etcdkeeper_manifests
  when:
    - inventory_hostname == groups['kube_control_plane'][0]

- name: Autoops | Apply manifests
  shell: "{{ bin_dir }}/kubectl apply -f {{ kube_config_dir }}/addons/etcdkeeper/{{ item.item }}.yml"
  with_items: "{{ etcdkeeper_manifests.results }}"
  when:
    - inventory_hostname == groups['kube_control_plane'][0]

- block:

- name: get remote file contents

command: "cat {{ etcdkeeper_cacert_file }}"

- name: get remote file contents

command: "cat {{ etcdkeeper_cert_file }}"

- name: get remote file contents

command: "cat {{ etcdkeeper_key_file }}"

when:

- inventory_hostname == groups['etcd'][0]

- name: show key contents

debug:

var: hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.stdout

- name: Autoops | Create addon dir

file:

path: "{{ item }}"

state: directory

owner: root

group: root

mode: 0755

recurse: yes

with_items:

- "{{ kube_config_dir }}/addons/etcdkeeper"

when:

- inventory_hostname == groups['kube_control_plane'][0]

- name: Autoops | Create manifests

template:

src: "{{ item }}.yml.j2"

dest: "{{ kube_config_dir }}/addons/etcdkeeper/{{ item }}.yml"

vars:

etcdkeeper_cacert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.stdout }}"

etcdkeeper_cert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cert_file.stdout }}"

etcdkeeper_key: "{{ hostvars[groups['etcd'][0]].etcdkeeper_key_file.stdout }}"

with_items:

- etcdkeeper_ns

- etcdkeeper_cm

- etcdkeeper_deploy

- etcdkeeper_svc

when:

- inventory_hostname == groups['kube_control_plane'][0]

- name: Autoops | Apply manifests

shell: "{{ bin_dir }}/kubectl apply -f {{ kube_config_dir }}/addons/etcdkeeper/{{ item.item }}.yml"

with_items: "{{ etcdkeeper_manifests.results }}"

when:

- inventory_hostname == groups['kube_control_plane'][0]

结果发现, 生成的配置里, 换行符都被换成了空格尝试使用slurp模块:

- block:
  - name: Etcdkeeper | get remote ca file contents
    slurp:
      src: "{{ etcdkeeper_cacert_file }}"
    register: etcdkeeper_cacert_file

  - name: Etcdkeeper | get remote cert file contents
    slurp:
      src: "{{ etcdkeeper_cert_file }}"
    register: etcdkeeper_cert_file

  - name: Etcdkeeper | get remote key file contents
    slurp:
      src: "{{ etcdkeeper_key_file }}"
    register: etcdkeeper_key_file
  when:
    - inventory_hostname == groups['etcd'][0]


- name: Etcdkeeper | Create addon dir
  file:
    path: "{{ item }}"
    state: directory
    owner: root
    group: root
    mode: 0755
    recurse: yes
  with_items:
    - "{{ kube_config_dir }}/addons/etcdkeeper"
  when:
    - inventory_hostname == groups['kube_control_plane'][0]

- name: Etcdkeeper | Create manifests
  template: 
    src: "{{ item }}.yml.j2" 
    dest: "{{ kube_config_dir }}/addons/etcdkeeper/{{ item }}.yml"
  vars:
    etcdkeeper_cacert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.content | b64decode  | regex_replace('\n', '\n    ') }}"
    etcdkeeper_cert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cert_file.content | b64decode  | regex_replace('\n', '\n    ') }}"
    etcdkeeper_key: "{{ hostvars[groups['etcd'][0]].etcdkeeper_key_file.content | b64decode  | regex_replace('\n', '\n    ') }}"
  with_items:
    - etcdkeeper_ns
    - etcdkeeper_cm
    - etcdkeeper_deploy
    - etcdkeeper_svc
  register: etcdkeeper_manifests
  when:
    - inventory_hostname == groups['kube_control_plane'][0]

- block:

- name: Etcdkeeper | get remote ca file contents

slurp:

src: "{{ etcdkeeper_cacert_file }}"

- name: Etcdkeeper | get remote cert file contents

slurp:

src: "{{ etcdkeeper_cert_file }}"

- name: Etcdkeeper | get remote key file contents

slurp:

src: "{{ etcdkeeper_key_file }}"

when:

- inventory_hostname == groups['etcd'][0]

- name: Etcdkeeper | Create addon dir

file:

path: "{{ item }}"

state: directory

owner: root

group: root

mode: 0755

recurse: yes

with_items:

- "{{ kube_config_dir }}/addons/etcdkeeper"

when:

- inventory_hostname == groups['kube_control_plane'][0]

- name: Etcdkeeper | Create manifests

template:

src: "{{ item }}.yml.j2"

dest: "{{ kube_config_dir }}/addons/etcdkeeper/{{ item }}.yml"

vars:

etcdkeeper_cacert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cacert_file.content | b64decode | regex_replace('\n', '\n ') }}"

etcdkeeper_cert: "{{ hostvars[groups['etcd'][0]].etcdkeeper_cert_file.content | b64decode | regex_replace('\n', '\n ') }}"

etcdkeeper_key: "{{ hostvars[groups['etcd'][0]].etcdkeeper_key_file.content | b64decode | regex_replace('\n', '\n ') }}"

with_items:

- etcdkeeper_ns

- etcdkeeper_cm

- etcdkeeper_deploy

- etcdkeeper_svc

when:

- inventory_hostname == groups['kube_control_plane'][0]

结果换行符都保留了下来, 格式正确.

By sean, 3 years11/07/2021 ago

未分类

用sed命令进行整段替换

sed 整段插入：下面是一个xml文件 1.txt：

<?xml version="1.0" encoding="UTF-8" ?>
<ypr>
    <code>iuap-installer</code>
    <name>安装器</name>
    <type>Tools</type>
    <version>v1.0</version>
    <serialNum>20210927105443</serialNum>
    <description></description>
    <gitBranch>develop-20210924</gitBranch>
    <commitId>b26eb31</commitId>
    <osTypes>
        <osType>
            <os>centos</os>
            <arch>x86_64</arch>
        </osType>
    </osTypes>
    <actions>
        <action>
            <code>install</code>
            <tasks>
                <task>
                    <type>command</type>
                    <runner>local</runner>
                    <template></template>
                    <command>startup.sh</command>
                </task>
            </tasks>
        </action>
    </actions>
</ypr>

<?xml version="1.0" encoding="UTF-8" ?>

<ypr>

<code>iuap-installer</code>

<type>Tools</type>

<gitBranch>develop-20210924</gitBranch>

<os>centos</os>

</osType>

</osTypes>

<code>install</code>

<tasks>

<task>

<type>command</type>

<runner>local</runner>

<command>startup.sh</command>

</task>

</tasks>

</action>

</actions>

</ypr>

需求：要在匹配</actions>字段后插入如下整段内容：

dependency=`cat <<EOF
    <dependencies>
        <dependency>
          <name>Maven</name>
          <code>apache-maven</code>
          <type>Tools</type>
          <version>3.8.2</version>
          <serialNum>20210927</serialNum>
        </dependency>
    </dependencies>
EOF
`

dependency=`cat <<EOF

<name>Maven</name>

<code>apache-maven</code>

<type>Tools</type>

</dependency>

</dependencies>

EOF

方法一：

sed -e $'/<\/actions>/{;z;r/dev/stdin\n}' 1.txt <<<"$dependency"

1	sed -e $'/<\/actions>/{;z;r/dev/stdin\n}' 1.txt <<<"$dependency"

方法二：

sed -e "s@<\/actions>@&\n`echo "$dependency"|awk '{printf("%s\\\\n", $0);}'|sed -e 's/\\\n$//'`@g" 1.txt

1	sed -e "s@<\/actions>@&\n`echo "$dependency"\|awk '{printf("%s\\\\n", $0);}'\|sed -e 's/\\\n$//'`@g" 1.txt

参考： https://stackoverflow.com/questions/6684487/sed-replace-with-variable-with-multiple-lines https://www.coder.work/article/2570180

By sean, 3 years09/27/2021 ago