Gitlab 注册表的 Varnish 设置

Question

嗨，我正在尝试使用清漆缓存容器注册表

我现在的设置是这样的

Gitlab Rails --> varnish_node:6081 --> nginx:80 --> OpenStack s3:6780

每当我将 Rails 机器设置为直接访问 nginx 时，我都可以成功获取 s3 中的对象，但是当我在 Rails 中设置 varnish 节点时，我收到 403 错误

这是我的虚拟配置

default.vcl
vcl 4.0;

backend default {
    .host = "127.0.0.1";
    .port = "80";
}

sub vcl_recv {
    # Set the client IP address in a custom header
    set req.http.X-Client-IP = client.ip;
    set req.backend_hint = default;

}

sub vcl_backend_response {
    # Remove or modify any headers that may cause authorization issues
    unset beresp.http.X-Some-Header;
}

nginx.conf

upstream hpc {
     server hpc.example.com:6780 max_fails=3 fail_timeout=30s;

}
server {
    listen 80 default_server;
        server_name hpc.example.com;

    location / {
                proxy_pass https://hpc;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_http_version 1.1;
                proxy_set_header X-Forwarded-Proto $scheme;   
    }

}

nginx.logs

127.0.0.1 - - [08/May/2024:21:59:12 +0300] "GET /stage-primary-glb-registry?list-type=2&max-keys=1&prefix= HTTP/1.1" 403 193 "-" "docker-distribution/v3.39.2-gitlab (go1.17.6) aws-sdk-go/1.40.7 (go1.17.6; linux; amd64)" "10.20.20.21:6081" "AWS4-HMAC-SHA256 Credential=6f1bfb7443cc45b5a16e5a1cbc1e7976/20240508/mars/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=4a1b957d696d27a9a9d1ca5942e6c241486567a983a2815c78334ac9625002e7" "10.20.20.20"

 *   << Request  >> 221
 *   Begin          req 220 rxreq
 *   Timestamp      Start: 1715194922.190385 0.000000 0.000000
 *   Timestamp      Req: 1715194922.190385 0.000000 0.000000
 *   VCL_use        boot
 *   ReqStart       10.20.20.20 35344 a0
 *   ReqMethod      GET
 *   ReqURL         /stage-primary-glb-registry?list-type=2&max-keys=1&prefix=
 *   ReqProtocol    HTTP/1.1
 *   ReqHeader      Host: 10.20.20.21:6081
 *   ReqHeader      User-Agent: docker-distribution/v3.39.2-gitlab (go1.17.6) aws-sdk-go/1.40.7 (go1.17.6; linux; amd64)
 *   ReqHeader      Authorization: AWS4-HMAC-SHA256 Credential=6f1bfb7443cc45b5a16e5a1cbc1e7976/20240508/mars/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=2bbffe23574c9f771fdf87b7ba9823cd5720d301142a9fa015fd09c68d048313
 *   ReqHeader      X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 *   ReqHeader      X-Amz-Date: 20240508T190202Z
 *   ReqHeader      Accept-Encoding: gzip
 *   ReqHeader      X-Forwarded-For: 10.20.20.20
 *   ReqHeader      Via: 1.1 glb-s-adp-rails-primary-4 (Varnish/7.5)
 *   VCL_call       RECV
 *   ReqHeader      X-Client-IP: 10.20.20.20
 *   VCL_return     pass
 *   VCL_call       HASH
 *   VCL_return     lookup
 *   VCL_call       PASS
 *   VCL_return     fetch
 *   Link           bereq 222 pass
 *   Timestamp      Fetch: 1715194922.257731 0.067345 0.067345
 *   RespProtocol   HTTP/1.1
 *   RespStatus     403
 *   RespReason     Forbidden
 *   RespHeader     Server: nginx/1.18.0 (Ubuntu)
 *   RespHeader     Date: Wed, 08 May 2024 19:02:02 GMT
 *   RespHeader     Content-Type: application/xml
 *   RespHeader     Content-Length: 193
 *   RespHeader     x-amz-request-id: tx0000033f0ecab4cc0e4a6-00663bcc2a-17b6ba11-default
 *   RespHeader     accept-ranges: bytes
 *   RespHeader     X-Varnish: 221
 *   RespHeader     Age: 0
 *   RespHeader     Via: 1.1 glb-s-adp-rails-primary-4 (Varnish/7.5)
 *   VCL_call       DELIVER
 *   VCL_return     deliver
 *   Timestamp      Process: 1715194922.257737 0.067352 0.000006
 *   Filters
 *   RespHeader     Connection: keep-alive
 *   Timestamp      Resp: 1715194922.257767 0.067381 0.000029
 *   ReqAcct        579 0 579 337 193 530
 *   End
 * List item

Answer 1

如果 Nginx 可以工作，而 Varnish 不基于您共享的配置，您应该通过向 Varnish 添加相同的标头来模仿 Nginx 的行为。

我看到缺少一个

X-Real-IP

标头和

X-Forwarded-Proto

标头。如果这就是你得到 403 的原因，你应该将它们添加到 Varnish 中。

但是，403错误本身有点抽象。我建议您查看 GitLab 设置的日志并检查为什么开始生成 403。

• 授权标头是否未正确传递？
• 是否是IP地址不匹配？
• 与缺失的

  X-Forwarded-Proto

  有关系吗？
• 或者日志中是否有另一个错误可以解释发生了什么？

首先尝试发送与 Nginx 相同的标头，然后查看 GitLab 日志以查看是否有更多可用的上下文信息。