如何重现VCL 3.0 vcl_recv的行为/在VCL 4.0中重新启动

Question

我正在将旧的Varnish 3服务器升级到Varnish 6.11

此Varnish服务器已设置为避免使用pipe缓存大文件（超过100mb）。首先将请求正常发送到后端，如果响应Content-Length标头的大小超过100mb，则会重试该请求：vcl_recv在内部被第二次调用，并在请求中添加了新的x-pipe标头从vlc_recv到pipe而不是hash。那时就是这样做的方式：

添加到vcl_recv：

  /* Bypass cache for large files.  The x-pipe header is
     set in vcl_fetch when a too large file is detected. */
  if (req.http.x-pipe && req.restarts > 0) {
    remove req.http.x-pipe;
    return (pipe);
  }

已添加到vcl_fetch：

  # don't cache files larger than 10MB
  /* Don't try to cache too large files.  It appears
     Varnish just crashes if we don't filter them. */
  if (beresp.http.Content-Length ~ "[0-9]{8,}" ) {
    set req.http.x-pipe = "1";
    return (restart);
  }

现在vcl_fetch更改为vcl_backend_response，restart操作已消失。用retry替换restart不会实现相同的行为，因为第二次没有调用vcl_recv函数。 retry仅重试后端请求。

在pass中返回vcl_backend_response的原因也与pipe存在相同：将请求发送给客户端之前，必须先由Varnish将请求读入内存，这是避免使用pipe的原因。

我的问题是：如何使用VCL 4.0 pipe（将字节直接从后端发送到客户端，而不进行任何处理）？

Answer 1

这是可行的，但是由于客户端和后端线程之间的分离，自Varnish Cache 4.0起，事情变得稍微复杂了。

想法是，您需要（1）从vcl_backend_response跳转到vcl_backend_error； （2）创建综合响应并将其返回给客户端线程（理想情况下将其缓存以避免请求序列化）； （3）在vcl_deliver期间检查先前的响应，并在客户端线程中执行重新启动。以下测试案例显示了一个工作示例：

varnishtest "..."

server s1 {
    rxreq
    txresp -hdr "X-Large-Response: 1"
} -repeat 3 -start

varnish v1 -vcl+backend {
    sub vcl_recv {
        if (req.restarts == 0) {
            unset req.http.X-Restart-And-Pipe;
        } elsif (req.http.X-Restart-And-Pipe) {
            return (pipe);
        }
    }

    sub vcl_deliver {
        if (resp.http.X-Restart-And-Pipe) {
            set req.http.X-Restart-And-Pipe = "1";
            return (restart);
        }
    }

    sub vcl_backend_fetch {
        if (bereq.retries == 0)  {
            unset bereq.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_response {
        if (beresp.http.X-Large-Response) {
            set bereq.http.X-Restart-And-Pipe = "1";
            return (error);
        } else {
            unset beresp.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_error {
        if (bereq.http.X-Restart-And-Pipe) {
            set beresp.http.X-Restart-And-Pipe = "1";
            set beresp.ttl = 1s;
            set beresp.grace = 0s;
            set beresp.keep = 0s;
            return (deliver);
        }
    }
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -run

varnish v1 -expect n_object == 1
varnish v1 -expect sess_conn == 2
varnish v1 -expect client_req == 2
varnish v1 -expect s_sess == 2
varnish v1 -expect s_pipe == 2

仅从Varnish Cache 6.3.0开始，使用vcl_backend_response的坏消息才能从vcl_backend_error跳到return (error)。仍然可以使用较旧的版本，但是解决方案有点笨拙：首先您需要跳转到vcl_backend_fetch，然后使用始终损坏的后端：

varnishtest "..."

server s1 {
    rxreq
    txresp -hdr "X-Large-Response: 1"
} -repeat 3 -start

varnish v1 -vcl+backend {
    backend always_broken_be {
        .host = "127.0.0.1";
        .port = "666";
    }

    sub vcl_recv {
        if (req.restarts == 0) {
            unset req.http.X-Restart-And-Pipe;
        } elsif (req.http.X-Restart-And-Pipe) {
            return (pipe);
        }
    }

    sub vcl_deliver {
        if (resp.http.X-Restart-And-Pipe) {
            set req.http.X-Restart-And-Pipe = "1";
            return (restart);
        }
    }

    sub vcl_backend_fetch {
        if (bereq.retries == 0)  {
            unset bereq.http.X-Restart-And-Pipe;
        } elsif (bereq.http.X-Restart-And-Pipe) {
            set bereq.backend = always_broken_be;
        }
    }

    sub vcl_backend_response {
        if (beresp.http.X-Large-Response) {
            set bereq.http.X-Restart-And-Pipe = "1";
            return (retry);
        } else {
            unset beresp.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_error {
        if (bereq.http.X-Restart-And-Pipe) {
            set beresp.http.X-Restart-And-Pipe = "1";
            set beresp.ttl = 1s;
            set beresp.grace = 0s;
            set beresp.keep = 0s;
            return (deliver);
        }
    }
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -run

varnish v1 -expect n_object == 1
varnish v1 -expect sess_conn == 2
varnish v1 -expect client_req == 2
varnish v1 -expect s_sess == 2
varnish v1 -expect s_pipe == 2