如何在Kubernetes集群中对一个NodePort服务的NodePort进行tcpdump？

问题描述 投票：0回答：1

我想对Kubernetes集群中指向一个pod的NodePort服务的节点端口（如30034）进行tcpdump，这个节点端口服务被映射到路径下的一个ingress资源中。当我使用配置在 ingress 里面的主机打入 ingress 时，我得到了目标 pod 的响应，但 tcpdump 没有追踪任何东西。(Ingress-->NodePortService->NodePort--[tcpdump]->pod)

我试过：sudo tcpdump -i any port 30034 -w tcp-dump.pcap，但没有捕捉到任何东西。

请你在这里提出建议。当流量通过入口控制器来时，tcpdump没有捕获任何东西，这是什么原因？https:/node-ip:30034:service。; 我收到tcpdump了。

谢谢。

docker kubernetes kubectl
1个回答
1
投票

大多数的 ingress 控制器实际上绕过了 kube-proxy 层，直接使用端点 IP，所以这似乎很可能与此有关。一般来说，由于大量的虚拟网络和时髦的iptables规则，在容器系统中获取数据包痕迹可能很困难。

1
投票

在Kubernetes中有效地使用TCPdump是比较棘手的，需要你给你的pod创建一个旁车。你所面临的其实是预期的行为。

运行像TCPdump或ngrep这样的好的老东西不会产生太多有趣的信息，因为你在默认情况下直接链接到桥接网络或覆盖。

好消息是，你可以将你的TCPdump容器链接到主机网络，甚至更好的是，链接到容器网络栈。 来源 如何在Docker中有效地进行TCPdump？

事情是这样的，你有两个入口点，一个是nodeIP:NodePort，第二个是ClusterIP:Port。这两个都是指向kubernetes iptables上设置的同一套端点随机化规则。

只要能发生在任何节点上，就很难配置tcpdump只在一个点上捕捉所有有趣的流量。

我所知道的最好的分析工具是Istio，但它主要针对HTTP流量工作。

考虑到这一点，最好的解决方案是为服务后面的每个 pod 使用 tcpdumper sidecar。

让我们通过一个例子来了解如何实现这个目标吧 

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: web-app
        image: nginx
        imagePullPolicy: Always        
        ports:
        - containerPort: 80
          protocol: TCP
      - name: tcpdumper
        image: docker.io/dockersec/tcpdump
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: web-svc
  namespace: default
spec:
  ports:
  - nodePort: 30002
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: web
  type: NodePort

在这个manifest中，我们可以注意到树状的重要东西。我们有一个nginx容器和一个tcpdumper容器作为sidecar，我们有一个服务定义为NodePort。

要访问我们的sidecar，你必须运行以下命令。

$ kubectl attach -it web-app-db7f7c59-d4xm6 -c tcpdumper

Example: 

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        13d
web-svc      NodePort    10.108.142.180   <none>        80:30002/TCP   9d

$ curl localhost:30002
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

$ kubectl attach -it web-app-db7f7c59-d4xm6 -c tcpdumper
Unable to use a TTY - container tcpdumper did not allocate one
If you don't see a command prompt, try pressing enter.
> web-app-db7f7c59-d4xm6.80: Flags [P.], seq 1:78, ack 1, win 222, options [nop,nop,TS val 300957902 ecr 300958061], length 77: HTTP: GET / HTTP/1.1
12:03:16.884512 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [.], ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 0
12:03:16.884651 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [P.], seq 1:240, ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 239: HTTP: HTTP/1.1 200 OK
12:03:16.884705 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [P.], seq 240:852, ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 612: HTTP
12:03:16.884743 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 240, win 231, options [nop,nop,TS val 300957902 ecr 300958061], length 0
12:03:16.884785 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 852, win 240, options [nop,nop,TS val 300957902 ecr 300958061], length 0
12:03:16.889312 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [F.], seq 78, ack 852, win 240, options [nop,nop,TS val 300957903 ecr 300958061], length 0
12:03:16.889351 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [F.], seq 852, ack 79, win 217, options [nop,nop,TS val 300958062 ecr 300957903], length 0
12:03:16.889535 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 853, win 240, options [nop,nop,TS val 300957903 ecr 300958062], length 0
12:08:10.336319 IP6 fe80::ecee:eeff:feee:eeee > ff02::2: ICMP6, router solicitation, length 16
12:15:47.717966 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [S], seq 3314747302, win 28400, options [mss 1420,sackOK,TS val 301145611 ecr 0,nop,wscale 7], length 0
12:15:47.717993 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [S.], seq 2539474977, ack 3314747303, win 27760, options [mss 1400,sackOK,TS val 301145769 ecr 301145611,nop,wscale 7], length 0
12:15:47.718162 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 1, win 222, options [nop,nop,TS val 301145611 ecr 301145769], length 0
12:15:47.718164 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [P.], seq 1:78, ack 1, win 222, options [nop,nop,TS val 301145611 ecr 301145769], length 77: HTTP: GET / HTTP/1.1
12:15:47.718191 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [.], ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 0
12:15:47.718339 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [P.], seq 1:240, ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 239: HTTP: HTTP/1.1 200 OK
12:15:47.718403 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [P.], seq 240:852, ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 612: HTTP
12:15:47.718451 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 240, win 231, options [nop,nop,TS val 301145611 ecr 301145769], length 0
12:15:47.718489 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 852, win 240, options [nop,nop,TS val 301145611 ecr 301145769], length 0
12:15:47.723049 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [F.], seq 78, ack 852, win 240, options [nop,nop,TS val 301145612 ecr 301145769], length 0
12:15:47.723093 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [F.], seq 852, ack 79, win 217, options [nop,nop,TS val 301145770 ecr 301145612], length 0
12:15:47.723243 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 853, win 240, options [nop,nop,TS val 301145612 ecr 301145770], length 0
12:15:50.493995 IP 192.168.250.64.31340 > web-app-db7f7c59-d4xm6.80: Flags [S], seq 124258064, win 28400, options [mss 1420,sackOK,TS val 301146305 ecr 0,nop,wscale 7], length 0
12:15:50.494022 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.31340: Flags [S.], seq 3544403648, ack 124258065, win 27760, options [mss 1400,sackOK,TS val 301146463 ecr 301146305,nop,wscale 7], length 0
12:15:50.494189 IP 192.168.250.64.31340 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 1, win 222, options

你也可以看一下 嗅觉 工具，是一个kubectl插件，利用tcpdump和Wireshark在Kubernetes集群中的任何pod上启动远程捕获。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.