Logstash beats 输入“无效版本的 beats 协议”

问题描述 投票：0回答：3

我正在编写一个 kibana 插件和一个 logstash 管道。对于我的测试，我只是写了一个这样的 logstash 输入：

input {

   beats {
        port => 9600
        ssl => false
        ssl_verify_mode => "none"
    }

}

但是当我尝试打开与节点的连接时（上面的代码）：

invoke = (parameters, id, port, host) => {
        var fs = require('fs');

        console.log(`Sending message in beats, host= ${host}, port= ${port}, message= ${parameters.message}`);

        var connectionOptions = {
            host: host,
            port: port
        };

        var client = lumberjack.client(connectionOptions, {rejectUnauthorized: false, maxQueueSize: 500});

        client.writeDataFrame({"line": id + " " + parameters.message});
    }

logstash 给我“beats 协议的无效版本：22”和“beats 协议的无效版本：3”：

Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.11.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.11.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        ... 9 more
[2020-08-11T07:49:47,954][INFO ][org.logstash.beats.BeatsHandler] [local: 172.22.0.40:9600, remote: 172.22.0.1:33766] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
[2020-08-11T07:49:47,955][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:61) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:253) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.11.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.11.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        ... 11 more
node.js logstash kibana elk elastic-beats
3个回答
0
投票
  1. 您可以尝试使用 tcp 输入，而不是使用节拍输入。

例子：

input {
  tcp {
    port => "9600"
    codec => "json"
  }
}
  1. 如果你使用的是beats input，想使用Logstash对Filebeat收集的数据进行额外的处理，需要配置Filebeat使用Logstash。

为此，您编辑 Filebeat 配置文件以通过将其注释掉来禁用 Elasticsearch 输出，并通过取消注释 Logstash 部分来启用 Logstash 输出：

output.logstash:
     hosts: ["127.0.0.1:5044"]

您可以在 https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html 上阅读更多内容

0
投票

在我的案例中可能有很多情况，问题与我的 filebeat.yml 有关，我在我的 logstash 服务器上遇到以下错误

    nd it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 69
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:61) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:253) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.65.Final.jar:4.1.65.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 69
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.2.6.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.2.6.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
        ... 11 more

当我检查我的 filebeat 时，它说连接被拒绝

root@ip-10-0-8-193:~# filebeat test output
elasticsearch: http://10.0.13.37:5044...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.13.37
    dial up... OK
  TLS... WARN secure connection disabled
  talk to server... ERROR Get "http://10.0.13.37:5044": read tcp 10.0.8.193:34940->10.0.13.37:5044: read: connection reset by peer

当我仔细检查我的日志时，我从错误中发现了一个错误配置

[2022-10-12T11:08:06,107][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://10.0.14.30:9200"]}

以上错误意味着我没有正确配置输出我关闭了 elasticsearch 行但错过了输出行

解决方案在我的案例中
我回到我的 filebeat.yml 并进行所需的更改

# ---------------------------- Elasticsearch Output ----------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  #  hosts: ["localhost:9200"]

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["10.0.13.37:5044"]

确保我们正确地评论了 elasticsearch 输出

0
投票

要了解有关此错误的更多信息，我们需要查看 filbeat.yml。您使用的输入插件是一个有效的输入插件，但在 filbert.yml 中您可能没有 output.logstash 值或者可能犯了一些其他错误。您能检查一下您是否已将输出发送到 Elasticsearch 或 Logstash 吗？

请确保您的 filebeat.yml 中有这行代码

output.logstash:
  hosts: ["127.0.0.1:5044"]
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.