向插件添加ID时，Logstash 6.2.4崩溃（预期＃之一）

Question

我正在尝试将ID字段添加到我的Logstash 6.2.4配置中。我想做的是调试“ http://localhost:9600/_node/stats/pipelines”，因此我需要一些名称（id是配置中没有id字段的随机UUID）。我找到了documentation about plugin id。像这样对我有效：

input {
    http {
        port => "${HTTP_PORT_FOR_EVENTS:8089}"
        additional_codecs => {"application/json"=>"json"}
        id => "http_events"
        tags => [ "test" ]
    }
}

filter {
    if "test" in [tags]  {
        mutate {
            remove_field => [ "headers", "host" ]   
        }
    }
}

但是它像这样崩溃：

input {
    http {
        port => "${HTTP_PORT_FOR_EVENTS:808}"
        additional_codecs => {"application/json"=>"json"}
        id => "http_events"
        tags => [ "test" ]
    }
}

filter {
    id => "test2"
    if "test" in [tags]  {
        mutate {
            remove_field => [ "headers", "host" ]   
        }
    }
}

出现此错误（我猜是由于Docker容器重启或其他原因导致两次关机之间的错误：]

无法执行动作{：action => LogStash :: PipelineAction :: Create / pipeline_id：main，：exception =>“ LogStash :: ConfigurationError”，：message =>“预计＃，{在第32行，列过滤器{\ r \ n id“，：backtrace => [” / usr / share / logstash / logstash-core / lib / logstash / compiler.rb：42：incompile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'“中的8（字节676）， “ /usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12：在block in compile_sources'", "org/jruby/RubyArray.java:2486:in map'中”，“ /usr/share/logstash/logstash-core/lib/logstash/compiler.rb： 11：在compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in中初始化'“，” /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169：在initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in中执行'“，” / usr / share / logstash / logstash- core / lib / logstash / agent.rb：315：inblock in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'“，” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:inblock in converge_state'", "org/jruby/RubyArray.java:1734:ineach'“， “ / usr / share / logstash / logstash-core / lib / logstash / agent中的” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:inconverge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:inblock in converge_state_and_update'“ rb：141：inwith_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:inconverge_state_and_update'“，” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:105:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in间隔'“，” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:94：在execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in block中执行'，“，” / usr / share / logstash / vendor / bundle / jruby / 2.3.0 / gems / stud-0.0.23 / lib / stud / task.rb：24：in`init中的块'“]}

收到SIGTERM。闭嘴将Logstash的日志发送到/ usr / share / logstash / logs，该日志现在通过log4j2.properties配置初始化模块{：module_name =>“ fb_apache”，：directory =>“ / usr / share / logstash / modules / fb_apache / configuration”}初始化模块{：module_name =>“ netflow”，：directory =>“ / usr / share / logstash / modules / netflow / configuration”}忽略“ pipelines.yml”文件，因为指定了模块或命令行选项启动Logstash {“ logstash.version” =>“ 6.2.4”}成功启动Logstash API端点{：port => 9600}SIGTERM已收到。正在关闭。

将Logstash的日志发送到/ usr / share / logstash / logs，该日志现在通过log4j2.properties配置初始化模块{：module_name =>“ fb_apache”，：directory =>“ / usr / share / logstash / modules / fb_apache / configuration”}初始化模块{：module_name =>“ netflow”，：directory =>“ / usr / share / logstash / modules / netflow / configuration”}忽略“ pipelines.yml”文件，因为指定了模块或命令行选项启动Logstash {“ logstash.version” =>“ 6.2.4”}成功启动Logstash API端点{：port => 9600}

无法执行动作{：action => LogStash :: PipelineAction :: Create / pipeline_id：main，：exception =>“ LogStash :: ConfigurationError”，：message =>“预计为＃，{在第24行，列过滤器{\ n id“，：backtrace => [” / usr / share / logstash / logstash-core / lib / logstash / compiler.rb：42：incompile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'“，” / usr / share / logstash / logstash-core / lib / logstash / compiler.rb：12：在block in compile_sources'", "org/jruby/RubyArray.java:2486:in map'“，” / usr / share / logstash / logstash-core / lib / logstash / compiler.rb：11：在compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in中初始化'“，” /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169：在initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in中执行'“，” / usr / share / logstash / logstash-core / lib / logstash / agent.rb：315：inblock in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'“，” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:inblock in converge_state'", "org/jruby/RubyArray.java:1734:ineach'“，” / usr / share / logstash / logstash-core / lib / logstash / agent.rb：299：在converge_state_and_update中的converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in块中“”，“ /usr/share/logstash/logstash-core/lib/logstash/agent.rb： 141：with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in中的converge_state_and_update'“，” /usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:inblo ck in execute'“，” /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in`block in initialize'“] }

这些配置也破坏了我的logstash：

input {
    rabbitmq {
        id => "test1"
        type => "event"
        exchange => "event"
        exclusive => true
    }
}


input {
    id => "test1"
    rabbitmq {
        type => "event"
        exchange => "event"
        exclusive => true
    }
}

Answer 1

我找到了解决方案。

ID必须放置在过滤器插件字段（例如grok，mutate或json）中，而不仅仅是在过滤器字段中：

filter {
    if "test" in [tags]  {
        mutate {
            id => "test2"
            remove_field => [ "headers", "host" ]   
        }
    }
}

也不是所有版本的插件都支持IP字段。如有必要，请更新插件：

logstash-plugin update logstash-filter-mutate