我有从不同传感器记录温度+湿度的脚本，并将每个传感器的数据存储到他的目录中，每天都会以YYYY-MM-DD.log格式生成新的日志。
${data_root}/A/0/*.log
${data_root}/A/1/*.log
ETC..
日志采用以下格式：
2018-03-02 03:48:14 25.00 27.10
（YYYY-MM-DD TIME温度湿度）
我无法理解如何正确配置我的logstash实例，我认为我的输入应该如下所示：
input {
file{ path => "/var/wlogs/a1/*.log" type=>"a1"}
file{ path => "/var/wlogs/a2/*.log" type=>"a2"}
etc..
}
过滤器看起来像这样：
filter{
if [type] == "a1" {
grok {
match => { "message" => "(?<timestamp>%{YEAR}-%{MONTHNUM:month}-%{MONTHDAY:day} %{TIME}) %{NUMBER:temperature:float} %{NUMBER:humidity:float}" }
}
}
if [type] == "a2" {....}
我试图将输出部分中的数据导出到ElasticSearch但没有成功。
output{
elasticsearch { hosts =>["ec2-xxxxxx.eu-west-2.compute.amazonaws.com:9200"] user=>"elastic" password=>"pass" index=>"{type}"}
stdout{ codec => rubydebug}
}
这是我尝试运行时的控制台输出：
ubuntu@ip-xxx-xxx:/usr/share/logstash$ sudo bin/logstash -f ~/logstash.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-03-02 13:43:34.633 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[INFO ] 2018-03-02 13:43:34.647 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[WARN ] 2018-03-02 13:43:35.063 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-03-02 13:43:35.209 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.2"}
[INFO ] 2018-03-02 13:43:35.430 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-03-02 13:43:36.145 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-03-02 13:43:36.318 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:[email protected]:9200/]}}
[INFO ] 2018-03-02 13:43:36.327 [[main]-pipeline-manager] elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://elastic:[email protected]:9200/, :path=>"/"}
[WARN ] 2018-03-02 13:43:36.447 [[main]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"http://elastic:[email protected]:9200/"}
[INFO ] 2018-03-02 13:43:36.610 [[main]-pipeline-manager] elasticsearch - ES Output version determined {:es_version=>nil}
[WARN ] 2018-03-02 13:43:36.611 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[INFO ] 2018-03-02 13:43:36.616 [[main]-pipeline-manager] elasticsearch - Using mapping template from {:path=>nil}
[INFO ] 2018-03-02 13:43:36.619 [[main]-pipeline-manager] elasticsearch - Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[INFO ] 2018-03-02 13:43:36.626 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//ec2-no.eu-west-2.compute.amazonaws.com:9200"]}
[INFO ] 2018-03-02 13:43:37.054 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x25b5f422@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[INFO ] 2018-03-02 13:43:37.081 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}
请帮我弄清楚我做错了什么以及如何解决它:)提前谢谢
P.S：我正在使用最新版本的ElasticSearch，Kibana和Logstash
在日志中看不到任何错误。让我觉得日志文件可能已在之前的尝试中被读取过。由于文件偏移量保存在主目录的sincedb文件中，您是否可以停止logstash，删除该文件并重试？
有关sincedb文件的更多详细信息，请参阅https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html