我一直在使用docker中运行的fluent-plugin-elasticsearch插件来获得fluentd的ARM版本（针对Raspberry Pi 3和4）。我一直无法找到合适的docker映像，因此我自己构建了一个（如果有人知道在哪里可以找到我，我将不胜感激）。我从fluentd-docker-image存储库开始（不包括Elasticsearch插件），并根据需要使用fluentd-kubernetes-daemonset存储库（包括Elasticsearch插件）对其进行了修改。好消息是，它在Raspberry Pi上启动正常。坏消息是它似乎甚至没有尝试连接到ElasticSearch（Raspberry Pi网络的外部。日志文件看起来像这样：
rich_youngkin:efk-kubernetes$ ks logs fluentd-f8clh
2020-02-29 23:55:27 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-concat' version '2.4.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-dedot_filter' version '1.0.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.12'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.4'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-grok-parser' version '2.6.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-json-in-json-2' version '1.0.2'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '2.3.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.6.1'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.0.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.2.0'
2020-02-29 23:55:27 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.1'
2020-02-29 23:55:27 +0000 [info]: gem 'fluentd' version '1.9.2'
2020-02-29 23:55:27 +0000 [warn]: [output_docker1] 'time_format' specified without 'time_key', will be ignored
2020-02-29 23:55:27 +0000 [warn]: [output1] 'time_format' specified without 'time_key', will be ignored
2020-02-29 23:55:28 +0000 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2020-02-29 23:55:28 +0000 [info]: using configuration file: <ROOT>
<source>
@type forward
@id input1
@label @mainstream
port 24224
</source>
<filter **>
@type stdout
</filter>
<label @mainstream>
<match docker.**>
@type file
@id output_docker1
path "/fluentd/log/docker.*.log"
symlink_path "/fluentd/log/docker.log"
append true
time_slice_format %Y%m%d
time_slice_wait 1m
time_format %Y%m%dT%H%M%S%z
<buffer time>
timekey_wait 1m
timekey 86400
path /fluentd/log/docker.*.log
</buffer>
<inject>
time_format %Y%m%dT%H%M%S%z
</inject>
</match>
<match **>
@type file
@id output1
path "/fluentd/log/data.*.log"
symlink_path "/fluentd/log/data.log"
append true
time_slice_format %Y%m%d
time_slice_wait 10m
time_format %Y%m%dT%H%M%S%z
<buffer time>
timekey_wait 10m
timekey 86400
path /fluentd/log/data.*.log
</buffer>
<inject>
time_format %Y%m%dT%H%M%S%z
</inject>
</match>
<match **>
@type elasticsearch
@id out_es
@log_level "info"
include_tag_key true
host "10.0.0.223"
port 9200
path ""
scheme http
ssl_verify true
ssl_version TLSv1_2
user
password xxxxxx
reload_connections false
reconnect_on_error true
reload_on_failure true
log_es_400_reason false
logstash_prefix "logstash"
logstash_dateformat "%Y.%m.%d"
logstash_format true
index_name "logstash"
type_name "fluentd"
include_timestamp false
template_name
template_file
template_overwrite false
sniffer_class_name "Fluent::Plugin::ElasticsearchSimpleSniffer"
request_timeout 5s
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
</label>
</ROOT>
2020-02-29 23:55:28 +0000 [info]: starting fluentd-1.9.2 pid=6 ruby="2.6.5"
2020-02-29 23:55:28 +0000 [info]: spawn command to main: cmdline=["/usr/local/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/local/bundle/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "-p", "/fluentd/plugins", "-r", "/usr/local/bundle/gems/fluent-plugin-elasticsearch-4.0.4/lib/fluent/plugin/elasticsearch_simple_sniffer.rb", "--under-supervisor"]
2020-02-29 23:55:31 +0000 [info]: adding match in @mainstream pattern="docker.**" type="file"
2020-02-29 23:55:31 +0000 [warn]: #0 [output_docker1] 'time_format' specified without 'time_key', will be ignored
2020-02-29 23:55:31 +0000 [info]: adding match in @mainstream pattern="**" type="file"
2020-02-29 23:55:31 +0000 [warn]: #0 [output1] 'time_format' specified without 'time_key', will be ignored
2020-02-29 23:55:31 +0000 [info]: adding match in @mainstream pattern="**" type="elasticsearch"
2020-02-29 23:55:32 +0000 [info]: #0 [out_es] Detected ES 6.x: ES 7.x will only accept `_doc` in type_name.
2020-02-29 23:55:32 +0000 [info]: adding filter pattern="**" type="stdout"
2020-02-29 23:55:32 +0000 [info]: adding source type="forward"
2020-02-29 23:55:32 +0000 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2020-02-29 23:55:32 +0000 [info]: #0 starting fluentd worker pid=21 ppid=6 worker=0
2020-02-29 23:55:32 +0000 [info]: #0 [input1] listening port port=24224 bind="0.0.0.0"
2020-02-29 23:55:32 +0000 [info]: #0 fluentd worker is now running worker=0
2020-02-29 23:55:32.241579761 +0000 fluent.info: {"pid":21,"ppid":6,"worker":0,"message":"starting fluentd worker pid=21 ppid=6 worker=0"}
2020-02-29 23:55:32 +0000 [warn]: #0 no patterns matched tag="fluent.info"
2020-02-29 23:55:32.248536560 +0000 fluent.info: {"port":24224,"bind":"0.0.0.0","message":"[input1] listening port port=24224 bind=\"0.0.0.0\""}
2020-02-29 23:55:32 +0000 [warn]: #0 no patterns matched tag="fluent.info"
2020-02-29 23:55:32.251654489 +0000 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
为了进一步缩小日志范围，流利的似乎对Elasticsearch有所了解
2020-02-29 23:55:32 +0000 [info]: #0 [out_es] Detected ES 6.x: ES 7.x will only accept `_doc` in type_name.
但是我看不到任何有关连接甚至试图连接到Elasticsearch的消息。我希望成功时会看到类似这样的消息，或者某种错误消息。
2020-02-28 21:56:26 +0000 [info]: #0 [out_es] Connection opened to Elasticsearch cluster => {:host=>"10.0.0.223", :port=>9200, :scheme=>"http", :path=>""}
配置已记录在上面，对我来说似乎是正确的。日志中记录的命令行对我来说也很好。
/usr/local/bin/ruby -Eascii-8bit:ascii-8bit /usr/local/bundle/bin/fluentd -c /fluentd/etc/fluent.conf -p /fluentd/plugins -r /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.0.4/lib/fluent/plugin/elasticsearch_simple_sniffer.rb --under-supervisor
登录到pod并连接到Elasticsearch也可以：
$ curl http://10.0.0.223:9200
{
"name" : "7j27EYH",
"cluster_name" : "elasticsearch_rich_youngkin",
"cluster_uuid" : "ahiZBT3yQLCP-W4gdA5TsQ",
"version" : {
"number" : "6.8.6",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "3d9f765",
"build_date" : "2019-12-13T17:11:52.013738Z",
"build_snapshot" : false,
"lucene_version" : "7.7.2",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
有人可以说明与Elasticsearch主机之间缺乏连通性吗？
谢谢，丰富
如Efrat Levitan所述，问题确实是版本不匹配。我安装了Elasticsearch 7.6.0和相应的Kibana版本7.6.0，可以流畅地尝试连接。我的连接被拒绝，所以进展:)进入下一个问题...