Kibana未连接到Elasticsearch([“警告”,“ elasticsearch”,“ admin”],“ pid”:12,“消息”:“没有活动的连接”}))]

问题描述 投票:0回答:1

上下文:我整整一周都在苦苦挣扎,以确保可以正常运行此堆栈:filebeat-> kafka-> logstash-> elasticsearch-kibana,每个都在自己的docker中(您将在此处找到大约3或4个其他问题由于不同的尝试而没有答案)。我决定缩小堆栈大小,然后逐块移动,直到到达最终的docker-compose。然后,我尝试了我可以想象的最简单的堆栈来推进我可以想象的最简单的日志,并且我遇到了上面提到的问题。

问题:我正尝试从命令行三个docker容器直接运行:filebeat,elasticsearch和kibana。当我尝试启动kibana时,出现“没有任何生活联系”。我正在认真遵循another stackoverflow question中提供的答案。为什么我无法从Kibana容器连接到Elasticsearch容器?

这是所有三个泊坞窗命令:

docker run -d -p 9200:9200 -e "discovery.type=single-node" --volume C:\Dockers\simplest-try\esdata:/usr/share/elasticsearch/data --name elasticsearch_container docker.elastic.co/elasticsearch/elasticsearch:7.5.2

docker run -d --mount type=bind,source=C:\Dockers\simplest-try\filebeat.yml,target=/usr/share/filebeat/filebeat.yml --volume C:\Dockers\simplest-try\mylogs:/mylogs docker.elastic.co/beats/filebeat:7.5.2 

docker run -d --name kibana -p 5601:5601 --link elasticsearch_container:elasticsearch_alias -e "ELASTICSEARCH_URL=http://elasticsearch_alias:9200" docker.elastic.co/kibana/kibana:7.5.2

ElasticSearch已启动并正在运行:

C:\Dockers\simplest-try>curl localhost:9200
{
  "name" : "ffaa2d39a8b2",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "QWYLaAqwSqu76fNwFtZ5AA",
  "version" : {
    "number" : "7.5.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "8bec50e1e0ad29dad5653712cf3bb580cd1afcdf",
    "build_date" : "2020-01-15T12:11:52.313576Z",
    "build_snapshot" : false,
    "lucene_version" : "8.3.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Kibana容器控制台:

{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins-system"],"pid":6,"message":"Setting up [15] plugins: [security,licensing,code,timelion,features,spaces,translations,uiActions,newsfeed,inspector,embeddable,advancedUiActions,expressions,eui_utils,data]"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","security"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["warning","plugins","security","config"],"pid":6,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["warning","plugins","security","config"],"pid":6,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","licensing"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","code"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","timelion"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","features"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","spaces"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","translations"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","data"],"pid":6,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-06T14:53:41Z","tags":["error","elasticsearch","data"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/_xpack => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"}
{"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","legacy-plugins"],"pid":6,"path":"/usr/share/kibana/src/legacy/core_plugins/visualizations","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/visualizations"}
{"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["info","plugins-system"],"pid":6,"message":"Starting [8] plugins: [security,licensing,code,timelion,features,spaces,translations,data]"}
{"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"Unable to revive connection: http://elasticsearch:9200/"}
{"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"No living connections"}
{"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch for the [data] cluster. Error: No Living connections"}
{"type":"log","@timestamp":"2020-02-06T14:53:43Z","tags":["error","elasticsearch","admin"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/.kibana => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"}
{"type":"log","@timestamp":"2020-02-06T14:53:43Z","tags":["error","elasticsearch","admin"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/.kibana_task_manager => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"}
{"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"Unable to revive connection: http://elasticsearch:9200/"}
{"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"No living connections"}
{"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","migrations"],"pid":6,"message":"Unable to connect to Elasticsearch. Error: No Living connections"}

尽管不符合我的问题标题,但以下是有关Filebeat的详细信息:Filebeat尝试占用我的日志文件

2020-02-06T14:32:23.782Z    INFO    crawler/crawler.go:72   Loading Inputs: 1
2020-02-06T14:32:23.782Z    INFO    log/input.go:152    Configured paths: [/mylogs/*.log]
2020-02-06T14:32:23.782Z    INFO    input/input.go:114  Starting input of type: log; ID: 4094557846902174710 
2020-02-06T14:32:23.782Z    INFO    crawler/crawler.go:106  Loading and starting Inputs completed. Enabled inputs: 1
2020-02-06T14:32:23.788Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/y.log
2020-02-06T14:32:23.790Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/x.log

filebeat.yml

filebeat.inputs:
- type: log
  paths:
  - '/mylogs/*.log'
  json.message_key: log
  json.keys_under_root: true
  processors:
  - add_docker_metadata: ~
output.elasticsearch:
  hosts: ["localhost:9200"]

***已编辑

Ibexit建议后的日志

2020-02-12T21:33:03.575Z    INFO    instance/beat.go:610    Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2020-02-12T21:33:03.588Z    INFO    instance/beat.go:618    Beat ID: d0c71c07-23e0-44e5-b497-195ee9552fe8
2020-02-12T21:33:03.588Z    INFO    [seccomp]   seccomp/seccomp.go:124  Syscall filter successfully installed
2020-02-12T21:33:03.588Z    INFO    [beat]  instance/beat.go:941    Beat info   {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "d0c71c07-23e0-44e5-b497-195ee9552fe8"}}}
2020-02-12T21:33:03.588Z    INFO    [beat]  instance/beat.go:950    Build info  {"system_info": {"build": {"commit": "a9c141434cd6b25d7a74a9c770be6b70643dc767", "libbeat": "7.5.2", "time": "2020-01-15T11:13:22.000Z", "version": "7.5.2"}}}
2020-02-12T21:33:03.588Z    INFO    [beat]  instance/beat.go:953    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.12"}}}
2020-02-12T21:33:03.590Z    INFO    [beat]  instance/beat.go:957    Host info   {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-02-12T20:32:39Z","containerized":true,"name":"fcfaea4080e7","ip":["127.0.0.1/8","172.17.0.3/16"],"kernel_version":"4.19.76-linuxkit","mac":["02:42:ac:11:00:03"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":7,"patch":1908,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}}
2020-02-12T21:33:03.590Z    INFO    [beat]  instance/beat.go:986    Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-02-12T21:33:02.690Z"}}}
2020-02-12T21:33:03.590Z    INFO    instance/beat.go:297    Setup Beat: filebeat; Version: 7.5.2
2020-02-12T21:33:03.590Z    INFO    [index-management]  idxmgmt/std.go:182  Set output.elasticsearch.index to 'filebeat-7.5.2' as ILM is enabled.
2020-02-12T21:33:03.591Z    INFO    elasticsearch/client.go:171 Elasticsearch url: http://elasticsearch:9200
2020-02-12T21:33:03.591Z    INFO    [publisher] pipeline/module.go:97   Beat name: fcfaea4080e7
2020-02-12T21:33:03.593Z    INFO    instance/beat.go:429    filebeat start running.
2020-02-12T21:33:03.593Z    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2020-02-12T21:33:03.594Z    INFO    registrar/migrate.go:104    No registry home found. Create: /usr/share/filebeat/data/registry/filebeat
2020-02-12T21:33:03.594Z    INFO    registrar/migrate.go:112    Initialize registry meta file
2020-02-12T21:33:03.600Z    INFO    registrar/registrar.go:108  No registry file found under: /usr/share/filebeat/data/registry/filebeat/data.json. Creating a new registry file.
2020-02-12T21:33:03.611Z    INFO    registrar/registrar.go:145  Loading registrar data from /usr/share/filebeat/data/registry/filebeat/data.json
2020-02-12T21:33:03.611Z    INFO    registrar/registrar.go:152  States Loaded from registrar: 0
2020-02-12T21:33:03.612Z    INFO    crawler/crawler.go:72   Loading Inputs: 1
2020-02-12T21:33:03.612Z    INFO    log/input.go:152    Configured paths: [/mylogs/*.log]
2020-02-12T21:33:03.612Z    INFO    input/input.go:114  Starting input of type: log; ID: 4094557846902174710 
2020-02-12T21:33:03.612Z    INFO    crawler/crawler.go:106  Loading and starting Inputs completed. Enabled inputs: 1
2020-02-12T21:33:03.640Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/b.log
2020-02-12T21:33:03.640Z    ERROR   readjson/json.go:52 Error decoding JSON: invalid character '\'' looking for beginning of object key string
2020-02-12T21:33:03.642Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/c.log
2020-02-12T21:33:03.644Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/w.log
2020-02-12T21:33:03.645Z    ERROR   readjson/json.go:52 Error decoding JSON: invalid character 'q' looking for beginning of value
2020-02-12T21:33:03.645Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/x.log
2020-02-12T21:33:03.652Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/y.log
2020-02-12T21:33:04.654Z    INFO    pipeline/output.go:95   Connecting to backoff(elasticsearch(http://elasticsearch:9200))
2020-02-12T21:33:04.684Z    INFO    elasticsearch/client.go:753 Attempting to connect to Elasticsearch version 7.5.2
2020-02-12T21:33:04.720Z    INFO    [index-management]  idxmgmt/std.go:256  Auto ILM enable success.
2020-02-12T21:33:04.724Z    INFO    [index-management.ilm]  ilm/std.go:138  do not generate ilm policy: exists=true, overwrite=false
2020-02-12T21:33:04.724Z    INFO    [index-management]  idxmgmt/std.go:269  ILM policy successfully loaded.
2020-02-12T21:33:04.725Z    INFO    [index-management]  idxmgmt/std.go:408  Set setup.template.name to '{filebeat-7.5.2 {now/d}-000001}' as ILM is enabled.
2020-02-12T21:33:04.725Z    INFO    [index-management]  idxmgmt/std.go:413  Set setup.template.pattern to 'filebeat-7.5.2-*' as ILM is enabled.
2020-02-12T21:33:04.725Z    INFO    [index-management]  idxmgmt/std.go:447  Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.5.2 {now/d}-000001} as ILM is enabled.
2020-02-12T21:33:04.725Z    INFO    [index-management]  idxmgmt/std.go:451  Set settings.index.lifecycle.name in template to {filebeat-7.5.2 {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2020-02-12T21:33:04.730Z    INFO    template/load.go:89 Template filebeat-7.5.2 already exists and will not be overwritten.
2020-02-12T21:33:04.730Z    INFO    [index-management]  idxmgmt/std.go:293  Loaded index template.
2020-02-12T21:33:04.734Z    INFO    [index-management]  idxmgmt/std.go:304  Write alias successfully generated.
2020-02-12T21:33:04.736Z    INFO    pipeline/output.go:105  Connection to backoff(elasticsearch(http://elasticsearch:9200)) established
2020-02-12T21:33:33.595Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":50}},"total":{"ticks":100,"time":{"ms":107},"value":100},"user":{"ticks":50,"time":{"ms":57}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":30060}},"memstats":{"gc_next":8351264,"memory_alloc":4760176,"memory_total":12037984,"rss":43970560},"runtime":{"goroutines":42}},"filebeat":{"events":{"added":8,"done":8},"harvester":{"open_files":5,"running":5,"started":5}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":3,"batches":1,"total":3},"read":{"bytes":2942},"type":"elasticsearch","write":{"bytes":2545}},"pipeline":{"clients":1,"events":{"active":0,"filtered":5,"published":3,"retry":3,"total":8},"queue":{"acked":3}}},"registrar":{"states":{"current":5,"update":8},"writes":{"success":7,"total":7}},"system":{"cpu":{"cores":2},"load":{"1":0.02,"15":0.08,"5":0.1,"norm":{"1":0.01,"15":0.04,"5":0.05}}}}}}
2020-02-12T21:33:58.657Z    ERROR   readjson/json.go:52 Error decoding JSON: invalid character 'E' looking for beginning of value
2020-02-12T21:33:58.657Z    ERROR   readjson/json.go:52 Error decoding JSON: invalid character 'a' looking for beginning of value
2020-02-12T21:34:03.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":13}},"total":{"ticks":120,"time":{"ms":16},"value":120},"user":{"ticks":60,"time":{"ms":3}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":60059}},"memstats":{"gc_next":8351264,"memory_alloc":5345000,"memory_total":12622808},"runtime":{"goroutines":42}},"filebeat":{"events":{"added":2,"done":2},"harvester":{"open_files":5,"running":5}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":2,"batches":1,"total":2},"read":{"bytes":351},"write":{"bytes":1062}},"pipeline":{"clients":1,"events":{"active":0,"published":2,"total":2},"queue":{"acked":2}}},"registrar":{"states":{"current":5,"update":2},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.01,"15":0.08,"5":0.09,"norm":{"1":0.005,"15":0.04,"5":0.045}}}}}}
2020-02-12T21:34:33.599Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":70,"time":{"ms":10}},"total":{"ticks":130,"time":{"ms":14},"value":130},"user":{"ticks":60,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":90059}},"memstats":{"gc_next":8351264,"memory_alloc":5714936,"memory_total":12992744,"rss":380928},"runtime":{"goroutines":42}},"filebeat":{"harvester":{"open_files":5,"running":5}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":5}},"system":{"load":{"1":0.07,"15":0.08,"5":0.1,"norm":{"1":0.035,"15":0.04,"5":0.05}}}}}}
2020-02-12T21:34:33.686Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/d.log
2020-02-12T21:35:03.597Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":80,"time":{"ms":16}},"total":{"ticks":140,"time":{"ms":21},"value":140},"user":{"ticks":60,"time":{"ms":5}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":120059}},"memstats":{"gc_next":8351264,"memory_alloc":6130552,"memory_total":13408360},"runtime":{"goroutines":46}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":6,"running":6,"started":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":6,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.15,"15":0.09,"5":0.12,"norm":{"1":0.075,"15":0.045,"5":0.06}}}}}}
2020-02-12T21:35:33.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":100,"time":{"ms":14}},"total":{"ticks":170,"time":{"ms":23},"value":170},"user":{"ticks":70,"time":{"ms":9}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":150060}},"memstats":{"gc_next":7948720,"memory_alloc":4110408,"memory_total":13866968},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.09,"15":0.08,"5":0.11,"norm":{"1":0.045,"15":0.04,"5":0.055}}}}}}
2020-02-12T21:36:03.597Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":110,"time":{"ms":7}},"total":{"ticks":190,"time":{"ms":9},"value":190},"user":{"ticks":80,"time":{"ms":2}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":180059}},"memstats":{"gc_next":7948720,"memory_alloc":4399584,"memory_total":14156144},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.38,"15":0.11,"5":0.18,"norm":{"1":0.19,"15":0.055,"5":0.09}}}}}}
2020-02-12T21:36:33.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":120,"time":{"ms":11}},"total":{"ticks":200,"time":{"ms":15},"value":200},"user":{"ticks":80,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":210059}},"memstats":{"gc_next":7948720,"memory_alloc":4776320,"memory_total":14532880},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.23,"15":0.1,"5":0.16,"norm":{"1":0.115,"15":0.05,"5":0.08}}}}}}
2020-02-12T21:37:03.600Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":120,"time":{"ms":9}},"total":{"ticks":210,"time":{"ms":16},"value":210},"user":{"ticks":90,"time":{"ms":7}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":240059}},"memstats":{"gc_next":7948720,"memory_alloc":5142416,"memory_total":14898976},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.14,"15":0.1,"5":0.14,"norm":{"1":0.07,"15":0.05,"5":0.07}}}}}}
2020-02-12T21:37:33.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":140,"time":{"ms":12}},"total":{"ticks":240,"time":{"ms":24},"value":240},"user":{"ticks":100,"time":{"ms":12}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":270060}},"memstats":{"gc_next":7946160,"memory_alloc":4111832,"memory_total":15348288},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.08,"15":0.09,"5":0.13,"norm":{"1":0.04,"15":0.045,"5":0.065}}}}}}
2020-02-12T21:38:03.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":150,"time":{"ms":11}},"total":{"ticks":250,"time":{"ms":12},"value":250},"user":{"ticks":100,"time":{"ms":1}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":300060}},"memstats":{"gc_next":7946160,"memory_alloc":4489960,"memory_total":15726416},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.1,"15":0.09,"5":0.13,"norm":{"1":0.05,"15":0.045,"5":0.065}}}}}}
2020-02-12T21:38:08.676Z    INFO    log/harvester.go:276    File is inactive: /mylogs/w.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:38:08.676Z    INFO    log/harvester.go:276    File is inactive: /mylogs/c.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:38:08.678Z    INFO    log/harvester.go:276    File is inactive: /mylogs/b.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:38:08.678Z    INFO    log/harvester.go:276    File is inactive: /mylogs/y.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:38:13.706Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/y.log
2020-02-12T21:38:33.594Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":150,"time":{"ms":5}},"total":{"ticks":250,"time":{"ms":9},"value":250},"user":{"ticks":100,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":330059}},"memstats":{"gc_next":7946160,"memory_alloc":5014240,"memory_total":16250696},"runtime":{"goroutines":34}},"filebeat":{"events":{"added":5,"done":5},"harvester":{"closed":4,"open_files":3,"running":3,"started":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":5,"total":5}}},"registrar":{"states":{"current":6,"update":5},"writes":{"success":5,"total":5}},"system":{"load":{"1":0.88,"15":0.15,"5":0.31,"norm":{"1":0.44,"15":0.075,"5":0.155}}}}}}
2020-02-12T21:39:03.595Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":160,"time":{"ms":6}},"total":{"ticks":270,"time":{"ms":8},"value":270},"user":{"ticks":110,"time":{"ms":2}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":360059}},"memstats":{"gc_next":7946160,"memory_alloc":5284712,"memory_total":16521168},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.68,"15":0.16,"5":0.31,"norm":{"1":0.34,"15":0.08,"5":0.155}}}}}}
2020-02-12T21:39:03.676Z    INFO    log/harvester.go:276    File is inactive: /mylogs/x.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:39:33.596Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":160,"time":{"ms":5}},"total":{"ticks":270,"time":{"ms":12},"value":270},"user":{"ticks":110,"time":{"ms":7}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":390059}},"memstats":{"gc_next":7666032,"memory_alloc":3879448,"memory_total":16793464},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"closed":1,"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":6,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.48,"15":0.16,"5":0.3,"norm":{"1":0.24,"15":0.08,"5":0.15}}}}}}
2020-02-12T21:39:38.705Z    INFO    log/harvester.go:276    File is inactive: /mylogs/d.log. Closing because close_inactive of 5m0s reached.
2020-02-12T21:39:43.714Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/d.log
2020-02-12T21:39:43.715Z    ERROR   readjson/json.go:52 Error decoding JSON: EOF
2020-02-12T21:39:49.724Z    INFO    log/harvester.go:264    File was truncated. Begin reading file from offset 0: /mylogs/d.log
2020-02-12T21:39:53.720Z    INFO    log/harvester.go:251    Harvester started for file: /mylogs/d.log
2020-02-12T21:39:53.721Z    ERROR   readjson/json.go:52 Error decoding JSON: EOF
2020-02-12T21:40:03.597Z    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":190,"time":{"ms":30}},"total":{"ticks":320,"time":{"ms":46},"value":320},"user":{"ticks":130,"time":{"ms":16}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":420059}},"memstats":{"gc_next":7666032,"memory_alloc":4930512,"memory_total":17844528},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":8,"done":8},"harvester":{"closed":2,"open_files":2,"running":2,"started":2},"input":{"log":{"files":{"truncated":1}}}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":4,"batches":2,"total":4},"read":{"bytes":702},"write":{"bytes":2270}},"pipeline":{"clients":1,"events":{"active":0,"filtered":4,"published":4,"total":8},"queue":{"acked":4}}},"registrar":{"states":{"current":6,"update":8},"writes":{"success":6,"total":6}},"system":{"load":{"1":0.59,"15":0.17,"5":0.33,"norm":{"1":0.295,"15":0.085,"5":0.165}}}}}}

上下文:我整整一个星期都在努力挣扎,以使其运行起来:filebeat-> kafka-> logstash-> elasticsearch-kibana,每个人都在自己的docker中(您会发现另外3或4个...

docker elasticsearch kibana filebeat
1个回答
1
投票

问题在于,这三个容器在网络方面相互分离和/或配置错误。让我们讨论实际发生的情况以及如何解决它:

1。 Elasticsearch

© www.soinside.com 2019 - 2024. All rights reserved.