上下文:我整整一周都在苦苦挣扎,以确保可以正常运行此堆栈:filebeat-> kafka-> logstash-> elasticsearch-kibana,每个都在自己的docker中(您将在此处找到大约3或4个其他问题由于不同的尝试而没有答案)。我决定缩小堆栈大小,然后逐块移动,直到到达最终的docker-compose。然后,我尝试了我可以想象的最简单的堆栈来推进我可以想象的最简单的日志,并且我遇到了上面提到的问题。
问题:我正尝试从命令行三个docker容器直接运行:filebeat,elasticsearch和kibana。当我尝试启动kibana时,出现“没有任何生活联系”。我正在认真遵循another stackoverflow question中提供的答案。为什么我无法从Kibana容器连接到Elasticsearch容器?
这是所有三个泊坞窗命令:
docker run -d -p 9200:9200 -e "discovery.type=single-node" --volume C:\Dockers\simplest-try\esdata:/usr/share/elasticsearch/data --name elasticsearch_container docker.elastic.co/elasticsearch/elasticsearch:7.5.2 docker run -d --mount type=bind,source=C:\Dockers\simplest-try\filebeat.yml,target=/usr/share/filebeat/filebeat.yml --volume C:\Dockers\simplest-try\mylogs:/mylogs docker.elastic.co/beats/filebeat:7.5.2 docker run -d --name kibana -p 5601:5601 --link elasticsearch_container:elasticsearch_alias -e "ELASTICSEARCH_URL=http://elasticsearch_alias:9200" docker.elastic.co/kibana/kibana:7.5.2
ElasticSearch已启动并正在运行:
C:\Dockers\simplest-try>curl localhost:9200 { "name" : "ffaa2d39a8b2", "cluster_name" : "docker-cluster", "cluster_uuid" : "QWYLaAqwSqu76fNwFtZ5AA", "version" : { "number" : "7.5.2", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "8bec50e1e0ad29dad5653712cf3bb580cd1afcdf", "build_date" : "2020-01-15T12:11:52.313576Z", "build_snapshot" : false, "lucene_version" : "8.3.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }
Kibana容器控制台:
{"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins-system"],"pid":6,"message":"Setting up [15] plugins: [security,licensing,code,timelion,features,spaces,translations,uiActions,newsfeed,inspector,embeddable,advancedUiActions,expressions,eui_utils,data]"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","security"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["warning","plugins","security","config"],"pid":6,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["warning","plugins","security","config"],"pid":6,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","licensing"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","code"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","timelion"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","features"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","spaces"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","translations"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:25Z","tags":["info","plugins","data"],"pid":6,"message":"Setting up plugin"} {"type":"log","@timestamp":"2020-02-06T14:53:41Z","tags":["error","elasticsearch","data"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/_xpack => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"} {"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","legacy-plugins"],"pid":6,"path":"/usr/share/kibana/src/legacy/core_plugins/visualizations","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/visualizations"} {"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["info","plugins-system"],"pid":6,"message":"Starting [8] plugins: [security,licensing,code,timelion,features,spaces,translations,data]"} {"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"Unable to revive connection: http://elasticsearch:9200/"} {"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"No living connections"} {"type":"log","@timestamp":"2020-02-06T14:53:42Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch for the [data] cluster. Error: No Living connections"} {"type":"log","@timestamp":"2020-02-06T14:53:43Z","tags":["error","elasticsearch","admin"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/.kibana => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"} {"type":"log","@timestamp":"2020-02-06T14:53:43Z","tags":["error","elasticsearch","admin"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/.kibana_task_manager => getaddrinfo ENOTFOUND elasticsearch elasticsearch:9200"} {"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"Unable to revive connection: http://elasticsearch:9200/"} {"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"No living connections"} {"type":"log","@timestamp":"2020-02-06T14:53:44Z","tags":["warning","migrations"],"pid":6,"message":"Unable to connect to Elasticsearch. Error: No Living connections"}
尽管不符合我的问题标题,但以下是有关Filebeat的详细信息:Filebeat尝试占用我的日志文件
2020-02-06T14:32:23.782Z INFO crawler/crawler.go:72 Loading Inputs: 1 2020-02-06T14:32:23.782Z INFO log/input.go:152 Configured paths: [/mylogs/*.log] 2020-02-06T14:32:23.782Z INFO input/input.go:114 Starting input of type: log; ID: 4094557846902174710 2020-02-06T14:32:23.782Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1 2020-02-06T14:32:23.788Z INFO log/harvester.go:251 Harvester started for file: /mylogs/y.log 2020-02-06T14:32:23.790Z INFO log/harvester.go:251 Harvester started for file: /mylogs/x.log
filebeat.yml
filebeat.inputs: - type: log paths: - '/mylogs/*.log' json.message_key: log json.keys_under_root: true processors: - add_docker_metadata: ~ output.elasticsearch: hosts: ["localhost:9200"]
***已编辑
Ibexit建议后的日志
2020-02-12T21:33:03.575Z INFO instance/beat.go:610 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] 2020-02-12T21:33:03.588Z INFO instance/beat.go:618 Beat ID: d0c71c07-23e0-44e5-b497-195ee9552fe8 2020-02-12T21:33:03.588Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed 2020-02-12T21:33:03.588Z INFO [beat] instance/beat.go:941 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "d0c71c07-23e0-44e5-b497-195ee9552fe8"}}} 2020-02-12T21:33:03.588Z INFO [beat] instance/beat.go:950 Build info {"system_info": {"build": {"commit": "a9c141434cd6b25d7a74a9c770be6b70643dc767", "libbeat": "7.5.2", "time": "2020-01-15T11:13:22.000Z", "version": "7.5.2"}}} 2020-02-12T21:33:03.588Z INFO [beat] instance/beat.go:953 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.12"}}} 2020-02-12T21:33:03.590Z INFO [beat] instance/beat.go:957 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-02-12T20:32:39Z","containerized":true,"name":"fcfaea4080e7","ip":["127.0.0.1/8","172.17.0.3/16"],"kernel_version":"4.19.76-linuxkit","mac":["02:42:ac:11:00:03"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":7,"patch":1908,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}} 2020-02-12T21:33:03.590Z INFO [beat] instance/beat.go:986 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-02-12T21:33:02.690Z"}}} 2020-02-12T21:33:03.590Z INFO instance/beat.go:297 Setup Beat: filebeat; Version: 7.5.2 2020-02-12T21:33:03.590Z INFO [index-management] idxmgmt/std.go:182 Set output.elasticsearch.index to 'filebeat-7.5.2' as ILM is enabled. 2020-02-12T21:33:03.591Z INFO elasticsearch/client.go:171 Elasticsearch url: http://elasticsearch:9200 2020-02-12T21:33:03.591Z INFO [publisher] pipeline/module.go:97 Beat name: fcfaea4080e7 2020-02-12T21:33:03.593Z INFO instance/beat.go:429 filebeat start running. 2020-02-12T21:33:03.593Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s 2020-02-12T21:33:03.594Z INFO registrar/migrate.go:104 No registry home found. Create: /usr/share/filebeat/data/registry/filebeat 2020-02-12T21:33:03.594Z INFO registrar/migrate.go:112 Initialize registry meta file 2020-02-12T21:33:03.600Z INFO registrar/registrar.go:108 No registry file found under: /usr/share/filebeat/data/registry/filebeat/data.json. Creating a new registry file. 2020-02-12T21:33:03.611Z INFO registrar/registrar.go:145 Loading registrar data from /usr/share/filebeat/data/registry/filebeat/data.json 2020-02-12T21:33:03.611Z INFO registrar/registrar.go:152 States Loaded from registrar: 0 2020-02-12T21:33:03.612Z INFO crawler/crawler.go:72 Loading Inputs: 1 2020-02-12T21:33:03.612Z INFO log/input.go:152 Configured paths: [/mylogs/*.log] 2020-02-12T21:33:03.612Z INFO input/input.go:114 Starting input of type: log; ID: 4094557846902174710 2020-02-12T21:33:03.612Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1 2020-02-12T21:33:03.640Z INFO log/harvester.go:251 Harvester started for file: /mylogs/b.log 2020-02-12T21:33:03.640Z ERROR readjson/json.go:52 Error decoding JSON: invalid character '\'' looking for beginning of object key string 2020-02-12T21:33:03.642Z INFO log/harvester.go:251 Harvester started for file: /mylogs/c.log 2020-02-12T21:33:03.644Z INFO log/harvester.go:251 Harvester started for file: /mylogs/w.log 2020-02-12T21:33:03.645Z ERROR readjson/json.go:52 Error decoding JSON: invalid character 'q' looking for beginning of value 2020-02-12T21:33:03.645Z INFO log/harvester.go:251 Harvester started for file: /mylogs/x.log 2020-02-12T21:33:03.652Z INFO log/harvester.go:251 Harvester started for file: /mylogs/y.log 2020-02-12T21:33:04.654Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://elasticsearch:9200)) 2020-02-12T21:33:04.684Z INFO elasticsearch/client.go:753 Attempting to connect to Elasticsearch version 7.5.2 2020-02-12T21:33:04.720Z INFO [index-management] idxmgmt/std.go:256 Auto ILM enable success. 2020-02-12T21:33:04.724Z INFO [index-management.ilm] ilm/std.go:138 do not generate ilm policy: exists=true, overwrite=false 2020-02-12T21:33:04.724Z INFO [index-management] idxmgmt/std.go:269 ILM policy successfully loaded. 2020-02-12T21:33:04.725Z INFO [index-management] idxmgmt/std.go:408 Set setup.template.name to '{filebeat-7.5.2 {now/d}-000001}' as ILM is enabled. 2020-02-12T21:33:04.725Z INFO [index-management] idxmgmt/std.go:413 Set setup.template.pattern to 'filebeat-7.5.2-*' as ILM is enabled. 2020-02-12T21:33:04.725Z INFO [index-management] idxmgmt/std.go:447 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.5.2 {now/d}-000001} as ILM is enabled. 2020-02-12T21:33:04.725Z INFO [index-management] idxmgmt/std.go:451 Set settings.index.lifecycle.name in template to {filebeat-7.5.2 {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled. 2020-02-12T21:33:04.730Z INFO template/load.go:89 Template filebeat-7.5.2 already exists and will not be overwritten. 2020-02-12T21:33:04.730Z INFO [index-management] idxmgmt/std.go:293 Loaded index template. 2020-02-12T21:33:04.734Z INFO [index-management] idxmgmt/std.go:304 Write alias successfully generated. 2020-02-12T21:33:04.736Z INFO pipeline/output.go:105 Connection to backoff(elasticsearch(http://elasticsearch:9200)) established 2020-02-12T21:33:33.595Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":50}},"total":{"ticks":100,"time":{"ms":107},"value":100},"user":{"ticks":50,"time":{"ms":57}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":30060}},"memstats":{"gc_next":8351264,"memory_alloc":4760176,"memory_total":12037984,"rss":43970560},"runtime":{"goroutines":42}},"filebeat":{"events":{"added":8,"done":8},"harvester":{"open_files":5,"running":5,"started":5}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":3,"batches":1,"total":3},"read":{"bytes":2942},"type":"elasticsearch","write":{"bytes":2545}},"pipeline":{"clients":1,"events":{"active":0,"filtered":5,"published":3,"retry":3,"total":8},"queue":{"acked":3}}},"registrar":{"states":{"current":5,"update":8},"writes":{"success":7,"total":7}},"system":{"cpu":{"cores":2},"load":{"1":0.02,"15":0.08,"5":0.1,"norm":{"1":0.01,"15":0.04,"5":0.05}}}}}} 2020-02-12T21:33:58.657Z ERROR readjson/json.go:52 Error decoding JSON: invalid character 'E' looking for beginning of value 2020-02-12T21:33:58.657Z ERROR readjson/json.go:52 Error decoding JSON: invalid character 'a' looking for beginning of value 2020-02-12T21:34:03.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":13}},"total":{"ticks":120,"time":{"ms":16},"value":120},"user":{"ticks":60,"time":{"ms":3}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":60059}},"memstats":{"gc_next":8351264,"memory_alloc":5345000,"memory_total":12622808},"runtime":{"goroutines":42}},"filebeat":{"events":{"added":2,"done":2},"harvester":{"open_files":5,"running":5}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":2,"batches":1,"total":2},"read":{"bytes":351},"write":{"bytes":1062}},"pipeline":{"clients":1,"events":{"active":0,"published":2,"total":2},"queue":{"acked":2}}},"registrar":{"states":{"current":5,"update":2},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.01,"15":0.08,"5":0.09,"norm":{"1":0.005,"15":0.04,"5":0.045}}}}}} 2020-02-12T21:34:33.599Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":70,"time":{"ms":10}},"total":{"ticks":130,"time":{"ms":14},"value":130},"user":{"ticks":60,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":90059}},"memstats":{"gc_next":8351264,"memory_alloc":5714936,"memory_total":12992744,"rss":380928},"runtime":{"goroutines":42}},"filebeat":{"harvester":{"open_files":5,"running":5}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":5}},"system":{"load":{"1":0.07,"15":0.08,"5":0.1,"norm":{"1":0.035,"15":0.04,"5":0.05}}}}}} 2020-02-12T21:34:33.686Z INFO log/harvester.go:251 Harvester started for file: /mylogs/d.log 2020-02-12T21:35:03.597Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":80,"time":{"ms":16}},"total":{"ticks":140,"time":{"ms":21},"value":140},"user":{"ticks":60,"time":{"ms":5}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":120059}},"memstats":{"gc_next":8351264,"memory_alloc":6130552,"memory_total":13408360},"runtime":{"goroutines":46}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":6,"running":6,"started":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":6,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.15,"15":0.09,"5":0.12,"norm":{"1":0.075,"15":0.045,"5":0.06}}}}}} 2020-02-12T21:35:33.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":100,"time":{"ms":14}},"total":{"ticks":170,"time":{"ms":23},"value":170},"user":{"ticks":70,"time":{"ms":9}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":150060}},"memstats":{"gc_next":7948720,"memory_alloc":4110408,"memory_total":13866968},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.09,"15":0.08,"5":0.11,"norm":{"1":0.045,"15":0.04,"5":0.055}}}}}} 2020-02-12T21:36:03.597Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":110,"time":{"ms":7}},"total":{"ticks":190,"time":{"ms":9},"value":190},"user":{"ticks":80,"time":{"ms":2}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":180059}},"memstats":{"gc_next":7948720,"memory_alloc":4399584,"memory_total":14156144},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.38,"15":0.11,"5":0.18,"norm":{"1":0.19,"15":0.055,"5":0.09}}}}}} 2020-02-12T21:36:33.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":120,"time":{"ms":11}},"total":{"ticks":200,"time":{"ms":15},"value":200},"user":{"ticks":80,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":210059}},"memstats":{"gc_next":7948720,"memory_alloc":4776320,"memory_total":14532880},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.23,"15":0.1,"5":0.16,"norm":{"1":0.115,"15":0.05,"5":0.08}}}}}} 2020-02-12T21:37:03.600Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":120,"time":{"ms":9}},"total":{"ticks":210,"time":{"ms":16},"value":210},"user":{"ticks":90,"time":{"ms":7}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":240059}},"memstats":{"gc_next":7948720,"memory_alloc":5142416,"memory_total":14898976},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.14,"15":0.1,"5":0.14,"norm":{"1":0.07,"15":0.05,"5":0.07}}}}}} 2020-02-12T21:37:33.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":140,"time":{"ms":12}},"total":{"ticks":240,"time":{"ms":24},"value":240},"user":{"ticks":100,"time":{"ms":12}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":270060}},"memstats":{"gc_next":7946160,"memory_alloc":4111832,"memory_total":15348288},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.08,"15":0.09,"5":0.13,"norm":{"1":0.04,"15":0.045,"5":0.065}}}}}} 2020-02-12T21:38:03.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":150,"time":{"ms":11}},"total":{"ticks":250,"time":{"ms":12},"value":250},"user":{"ticks":100,"time":{"ms":1}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":12},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":300060}},"memstats":{"gc_next":7946160,"memory_alloc":4489960,"memory_total":15726416},"runtime":{"goroutines":46}},"filebeat":{"harvester":{"open_files":6,"running":6}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.1,"15":0.09,"5":0.13,"norm":{"1":0.05,"15":0.045,"5":0.065}}}}}} 2020-02-12T21:38:08.676Z INFO log/harvester.go:276 File is inactive: /mylogs/w.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:38:08.676Z INFO log/harvester.go:276 File is inactive: /mylogs/c.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:38:08.678Z INFO log/harvester.go:276 File is inactive: /mylogs/b.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:38:08.678Z INFO log/harvester.go:276 File is inactive: /mylogs/y.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:38:13.706Z INFO log/harvester.go:251 Harvester started for file: /mylogs/y.log 2020-02-12T21:38:33.594Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":150,"time":{"ms":5}},"total":{"ticks":250,"time":{"ms":9},"value":250},"user":{"ticks":100,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":330059}},"memstats":{"gc_next":7946160,"memory_alloc":5014240,"memory_total":16250696},"runtime":{"goroutines":34}},"filebeat":{"events":{"added":5,"done":5},"harvester":{"closed":4,"open_files":3,"running":3,"started":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":5,"total":5}}},"registrar":{"states":{"current":6,"update":5},"writes":{"success":5,"total":5}},"system":{"load":{"1":0.88,"15":0.15,"5":0.31,"norm":{"1":0.44,"15":0.075,"5":0.155}}}}}} 2020-02-12T21:39:03.595Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":160,"time":{"ms":6}},"total":{"ticks":270,"time":{"ms":8},"value":270},"user":{"ticks":110,"time":{"ms":2}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":360059}},"memstats":{"gc_next":7946160,"memory_alloc":5284712,"memory_total":16521168},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":6}},"system":{"load":{"1":0.68,"15":0.16,"5":0.31,"norm":{"1":0.34,"15":0.08,"5":0.155}}}}}} 2020-02-12T21:39:03.676Z INFO log/harvester.go:276 File is inactive: /mylogs/x.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:39:33.596Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":160,"time":{"ms":5}},"total":{"ticks":270,"time":{"ms":12},"value":270},"user":{"ticks":110,"time":{"ms":7}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":390059}},"memstats":{"gc_next":7666032,"memory_alloc":3879448,"memory_total":16793464},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"closed":1,"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":6,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.48,"15":0.16,"5":0.3,"norm":{"1":0.24,"15":0.08,"5":0.15}}}}}} 2020-02-12T21:39:38.705Z INFO log/harvester.go:276 File is inactive: /mylogs/d.log. Closing because close_inactive of 5m0s reached. 2020-02-12T21:39:43.714Z INFO log/harvester.go:251 Harvester started for file: /mylogs/d.log 2020-02-12T21:39:43.715Z ERROR readjson/json.go:52 Error decoding JSON: EOF 2020-02-12T21:39:49.724Z INFO log/harvester.go:264 File was truncated. Begin reading file from offset 0: /mylogs/d.log 2020-02-12T21:39:53.720Z INFO log/harvester.go:251 Harvester started for file: /mylogs/d.log 2020-02-12T21:39:53.721Z ERROR readjson/json.go:52 Error decoding JSON: EOF 2020-02-12T21:40:03.597Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":190,"time":{"ms":30}},"total":{"ticks":320,"time":{"ms":46},"value":320},"user":{"ticks":130,"time":{"ms":16}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"d28c1982-c6bd-43b4-bfbb-c439f909b057","uptime":{"ms":420059}},"memstats":{"gc_next":7666032,"memory_alloc":4930512,"memory_total":17844528},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":8,"done":8},"harvester":{"closed":2,"open_files":2,"running":2,"started":2},"input":{"log":{"files":{"truncated":1}}}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":4,"batches":2,"total":4},"read":{"bytes":702},"write":{"bytes":2270}},"pipeline":{"clients":1,"events":{"active":0,"filtered":4,"published":4,"total":8},"queue":{"acked":4}}},"registrar":{"states":{"current":6,"update":8},"writes":{"success":6,"total":6}},"system":{"load":{"1":0.59,"15":0.17,"5":0.33,"norm":{"1":0.295,"15":0.085,"5":0.165}}}}}}
上下文:我整整一个星期都在努力挣扎,以使其运行起来:filebeat-> kafka-> logstash-> elasticsearch-kibana,每个人都在自己的docker中(您会发现另外3或4个...
问题在于,这三个容器在网络方面相互分离和/或配置错误。让我们讨论实际发生的情况以及如何解决它:
1。 Elasticsearch