我构建了一个最新的elk（v7.3.2）集群进行测试，但是当我完成构建并在kibana中添加索引后，我从kibana日志中得到了错误：[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternyword field instead.我用谷歌搜索了一些解决方案，从站点https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html找出原因，然后按照网络上的说明将其更改为关键字，但是出现以下错误。
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [@timestamp] has unsupported parameters: [fielddata : true]"
}
],
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [@timestamp] has unsupported parameters: [fielddata : true]"
},
"status": 400
}
有人帮助
有我的文件拍配置文件：
filebeat.inputs:
- type: log
tail_files: true
paths:
- /var/log/cyk213.log
fields:
appid: appid_cyk_filebeat_config_output.kafka_cyk_213
output.kafka:
hosts:
["10.201.5.213:9092","10.201.5.213:9093","10.201.5.216:9092","10.201.5.216:9093","10.201.5.217:9092","10.201.5.217:9093"]
topic: 'topic_cyk_filebeat_config_output.kafka_cyk_213'
partition.round_robin:
reachable_only: false
required_acks: 1
compression: gzip
max_message_bytes: 1000000
和下面的elasticsearch配置：
cluster.initial_master_nodes:
- es_cluster_node01
- es_cluster_node02
cluster.name: cyk_es_cluster
node.name: es_cluster_node03
node.master: false
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0
http.port: 9200
transport.profiles.default.port: 9300
discovery.seed_hosts: ["es_cluster_node03","es_cluster_node02","es_cluster_node01"]
#discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 150s
discovery.zen.fd.ping_retries: 10
client.transport.ping_timeout: 60s
http.cors.enabled: true
http.cors.allow-origin: “*”
logger.org.elasticsearch.cluster.coordination: TRACE
还有kibana配置：
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: "http://10.201.5.217:9200"
@ DejanMarić，这是我的地图：
"appid_cyk_filebeat_config_output.kafka_cyk_213-2019.10.16" : {
"aliases" : { },
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"@version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"fields" : {
"properties" : {
"appid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"host" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1571204138225",
"number_of_shards" : "1",
"number_of_replicas" : "1",
"uuid" : "E_ulQ3NCQQaRTZDTpcO_DQ",
"version" : {
"created" : "7030299"
},
"provided_name" : "appid_cyk_filebeat_config_output.kafka_cyk_213-2019.10.16"
}
}
}
}
尝试为您的索引模式放入模板：
PUT _template/appid_cyk_filebeat_template
{
"index_patterns": [
"appid_cyk_filebeat_config_otuput.kafka*"
],
"settings": {
"index": {
"number_of_replicas": "1",
"number_of_shards": "1"
}
},
"mappings": {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"@version" : {
"type" : "keyword"
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text"
},
"hostname" : {
"type" : "text"
},
"id" : {
"type" : "text"
},
"type" : {
"type" : "text"
},
"version" : {
"type" : "text"
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text"
}
}
},
"fields" : {
"properties" : {
"appid" : {
"type" : "text"
}
}
},
"host" : {
"properties" : {
"name" : {
"type" : "text"
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text"
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text"
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text"
}
}
}
}
您现在可以将新文档添加到索引中：
PUT appid_cyk_filebeat_config_otuput.kafka/_doc/1
{
"@timestamp" : "2019-09-10"
}
{
"_index" : "appid_cyk_filebeat_config_otuput.kafka",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1
}
如果您没有通过Beats / Logstash或使用的任何方式获取它，那么您的配置就不好了。
尝试为您的索引模式放入模板：
PUT _template/appid_cyk_filebeat_template
{
"index_patterns": [
"appid_cyk_filebeat_config_otuput.kafka*"
],
"settings": {
"index": {
"number_of_replicas": "1",
"number_of_shards": "1"
}
},
"mappings": {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"@version" : {
"type" : "keyword"
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text"
},
"hostname" : {
"type" : "text"
},
"id" : {
"type" : "text"
},
"type" : {
"type" : "text"
},
"version" : {
"type" : "text"
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text"
}
}
},
"fields" : {
"properties" : {
"appid" : {
"type" : "text"
}
}
},
"host" : {
"properties" : {
"name" : {
"type" : "text"
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text"
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text"
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text"
}
}
}
}
您现在可以将新文档添加到索引中：
PUT appid_cyk_filebeat_config_otuput.kafka/_doc/1
{
"@timestamp" : "2019-09-10"
}
{
"_index" : "appid_cyk_filebeat_config_otuput.kafka",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1
}
如果您没有通过Beats / Logstash或使用的任何方式获取它，那么您的配置就不好了。