如何在 logstash.conf 中将文件名动态添加到我的索引中，出现 patternNotFound 错误

Question

我的文件夹结构如下：

main_directory.
|   .env
|   docker-compose.yml
|
+---elasticsearch
|   \---config
|           elasticsearch.yml
|
\---logstash
    +---config
    |   |   logstash.yml
    |   |   pipelines.yml
    |   |   
    |   \---pipeline
    |           logstash.conf
    |
    +---data
    |   |   .lock
    |   |   dummy.csv
    |   |   test.csv
    |   |   uuid
    |   |
    |   +---dead_letter_queue
    |   \---queue
    \---logs

我的

logstash.conf

如下：

input {
  file {
    path => "/usr/share/logstash/data/*.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  csv {
    separator => ","
    # columns => ["User_Id","First_Name"]
    autodetect_column_names => true
    skip_header => true
  }
  dissect{
    mapping => {
      "path" => "/usr/share/logstash/data/#%{file_name}.csv"

    }
  }
}

output {
  elasticsearch {
    hosts => "${ELASTICSEARCH_HOST}"
    user => "${ELASTICSEARCH_USERNAME}"
    password => "${ELASTICSEARCH_PASSWORD}"
    index => "my_index_prime_%{file_name}"
    ssl => true
    ssl_certificate_verification => false
    manage_template => false
  }
}

在

/data/

目录里面有很多像

test.csv, sample.csv, work_index_one.csv

等的csv

我在过滤器中使用

dissect

以

my_index_prime_${file_name}

的格式将文件名添加到我的索引中，但我得到了

patternNotFound

错误：

[2023-05-17T19:38:16,515][WARN ][org.logstash.dissect.Dissector][main][3b3f27d0e8ab71d66a5d2b9d3b16693ce141224b733810c11fea3544cb5beef7] Dissector mapping, pattern not found {"field"=>"path", "pattern"=>"/usr/share/logstash/data/#%{file_name}.csv", "event"=>{"tags"=>["_dissectfailure"], "@version"=>"1", "message"=>"002,Asif\r", "path"=>"/usr/share/logstash/data/dummy.csv", "First_Name"=>"Asif", "@timestamp"=>2023-05-17T19:38:16.308Z, "host"=>"f47d5a099289", "User_Id"=>"002"}}

任何建议将不胜感激！

我正在尝试将文件名动态附加到我的索引