将自定义弹性beanstalk日志获取到cloudwatch组

Question

遇到问题并且无法从 .ebextentions/logs.config 创建自定义 cloudwatch 日志组

这是我尝试过的不同文件。

1

--- 
files: 
  /opt/elasticbeanstalk/tasks/bundlelogs.d/celery_logs.conf: 
    content: |-
        /var/log/celery_beat.stdout.log
        /var/log/celery_flower.stdout.log
        /var/log/celery_worker.stdout.log
        /var/log/faust_worker.stdout.log
    group: root
    mode: "000755"
    owner: root

2

--- 
files: 
  "/opt/elasticbeanstalk/config/private/logtasks/bundle/applogs.conf" :
    mode: "000755"
    owner: root
    group: root
    content: |
      /var/log/celery_beat.stdout.log
      /var/log/celery_flower.stdout.log
      /var/log/celery_worker.stdout.log
      /var/log/faust_worker.stdout.log

3

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/celery_beat.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_beat.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_beat.stdout.log

      [/var/log/celery_flower.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_flower.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_flower.stdout.log

      [/var/log/celery_worker.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_worker.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_worker.stdout.log

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

日志正确显示在文件中：

/var/log/celery_beat.stdout.log
/var/log/celery_flower.stdout.log
/var/log/celery_worker.stdout.log
/var/log/faust_worker.stdout.log

但是没有创建日志组，也没有日志传输到其中。

我尝试过 15 种或更多其他类似的配置，但没有成功。

Answer 1

有人留下了答案，但它被删除了，所以我会继续，把它留在这里给大家。

我认为 OP 和我无法让它工作的原因是因为我们遵循本教程走上了错误的道路：

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html

在本教程中，他们教您使用

conf

中的

/opt/elasticbeanstalk/tasks/

文件，但这仅用于将日志导出到 Elastic Beanstalk Web 控制台，而不是导出到 CloudWatch 日志。

要将自定义日志导出到 CloudWatch 日志中，您必须使用本教程（这是OP问题中的选项＃3）

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html

它特别有一个“自定义日志文件流”部分——这是通过使用

.ebextensions

安装名为

awslogs

的包并自行配置来完成的。这个过程比简单地放入

conf

文件要复杂得多，但这就是必须完成的方法。

还请务必检查 Elastic Beanstalk 角色，以确保它具有与 CloudWatch 日志交互的权限。

Answer 2

最新的 EC 2 不再使用 awslogs。通过 SSH 连接到您的实例，并通过键入

检查您的实例是否正在使用最新的 cloudwatch 代理

sudo amazon-cloudwatch-agent-ctl -a status

如果它正在运行，请将以下配置文件添加到您的 ebextensions 中。

container_commands:   01_append_logs_config:
    command: |
      amazon-cloudwatch-agent-ctl -a append-config -m ec2 -c file:/tmp/custom-cloudwatch-config.json -s

files:   "/tmp/custom-cloudwatch-config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "logs": {
          "logs_collected": {
            "files": {
              "collect_list": [
                {
                  "file_path": "/var/log/messages",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/messages"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cfn-init.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cfn-init.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cfn-init-cmd.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cfn-init-cmd.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/dnf.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/dnf.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/eb-cfn-init.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/eb-cfn-init.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/celery_beat.stdout.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/celery_beat.stdout.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/celery_worker.stdout.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/celery_worker.stdout.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/secure",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/secure"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cron",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cron"]]}`",
                  "log_stream_name": "{instance_id}"
                }
              ]
            }
          }
        }
      }

检查 https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html 以附加将添加自定义日志文件的自定义配置文件。

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html中解释了配置文件生成。您可以通过运行向导来生成适合您情况的配置文件，并根据结果更改下面文件的内容。

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/UseCloudWatchUnifiedAgent.html