AWS CloudWatch的LogGroup从未发送日志与λ尽管SubscriptionFilter

问题描述 投票:0回答:1

目的:建立Cloudformation堆栈,需要一个日志组名作为参数,每当新的日志,这些日志日志组中露面,他们会发送到lambda函数进行处理,然后室壁运动流水,然后把这些日志文件所谓foobarbaz桶。

问题:lambda函数永远不会得到调用(拉姆达的CloudWatch的日志显示,它从来没有得到即使在新的数据写入日志组触发)。调用应该归功于自动发生到我成立了SubscriptionFilter资源。我没有看到任何错误。无论是发生似乎默默地失败。

注:在SubscriptionFilter的FilterPattern已被设置为空字符串。我有此意是从日志组发送的所有日志lambda函数。

这里是我的Cloudformation模板:

Parameters:
  LogGroupName:
    Type: String
    Description: The name of the log group who's logs we want to send to send to Lambda->Kinesis->S3

  AuditTrailPrefix:
    Type: String
    Description: Log files will be sent to the Logging account S3 bucket with this prefix in the bucket path

Resources:  
  AuditTrailFunctionPermissions:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref AuditTrailFunction
      Principal: logs.amazonaws.com
      SourceAccount: !Ref AWS::AccountId

  AuditTrailFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role: !GetAtt AuditTrailFunctionRole.Arn
      Code:
        ZipFile: >
          // do some stuff with the data and PUT it to KinesisFirehose
          // removed for brevity
      Runtime: nodejs8.10
      Timeout: 30

  AuditTrailFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
        Version: '2012-10-17'
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyDocument:
            Statement:
              - Action:
                  - firehose:PutRecord
                  - firehose:PutRecordBatch
                Effect: Allow
                Resource: !Sub arn:aws:firehose:${AWS::Region}:${AWS::AccountId}:deliverystream/${AuditTrailDeliveryStream}
            Version: '2012-10-17'
          PolicyName: root

  AuditTrailSubscription:
    Type: AWS::Logs::SubscriptionFilter
    DependsOn: AuditTrailFunctionPermissions
    Properties:
      DestinationArn: !GetAtt AuditTrailFunction.Arn
      FilterPattern: ''
      LogGroupName: !Ref LogGroupName

  AuditTrailDeliveryStream:
    Type: AWS::KinesisFirehose::DeliveryStream
    Properties:
      DeliveryStreamType: DirectPut
      S3DestinationConfiguration:
        BucketARN: arn:aws:s3:::foobarbaz
        BufferingHints:
          IntervalInSeconds: 60
          SizeInMBs: 50
        CompressionFormat: GZIP
        Prefix: !Ref AuditTrailPrefix
        RoleARN: !GetAtt DeliveryRole.Arn

  DeliveryRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          Effect: Allow
          Principal:
            Service: firehose.amazonaws.com
          Action: sts:AssumeRole

      Policies:
        - PolicyName: firehose_delivery_policy
          PolicyDocument:
            Statement:
              Effect: Allow
              Action:
                - s3:AbortMultipartUpload
                - s3:GetBucketLocation
                - s3:GetObject
                - s3:ListBucket
                - s3:ListBucketMultipartUploads
                - s3:PutObject
              Resource:
                - arn:aws:s3:::foobarbaz
                - arn:aws:s3:::foobarbaz/${AuditTrailPrefix}*
aws-lambda amazon-cloudformation amazon-cloudwatch amazon-kinesis amazon-kinesis-firehose
1个回答
1
投票

我看不出什么错,但这里有一些技巧来解决:

  1. 当新的日志中获取的上传到您的日志组lambda函数只会被调用。它不会被调用因为这是已经在日志组中设置了订阅过滤器之前的数据。
  2. 如果第1条是不是这种情况(即你有新的数据越来越上传),去的CloudWatch - >指标和搜索日志组名称。你应该找到4个与您的订阅过滤器指标:ForwardedBytes,ForwardedLogEvents,DeliveryErrors,DeliveryThrottling。见this的描述。如果DeliveryErrors或DeliveryThrottling指标> 0,则有问题。
  3. 与DeliveryErrors最有可能的问题是权限问题。我看不出什么错你的,但是这就是我会仔细首先检查。
  4. 您可以使用AWS CLI手动调试预订建立。 (见this)。这可以帮助你找出一块设置的可能会造成问题。
© www.soinside.com 2019 - 2024. All rights reserved.