我有 json 格式的 CloudWatch 日志,其条目类似于以下内容:
{
"message": "resource_liked",
"context": {
"date": {
"date": "2021-05-07 16:52:11.000000",
"timezone_type": 3,
"timezone": "UTC"
},
...
我正在尝试编写一个 CloudWatch Insights 查询来制作一个简单的直方图:每小时日志中的事件数。
但是,我无法使用日志条目的
@timestamp
属性。我需要在条目的消息正文中使用 context.date.date
。
使用
@timestamp
编写此查询非常简单:
stats count(*) by datefloor(@timestamp, 1h)
但是,我不确定如何使用消息的
context.date.date
来代替。
我认为我需要将看起来像
2021-05-07 16:52:11.000000
的日期时间格式化为 aws 理解的日期时间,但我找不到如何操作。
stats count(*) by datefloor(context.date.date, 1h)
->“无效日期”
stats count(*) by datefloor(toMillis(context.date.date), 1h)
->“无效日期”
stats count(*) by datefloor(substr(context.date.date, 0, 19), 1h)
->“无效日期”
stats count(*) by datefloor(concat(replace(substr(context.date.date, 0, 23), ' ', 'T'), '-00:00'), 1h)
-> 日期无效。这使得该字段看起来与 @timestamp 的显示方式完全相同。
| parse @message '"date": "*:' as hour
| stats count() as cnt by hour
可能有助于将字符串日期时间转换为数值毫秒,请注意,由于闰年计算,最大年份为 2100。
fields "2021-05-07 16:52:11.000000" as reqDateTime
| parse reqDateTime "*-*-* *:*:*.*" as reqYear, reqM, reqD, reqH, reqMin, reqSec, reqMilliSec
| fields reqYear - 1970 as reqYearDiff, reqYear % 4 == 0 as reqIsLeapYear, reqM/1 as reqMonth, reqD/1 as reqDay, reqH/1 as reqHour, reqMin/1 as reqMinute, reqSec/1 as reqSecond, reqMilliSec/1 as reqMilliSecond
| fields ((reqYearDiff * 365) + ((reqYear % 4 == 1) * 1) + floor(reqYearDiff / 4) # as yearsToDays
+ ((reqMonth == 2) * 31) #
+ ((reqMonth == 3) * 59) #
+ ((reqMonth == 4) * 90) #
+ ((reqMonth == 5) * 120) #
+ ((reqMonth == 6) * 151) #
+ ((reqMonth == 7) * 181) #
+ ((reqMonth == 8) * 212) #
+ ((reqMonth == 9) * 243) #
+ ((reqMonth == 10) * 273) #
+ ((reqMonth == 11) * 304) #
+ ((reqMonth == 12) * 334) #
+ ((reqMonth > 2) and (reqIsLeapYear == 1)) # as monthsToDays
+ reqDay - 1) * 24 * 60 * 60 * 1000 # as daysToMilliSeconds
+ reqHour * 60 * 60 * 1000 # as hoursToMilliSeconds
+ reqMinute * 60 * 1000 # as minutesToMilliSeconds
+ reqSecond * 1000 # as secondsToMilliSeconds
+ reqMilliSecond
as reqMilliSeconds
| display reqMilliSeconds, fromMillis(reqMilliSeconds), reqYear, reqMonth, reqDay, reqHour, reqMinute, reqSecond, reqMilliSecond
| limit 1
我非常喜欢@amacrobert提出的
datefloor
方法,所以我尝试了一下,它确实可以通过调整来工作
fields @timestamp, @message, @logStream, @log
| filter @message like /Whatever/
| sort @timestamp desc
| stats count() as log_count by datefloor(@timestamp, 1h) as date_to_hour
| display date_to_hour, log_count