需要一些帮助来解析 Snowlake 中的数据。 HL7 特别是我看到了一些奇怪的行为 [关闭]

Question

我有这样的文件格式：

然后我从这样的阶段查询数据：

我得到了解析的文件，但它额外解析了我的 HL7 文件的片段，所以它们在不同的行结束，像这样：

图中那两个FT1段应该和上面的MSH在同一行。它上面的 MSH 有 3 个 FT1 段，但由于某种原因，第 2 段和第 3 段在它们自己的线上。

Answer 1

HL7 v2 使用管道分隔字段和换行符分隔段。一条消息由一个或（通常）多个以三字符代码为前缀的段组成。新消息总是以前缀

MSH

开头，因此我们可以使用它来组合完整的消息。为了将 HL7 v2 的许多片段组合成一条完整的消息，您可以在此处浏览简短的脚本。

虽然最后一步显示了如何获得完全组装的 HL7 v2 消息，但您可以按照脚本逐步构建它以完成您需要做的事情。

-- First, we want a file format that selects whole lines for the HL7 v2 segments.
-- This is important in case a line includes a comma.
create or replace file format SEGMENTS 
compression = 'AUTO' 
field_delimiter = 'NONE'
record_delimiter = '\n' 
skip_header = 0 
trim_space = true;


-- Now we can select from the stage (named HL7 in this example).
-- We'll need file names and line numbers later, so we'll add them now:
select   metadata$file_row_number as LINE_NUMBER
        ,metadata$filename as FILE_NAME
        ,$1 as HL7_SEGMENT
from     @HL7
(file_format => SEGMENTS)
;
-- This returns: individual HL7 segments (not messages).


-- We'll now add a conditional_true_event number to see when a new HL7 message starts:
select   metadata$file_row_number as LINE_NUMBER
        ,metadata$filename as FILE_NAME
        ,$1 as HL7_SEGMENT 
        ,conditional_true_event(left(HL7_SEGMENT, 3) = 'MSH') over (partition by FILE_NAME order by LINE_NUMBER) as MESSAGE_NUMBER
from     @HL7
(file_format => SEGMENTS)
;
-- This returns: individual HL7 segments, now with an ordered message number.

-- If we want to get a complete HL7 message assembled, we can use that ordered message
-- number from the previous example:
with ORDERED_SEGMENTS as
(
    select   metadata$file_row_number as LINE_NUMBER
            ,$1 as HL7_SEGMENT
            ,metadata$filename as FILE_NAME
            ,conditional_true_event(left(HL7_SEGMENT, 3) = 'MSH') over (partition by FILE_NAME order by LINE_NUMBER) as MESSAGE_NUMBER
    from     @HL7
    (file_format => SEGMENTS)
)
select   listagg(HL7_SEGMENT, '\\n') as HL7_MESSAGE
from     ORDERED_SEGMENTS
group by FILE_NAME, MESSAGE_NUMBER
;
-- This returns: complete HL7 messages.