当我的查询在结构数组中包含空值时,我无法在 BigQuery 中使用 SQL。这是我收到的错误:
Array cannot have a null element; error in writing field permissions.p.accts; error in writing field permissions.p; error in writing field permissions
我已经看过 ARRAY_AGG
函数的
Google 文档,建议使用
IGNORE NULLS
选项,但据我所知,这仅适用于标量值数组,不适用于对象数组(STRUCT 数组) )。我尝试了多种不同的方法来使用 ARRAY_AGG 函数,但无法让它在以下场景中工作。因此,我正在寻找一种解决方案,即使存在空值,也可以让我的查询正常工作(请参阅下面的第二个示例)。
这是一个运行良好的示例查询:
with example as (
select JSON '{"permissions":{"p":[{"accts":["abc"],"perms":["def"]}]}}' as json_data
)
select STRUCT(
ARRAY(SELECT AS STRUCT
JSON_VALUE_ARRAY(permission,'$.accts') as accts,
JSON_VALUE_ARRAY(permission,'$.perms') as perms
FROM UNNEST(
JSON_QUERY_ARRAY(example.json_data, '$.permissions.p')
) as permission
) as p
) as permissions
from example
;
现在这是一个具有空值的相同查询的示例,它会抛出上面提到的错误。此查询中唯一的区别是
"accts":["abc"]
已更改为 "accts":[null]
。
with example as (
select JSON '{"permissions":{"p":[{"accts":[null],"perms":["def"]}]}}' as json_data
)
select STRUCT(
ARRAY(SELECT AS STRUCT
JSON_VALUE_ARRAY(permission,'$.accts') as accts,
JSON_VALUE_ARRAY(permission,'$.perms') as perms
FROM UNNEST(
JSON_QUERY_ARRAY(example.json_data, '$.permissions.p')
) as permission
) as p
) as permissions
from example
;
非常感谢您的帮助!
解析JSON并获得数组后,需要对该数组进行取消嵌套和过滤以消除空条目。这可以在用户函数 UDF 或 subSelect 中完成。
Create temp function JSON_VALUE_ARRAY_no_NULL(ARR any type, str string)
as
(( SELECT X from UNNEST(JSON_VALUE_ARRAY(ARR ,str )) X WHERE X IS NOT NULL ))
;
with example as (
select JSON '{"permissions":{"p":[{"accts":["abc"],"perms":["def"]}]}}' as json_data
union all select JSON '{"permissions":{"p":[{"accts":[null],"perms":["def"]}]}}'
)
select STRUCT(
ARRAY(SELECT AS STRUCT
(( SELECT X from UNNEST(JSON_VALUE_ARRAY(permission ,'$.accts' )) X WHERE X IS NOT NULL )) as accts,
JSON_VALUE_ARRAY_no_NULL(permission ,'$.accts' ) as accts3,
JSON_VALUE_ARRAY_no_NULL(permission,'$.perms') as perms
FROM UNNEST(
JSON_QUERY_ARRAY(example.json_data, '$.permissions.p')
) as permission
) as p
) as permissions
from example