数组索引74超出范围（溢）谷歌大查询

问题描述 投票：0回答：1

在大的查询，我在移动管道问题（|）-delimited文件到谷歌大查询。为了使这项工作，我决定在每一行整体作为一个字符串带来，然后执行SPLIT函数来得到我想要的领域中，我需要它的数据类型。

在这种情况下，我有这样的SELECT声明：

Select
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(0)] as INT64) AS FIELD1
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(1)] AS FIELD2
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(2)] AS FIELD3
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(3)] AS FIELD4
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(4)] AS FIELD5
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(5)] AS FIELD6
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(6)] as timestamp) AS FIELD7
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(7)] AS FIELD8
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(8)] AS FIELD9
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(9)] AS FIELD10
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(10)] AS FIELD11
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(11)] AS FIELD12
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(12)] AS FIELD13
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(13)] AS FIELD14
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(14)] AS FIELD15
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(15)] AS FIELD16
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(16)] AS FIELD17
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(17)] AS FIELD18
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(18)] AS FIELD19
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(19)] AS FIELD20
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(20)] AS FIELD21
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(21)] AS FIELD22
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(22)] AS FIELD23
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(23)] AS FIELD24
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(24)] AS FIELD25
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(25)] AS FIELD26
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(26)] AS FIELD27
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(27)] as INT64) AS FIELD28
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(28)] AS FIELD29
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(29)] AS FIELD30
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(30)] AS FIELD31
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(31)] as timestamp) AS FIELD32
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(32)] as INT64) AS FIELD33
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(33)] as INT64) AS FIELD34
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(34)] as timestamp) AS FIELD35
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(35)] as INT64) AS FIELD36
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(36)] as INT64) AS FIELD37
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(37)] as timestamp) AS FIELD38
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(38)] as numeric) AS FIELD39
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(39)] as INT64) AS FIELD40
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(40)] as INT64) AS FIELD41
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(41)] AS FIELD42
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(42)] AS FIELD43
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(43)] as INT64) AS FIELD44
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(44)] as numeric) AS FIELD45
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(45)] AS FIELD46
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(46)] as timestamp) AS FIELD47
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(47)] AS FIELD48
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(48)] as INT64) AS FIELD49
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(49)] as timestamp) AS FIELD50
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(50)] as INT64) AS FIELD51
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(51)] as INT64) AS FIELD52
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(52)] as INT64) AS FIELD53
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(53)] as INT64) AS FIELD54
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(54)] as timestamp) AS FIELD55
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(55)] as INT64) AS FIELD56
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(56)] as timestamp) AS FIELD57
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(57)] AS FIELD58
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(58)] AS FIELD59
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(59)] AS FIELD60
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(60)] AS FIELD61
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(61)] AS FIELD62
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(62)] AS FIELD63
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(63)] as FLOAT64) AS FIELD64
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(64)] as FLOAT64) AS FIELD65
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(65)] AS FIELD66
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(66)] as INT64) AS FIELD67
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(67)] as INT64) AS FIELD68
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(68)] as INT64) AS FIELD69
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(69)] AS FIELD70
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(70)] AS FIELD71
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(71)] AS FIELD72
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(72)] as timestamp) AS FIELD73
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(73)] as timestamp) AS FIELD74
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(74)] as timestamp) AS FIELD75
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(75)] as INT64) AS FIELD76
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(76)] as INT64) AS FIELD77
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(77)] as INT64) AS FIELD78
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(78)] as timestamp) AS FIELD79
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(79)] as timestamp) AS FIELD80
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(80)] as timestamp) AS FIELD81
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(81)] as timestamp) AS FIELD82
SPLIT(RECORDSTRING, '|"||"|')[OFFSET(82)] AS FIELD83
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(83)] as FLOAT64) AS FIELD84
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(84)] as FLOAT64) AS FIELD85
safe_cast(SPLIT(RECORDSTRING, '|"||"|')[OFFSET(85)] as FLOAT64) AS FIELD86
from `db.ds.tbl`

吉贝州这个查询是结构合理。但显示这种反应：

Array index 74 is out of bounds (overflow) google big query

如何对此作出回应？谢谢

仅仅是明确的。分隔符是|"||"|。例如：

6|"||"|Demo (60 Minutes) |"||"|demo60min|"||"|2017-09-13 16:30:00|"||"|2017-09-13 15:30:00

谢谢

google-bigquery
1个回答
2
投票

数组索引74超出范围（溢）谷歌大查询 如何对此作出回应？

相反，在使用线像[OFFSET(NN)]下面的

SPLIT(RECORDSTRING, '|"||"|')[OFFSET(NN)]

你应该使用[SAFE_OFFSET(NN)]如下面例子

SPLIT(RECORDSTRING, '|"||"|')[SAFE_OFFSET(NN)]

额外的更新

正如你所看到的 - 你的查询是相当笨重，难以阅读和维护

下面是如何将查询转换成一个紧凑简单的例子

#standardSQL
SELECT 
  SAFE_CAST(item[SAFE_OFFSET(0)] AS INT64) AS FIELD1,
  item[SAFE_OFFSET(1)] AS FIELD2,
  item[SAFE_OFFSET(2)] AS FIELD3,
  item[SAFE_OFFSET(3)] AS FIELD4,
  item[SAFE_OFFSET(4)] AS FIELD5,
  item[SAFE_OFFSET(5)] AS FIELD6,
  SAFE_CAST(item[SAFE_OFFSET(6)] AS TIMESTAMP) AS FIELD7
FROM `project.dataset.table` t,
UNNEST([STRUCT<item ARRAY<STRING>>(SPLIT(RECORDSTRING, '|"||"|'))])

你可以从你的问题样本数据测试它下面的例子中

#standardSQL
WITH `project.dataset.table` AS (
  SELECT '''6|"||"|Demo (60 Minutes) |"||"|demo60min|"||"|2017-09-13 16:30:00|"||"|2017-09-13 15:30:00''' AS RECORDSTRING
)
SELECT 
  SAFE_CAST(item[SAFE_OFFSET(0)] AS INT64) AS FIELD1,
  item[SAFE_OFFSET(1)] AS FIELD2,
  item[SAFE_OFFSET(2)] AS FIELD3,
  item[SAFE_OFFSET(3)] AS FIELD4,
  item[SAFE_OFFSET(4)] AS FIELD5,
  item[SAFE_OFFSET(5)] AS FIELD6,
  SAFE_CAST(item[SAFE_OFFSET(6)] AS TIMESTAMP) AS FIELD7
FROM `project.dataset.table` t,
UNNEST([STRUCT<item ARRAY<STRING>>(SPLIT(RECORDSTRING, '|"||"|'))])

有结果

Row FIELD1  FIELD2              FIELD3      FIELD4              FIELD5                 FIELD6      FIELD7    
1   6       Demo (60 Minutes)   demo60min   2017-09-13 16:30:00 2017-09-13 15:30:00    null        null
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.