如何在 Google Big Query 中循环执行异常处理？

Question

我正在尝试通过使用 EXPORT DATA OPTIONS 命令将数据导出到 GCS，对各种数据集中存在的多个表进行每月备份。这些表名称存在于以下架构中的另一个表中，该表将由某人维护

| Sr.No | Table_Name                            | Date_filter   | Timeline |
|-------|---------------------------------------|---------------|----------|
| 1     | gcs_project_name.dataset_name.table1  | inserted_date | Monthly  |
| 2     | gcs_project_name.dataset_name2.table2 | inserted_date | Monthly  |

我用过程语言编写了一个脚本来制作一组此类查询字符串，该数组将基于 Date_filter 一次执行一个表的导出。下面提到了这样做的脚本

DECLARE folder_name DEFAULT CAST(DATE(CURRENT_DATE()) AS STRING);

DECLARE i INT64 DEFAULT 0;

DECLARE dynamic_query ARRAY <STRING>;

SET dynamic_query = ([![SELECT ARRAY_AGG(CONCAT("EXPORT DATA OPTIONS (uri = 'gs://data_backup/",folder_name,'/',Table_Name,"_*',\n format = 'CSV',","\n overwrite = TRUE,","\n header = TRUE,","\n field_delimiter = ',') AS","\n\n SELECT * FROM \n","   `",Table_Name,"`\n","WHERE \n DATE(",Date_filter,") BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 1 month), month) AND LAST_DAY(DATE_SUB(CURRENT_DATE(), INTERVAL 1 month), month)","\n LIMIT \n 1000000",";","\n")) FROM `google_sheets.backup_tables`][1]][1] );

LOOP
  SET i = i + 1;
  IF i > ARRAY_LENGTH(dynamic_query) THEN 
    SET boolean = FALSE;
    LEAVE; 
  END IF;
  EXECUTE IMMEDIATE dynamic_query[ORDINAL(i)];
END LOOP; 

SELECT dynamic_query;

dynamic_query的输出如下所述

现在我正在迭代这个数组并使用 EXECUTE IMMEDIATE

逐一执行查询

我面临的问题是 Date_filter 中提到的字段有时是 STRING 数据类型。因此，脚本失败并且不会继续执行其余表。为了解决这个问题，我想执行异常处理，其中执行不会停止，并且我能够看到已导出和失败的表的列表，就像 python 的 try-catch 块一样。

Answer 1

我弄清楚了如何在 Loop 命令中执行异常处理。 Loop 命令需要在另一个无限循环中执行，只有在遍历整个数组时才终止循环。这是参考代码

DECLARE folder_name DEFAULT CAST(DATE(CURRENT_DATE()) AS STRING);

DECLARE i INT64 DEFAULT 0;

DECLARE dummy STRING DEFAULT NULL;

DECLARE boolean DEFAULT TRUE;

DECLARE dynamic_query ARRAY <STRING>;

SET dynamic_query = ([![SELECT ARRAY_AGG(CONCAT("EXPORT DATA OPTIONS (uri = 'gs://data_backup/",folder_name,'/',Table_Name,"_*',\n format = 'CSV',","\n overwrite = TRUE,","\n header = TRUE,","\n field_delimiter = ',') AS","\n\n SELECT * FROM \n","   `",Table_Name,"`\n","WHERE \n DATE(",Date_filter,") BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 1 month), month) AND LAST_DAY(DATE_SUB(CURRENT_DATE(), INTERVAL 1 month), month)","\n LIMIT \n 1000000",";","\n")) FROM `google_sheets.backup_tables`][1]][1] );

WHILE boolean DO  
  LOOP
    BEGIN
    SET i = i + 1;
    IF i > ARRAY_LENGTH(dynamic_query) THEN 
      SET boolean = FALSE;
      LEAVE; 
    END IF;
    EXECUTE IMMEDIATE dynamic_query[ORDINAL(i)];
    
    EXCEPTION WHEN ERROR THEN
    
    SET dummy = @@error.message;
    
    END;
    
    CONTINUE;
  
  END LOOP; 
  
END WHILE;