列名未在Apache Drill中的JDBC扫描中传递给PostgreSQL

Question

虽然尝试为PostgreSQL运行SQL查询，而不是表中的列名引用它将*向下推送到数据库。

select
  m.id,
  cnt_c_no
from (
    select
      m_id
    from pg_test_main.test1.table1
    where
      last_date >= '2019-01-01 00:00:00'
  ) as m
left join (
    select
      ci.m_id,
      count(ci.c_no) as cnt_c_no
    from (
        select
          m_id,
          c_no
        from pg_test.test2.table2
      ) as ci
    inner join (
        select
          c_no
        from pg_test.test2.table3
        where
          is_del = 'F'
      ) as c on ci.c_no = c.c_no
    group by
      ci.m_id
  ) as join1 on m.m_id = join1.m_id;

00-00    Screen
00-01      Project(m_id=[$0], cnt_c_no=[$1])
00-02        Project(m_id=[$0], cnt_c_no=[$2])
00-03          HashJoin(condition=[=($0, $1)], joinType=[left], semi-join: =[false])
00-05            Jdbc(sql=[SELECT "m_id" FROM "test1"."table1" WHERE "last_date" >= '2019-01-01 00:00:00' ])
00-04            Project(m_id0=[$0], cnt_c_no=[$1])
00-06              HashAgg(group=[{0}], cnt_c_no=[COUNT($1)])
00-07                Project(m_id=[$0], c_no=[$1])
00-08                  HashJoin(condition=[=($1, $2)], joinType=[inner], semi-join: =[false])
00-10                    Project(m_id=[$3], c_no=[$1])
00-12                      Jdbc(sql=[SELECT * FROM "public"."table2" ])
00-09                    Project(c_no0=[$0])
00-11                      Project(c_no=[$0])
00-13                        SelectionVectorRemover
00-14                          Filter(condition=[=($60, 'F')])
00-15                            Jdbc(sql=[SELECT * FROM "public"."table3" ])

您可以看到，Jdbc Scan for table1使用的是列名。

但是Jdbc Scan for table2和table3没有使用列名。它把*下推到数据库。

如何控制jdbc扫描，以便它可以下推colume名称？

Apache Drill版本为1.16.0

Answer 1

我尝试在Drill 1.17和Drill 1.15上都使用MySQL来重现它，但是对于查询，类似于您指定的查询，所有查询都被推送到JDBC存储中：

SELECT m.person_id,
       cnt_c_no
FROM
  (SELECT person_id
   FROM mysql.`drill_mysql_test1`.person1
   WHERE date_field >= '2019-01-01 00:00:00') AS m
LEFT JOIN
  (SELECT ci.person_id,
          count(ci.last_name) AS cnt_c_no
   FROM
     (SELECT person_id,
             last_name
      FROM mysql.`drill_mysql_test`.person) AS ci
   INNER JOIN
     (SELECT last_name
      FROM mysql.`drill_mysql_test`.person2
      WHERE boolean_field = 'F' ) AS c ON ci.last_name = c.last_name
   GROUP BY ci.person_id) AS join1 ON m.person_id = join1.person_id

此查询的计划：

00-00    Screen
00-01      Project(person_id=[$0], cnt_c_no=[$1])
00-02        Jdbc(sql=[SELECT `t0`.`person_id`, `t5`.`cnt_c_no` FROM (SELECT `person_id` FROM `drill_mysql_test1`.`person1` WHERE `date_field` >= '2019-01-01 00:00:00') AS `t0` LEFT JOIN (SELECT `t1`.`person_id`, COUNT(`t1`.`last_name`) AS `cnt_c_no` FROM (SELECT `person_id`, `last_name` FROM `drill_mysql_test`.`person`) AS `t1` INNER JOIN (SELECT `last_name` FROM `drill_mysql_test`.`person2` WHERE `boolean_field` = 'F') AS `t3` ON `t1`.`last_name` = `t3`.`last_name` GROUP BY `t1`.`person_id`) AS `t5` ON `t0`.`person_id` = `t5`.`person_id` ])

您能否为Postgres表提供CTAS，所以我将尝试再次使用特定的数据类型来重现它。另外，如果可能，请检查是否在Drill 1.17上仍然复制了此问题。