Postgres:强制分析器使用位图扫描而不是索引扫描

问题描述 投票:1回答:1

在这种情况下,表有许多文本列,我需要在每列上执行(ilike)搜索。我继续为每个列创建一个杜松子酒索引(来自扩展名pg_trgm的gin_trgm_ops)以加速搜索,实际上增益很大。这些是常规索引的补充(一些查询使用简单的相等条件)。

但是,测试显示,如果太多条件被OR联系在一起,Postgres就无法选择正确的计划。

查询正确的计划:

db=# explain analyze select topics_id from t_topics_header a
join t_topics_group b using (topics_group_id) 
where a.subject ilike '%aaa%' or a.contents ilike '%aaa%' or a.topics_group_id = 7 and
( ( a.ext_col_01 ilike '%aaa%') or ( a.ext_col_50 ilike '%aaa%') or 
  ( a.ext_col_57 ilike '%aaa%') or ( a.ext_col_56 ilike '%aaa%') or 
  ( a.ext_col_63 ilike '%aaa%') or ( a.ext_col_64 ilike '%aaa%') or  
  ( a.ext_col_54 ilike '%aaa%') or ( a.ext_col_69 ilike '%aaa%') or  
  ( a.ext_col_31 ilike '%aaa%') or ( a.ext_col_32 ilike '%aaa%') or  
  ( a.ext_col_41 ilike '%aaa%') or ( a.ext_col_91 ilike '%aaa%') or  
  ( a.ext_col_42 ilike '%aaa%') or ( a.ext_col_92 ilike '%aaa%') or  
  ( a.ext_col_43 ilike '%aaa%') or ( a.ext_col_93 ilike '%aaa%') or  
  ( a.ext_col_44 ilike '%aaa%') or ( a.ext_col_94 ilike '%aaa%') or  
  ( a.ext_col_45 ilike '%aaa%') or ( a.ext_col_95 ilike '%aaa%')  
) order by topics_id desc OFFSET 0 LIMIT 10;

查询计划:

 Limit  (cost=2739.84..2739.87 rows=10 width=4) (actual time=0.437..0.437 rows=4 loops=1)
   ->  Sort  (cost=2739.84..2741.03 rows=473 width=4) (actual time=0.436..0.436 rows=4 loops=1)
         Sort Key: a.topics_id DESC
         Sort Method: quicksort  Memory: 25kB
         ->  Hash Join  (cost=708.03..2729.62 rows=473 width=4) (actual time=0.416..0.426 rows=4 loops=1)
               Hash Cond: (a.topics_group_id = b.topics_group_id)
               ->  Bitmap Heap Scan on t_topics_header a  (cost=706.72..2721.80 rows=473 width=8) (actual time=0.381..0.391 rows=4 loops=1)
                     Recheck Cond: ((subject ~~* '%aaa%'::text) OR (contents ~~* '%aaa%'::text) OR ((ext_col_01 ~~* '%aaa%'::text) OR (ext_col_50 ~~* '%aaa%'::text) OR (ext_col_57 ~~* '%aaa%'::text) OR (ext_col_56 ~~* '%aaa%'::text) OR 
(ext_col_63 ~~* '%aaa%'::text) OR (ext_col_64 ~~* '%aaa%'::text) OR (ext_col_54 ~~* '%aaa%'::text) OR (ext_col_69 ~~* '%aaa%'::text) OR (ext_col_31 ~~* '%aaa%'::text) OR (ext_col_32 ~~* '%aaa%'::text) OR (ext_col_41 ~~* '%aaa%'::text) O
R (ext_col_91 ~~* '%aaa%'::text) OR (ext_col_42 ~~* '%aaa%'::text) OR (ext_col_92 ~~* '%aaa%'::text) OR (ext_col_43 ~~* '%aaa%'::text) OR (ext_col_93 ~~* '%aaa%'::text) OR (ext_col_44 ~~* '%aaa%'::text) OR (ext_col_94 ~~* '%aaa%'::text)
 OR (ext_col_45 ~~* '%aaa%'::text) OR (ext_col_95 ~~* '%aaa%'::text)))
                     Filter: ((subject ~~* '%aaa%'::text) OR (contents ~~* '%aaa%'::text) OR ((topics_group_id = 7) AND ((ext_col_01 ~~* '%aaa%'::text) OR (ext_col_50 ~~* '%aaa%'::text) OR (ext_col_57 ~~* '%aaa%'::text) OR (ext_col_56 ~
~* '%aaa%'::text) OR (ext_col_63 ~~* '%aaa%'::text) OR (ext_col_64 ~~* '%aaa%'::text) OR (ext_col_54 ~~* '%aaa%'::text) OR (ext_col_69 ~~* '%aaa%'::text) OR (ext_col_31 ~~* '%aaa%'::text) OR (ext_col_32 ~~* '%aaa%'::text) OR (ext_col_41
 ~~* '%aaa%'::text) OR (ext_col_91 ~~* '%aaa%'::text) OR (ext_col_42 ~~* '%aaa%'::text) OR (ext_col_92 ~~* '%aaa%'::text) OR (ext_col_43 ~~* '%aaa%'::text) OR (ext_col_93 ~~* '%aaa%'::text) OR (ext_col_44 ~~* '%aaa%'::text) OR (ext_col_
94 ~~* '%aaa%'::text) OR (ext_col_45 ~~* '%aaa%'::text) OR (ext_col_95 ~~* '%aaa%'::text))))
                     Heap Blocks: exact=4
                     ->  BitmapOr  (cost=706.72..706.72 rows=516 width=0) (actual time=0.375..0.375 rows=0 loops=1)
                           ->  Bitmap Index Scan on t_topics_header_idx_subject_gin_trgm  (cost=0.00..12.91 rows=122 width=0) (actual time=0.066..0.066 rows=4 loops=1)
                                 Index Cond: (subject ~~* '%aaa%'::text)
                           ->  Bitmap Index Scan on t_topics_header_idx_contents_gin_trgm  (cost=0.00..8.00 rows=1 width=0) (actual time=0.016..0.016 rows=0 loops=1)
                                 Index Cond: (contents ~~* '%aaa%'::text)
                           ->  BitmapOr  (cost=685.32..685.32 rows=394 width=0) (actual time=0.292..0.292 rows=0 loops=1)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_01_gin_trgm  (cost=0.00..8.85 rows=113 width=0) (actual time=0.014..0.014 rows=0 loops=1)
                                   Index Cond: (ext_col_01 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_50_gin_trgm  (cost=0.00..12.06 rows=8 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                   Index Cond: (ext_col_50 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_57_gin_trgm  (cost=0.00..12.12 rows=16 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                   Index Cond: (ext_col_57 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_56_gin_trgm  (cost=0.00..80.00 rows=1 width=0) (actual time=0.023..0.023 rows=0 loops=1)
                                   Index Cond: (ext_col_56 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_63_gin_trgm  (cost=0.00..84.05 rows=7 width=0) (actual time=0.020..0.020 rows=0 loops=1)
                                   Index Cond: (ext_col_63 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_64_gin_trgm  (cost=0.00..72.04 rows=6 width=0) (actual time=0.020..0.020 rows=0 loops=1)
                                   Index Cond: (ext_col_64 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_54_gin_trgm  (cost=0.00..68.00 rows=1 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                   Index Cond: (ext_col_54 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_69_gin_trgm  (cost=0.00..12.02 rows=3 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                   Index Cond: (ext_col_69 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_31_gin_trgm  (cost=0.00..12.04 rows=6 width=0) (actual time=0.021..0.021 rows=0 loops=1)
                                   Index Cond: (ext_col_31 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_32_gin_trgm  (cost=0.00..16.76 rows=101 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                   Index Cond: (ext_col_32 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_41_gin_trgm  (cost=0.00..36.00 rows=1 width=0) (actual time=0.021..0.021 rows=0 loops=1)
                                   Index Cond: (ext_col_41 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_91_gin_trgm  (cost=0.00..20.58 rows=78 width=0) (actual time=0.020..0.020 rows=0 loops=1)
                                   Index Cond: (ext_col_91 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_42_gin_trgm  (cost=0.00..36.00 rows=1 width=0) (actual time=0.005..0.005 rows=0 loops=1)
                                   Index Cond: (ext_col_42 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_92_gin_trgm  (cost=0.00..12.19 rows=25 width=0) (actual time=0.008..0.008 rows=0 loops=1)
                                   Index Cond: (ext_col_92 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_43_gin_trgm  (cost=0.00..36.00 rows=1 width=0) (actual time=0.004..0.004 rows=0 loops=1)
                                   Index Cond: (ext_col_43 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_93_gin_trgm  (cost=0.00..8.20 rows=26 width=0) (actual time=0.006..0.006 rows=0 loops=1)
                                   Index Cond: (ext_col_93 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_44_gin_trgm  (cost=0.00..36.00 rows=1 width=0) (actual time=0.004..0.004 rows=0 loops=1)
                                   Index Cond: (ext_col_44 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_94_gin_trgm  (cost=0.00..8.04 rows=5 width=0) (actual time=0.006..0.006 rows=0 loops=1)
                                   Index Cond: (ext_col_94 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_45_gin_trgm  (cost=0.00..36.00 rows=1 width=0) (actual time=0.004..0.004 rows=0 loops=1)
                                   Index Cond: (ext_col_45 ~~* '%aaa%'::text)
                             ->  Bitmap Index Scan on t_topics_header_idx_ext_col_95_gin_trgm  (cost=0.00..76.00 rows=1 width=0) (actual time=0.004..0.004 rows=0 loops=1)
                                   Index Cond: (ext_col_95 ~~* '%aaa%'::text)
           ->  Hash  (cost=1.14..1.14 rows=14 width=4) (actual time=0.027..0.027 rows=14 loops=1)
                 Buckets: 1024  Batches: 1  Memory Usage: 9kB
                 ->  Seq Scan on t_topics_group b  (cost=0.00..1.14 rows=14 width=4) (actual time=0.013..0.019 rows=14 loops=1)
 Planning time: 3.767 ms
 Execution time: 0.687 ms

用坏计划查询:

db=# explain analyze select topics_id from t_topics_header a
join t_topics_group b using (topics_group_id) 
where a.subject ilike '%aaa%' or a.contents ilike '%aaa%' or a.topics_group_id = 7 and
( ( a.ext_col_01 ilike '%aaa%') or ( a.ext_col_50 ilike '%aaa%') or 
  ( a.ext_col_57 ilike '%aaa%') or ( a.ext_col_56 ilike '%aaa%') or 
  ( a.ext_col_63 ilike '%aaa%') or ( a.ext_col_64 ilike '%aaa%') or  
  ( a.ext_col_54 ilike '%aaa%') or ( a.ext_col_69 ilike '%aaa%') or  
  ( a.ext_col_31 ilike '%aaa%') or ( a.ext_col_32 ilike '%aaa%') or  
  ( a.ext_col_41 ilike '%aaa%') or ( a.ext_col_91 ilike '%aaa%') or  
  ( a.ext_col_42 ilike '%aaa%') or ( a.ext_col_92 ilike '%aaa%') or  
  ( a.ext_col_43 ilike '%aaa%') or ( a.ext_col_93 ilike '%aaa%') or  
  ( a.ext_col_44 ilike '%aaa%') or ( a.ext_col_94 ilike '%aaa%') or  
  ( a.ext_col_45 ilike '%aaa%') or ( a.ext_col_95 ilike '%aaa%') or
  ( a.ext_col_70 ilike '%aaa%')
) order by topics_id desc OFFSET 0 LIMIT 10;

请注意,唯一的区别是增加了( a.ext_col_70 ilike '%aaa%')

并且新的查询计划变为:

 Limit  (cost=0.43..2036.67 rows=10 width=4) (actual time=626.343..2784.151 rows=4 loops=1)
   ->  Nested Loop  (cost=0.43..418447.61 rows=2055 width=4) (actual time=626.341..2784.147 rows=4 loops=1)
         Join Filter: (a.topics_group_id = b.topics_group_id)
         Rows Removed by Join Filter: 52
         ->  Index Scan Backward using t_report_master_pkey on t_topics_header a  (cost=0.43..418014.89 rows=2055 width=8) (actual time=626.328..2784.119 rows=4 loops=1)
               Filter: ((subject ~~* '%aaa%'::text) OR (contents ~~* '%aaa%'::text) OR ((topics_group_id = 7) AND ((ext_col_01 ~~* '%aaa%'::text) OR (ext_col_50 ~~* '%aaa%'::text) OR (ext_col_57 ~~* '%aaa%'::text) OR (ext_col_56 ~~* '%a
aa%'::text) OR (ext_col_63 ~~* '%aaa%'::text) OR (ext_col_64 ~~* '%aaa%'::text) OR (ext_col_54 ~~* '%aaa%'::text) OR (ext_col_69 ~~* '%aaa%'::text) OR (ext_col_31 ~~* '%aaa%'::text) OR (ext_col_32 ~~* '%aaa%'::text) OR (ext_col_41 ~~* '
%aaa%'::text) OR (ext_col_91 ~~* '%aaa%'::text) OR (ext_col_42 ~~* '%aaa%'::text) OR (ext_col_92 ~~* '%aaa%'::text) OR (ext_col_43 ~~* '%aaa%'::text) OR (ext_col_93 ~~* '%aaa%'::text) OR (ext_col_44 ~~* '%aaa%'::text) OR (ext_col_94 ~~*
 '%aaa%'::text) OR (ext_col_45 ~~* '%aaa%'::text) OR (ext_col_95 ~~* '%aaa%'::text) OR (ext_col_70 ~~* '%aaa%'::text))))
               Rows Removed by Filter: 1237807
         ->  Materialize  (cost=0.00..1.21 rows=14 width=4) (actual time=0.002..0.003 rows=14 loops=4)
               ->  Seq Scan on t_topics_group b  (cost=0.00..1.14 rows=14 width=4) (actual time=0.004..0.007 rows=14 loops=1)
 Planning time: 3.704 ms
 Execution time: 2784.262 ms

我有两倍于ext_cols的显示量(我把它们减少到计划改变的最小断点)。我在每个ext_col上都有gin pg_trgm索引,除了它们之外的常规索引(双重检查)。我也在两个表上运行VACUUM ANALYZE。

DB版本是PostgreSQL 9.6.6

那么有没有办法提示使用位图索引到分析器?有任何想法吗 ?

编辑:显然字符串的长度很重要。使用3个字母或更少的字母,分析器选择(坏)索引扫描。使用4个或更多字符串,选择好(位图索引)计划。

postgresql performance
1个回答
2
投票

如果您坚持使用此设计,则可以在所有列的连接值上创建索引,并在where条件中使用该表达式。

就像是:

create function query_columns(p_rec t_topics_header)
  returns text
as
$$
  select concat_ws(' ', p_rec.ext_col_01, p_rec.ext_col_50, p_rec.ext_col_57, p_rec.ext_col_56);
$$
language sql
immutable;

concat_ws()函数内的列数调整为要查询的列数。我不确定GIN索引的索引表达式的最大长度是多少。也许你的很多专栏都会超过这个限制。

然后,您可以在该函数上创建索引:

create index on t_topics_header using gin ( (query_columns(x)) public.gin_trgm_ops);

那么以下应该使用索引:

select topics_id 
from t_topics_header a
  join t_topics_group b using (topics_group_id) 
where (a.subject ilike '%aaa%' or a.contents ilike '%aaa%' 
   or a.topics_group_id = 7 )
  and query_columns(a) ilike ='%aaa%';
© www.soinside.com 2019 - 2024. All rights reserved.