我需要一些帮助来提高查询的速度。
我有3张表：1- pairTable2 一张4列表：- genomic_accession：分组列（不关心这个问题）- assembly：分组列（不关心这个问题）- product_accession：用于在其他表中搜索的列- tmpcol：用于在其他表中搜索的列- 2- SBPDB 一张1列表：- product_accession：用于在其他表中搜索的列- 3- cacheDB 一张1列表：- product_accession：用于在其他表中搜索的列。
我们的想法是在表1中建立一个名为SBP的布尔列，它是指 TRUE 如果列的值 product_accession 或 tmpcol 中唯一的一列内。SBPDB;并且，在表1中创建一个名为SBP的布尔值列，它是。TRUE 如果列的值 product_accession 或 tmpcol 中唯一的一列内。cacheDB.
我使用R与 DBI 和 dplyr 作为后端，那么查询可能看起来很奇怪.但是，我试图做的查询是。
SELECT "genomic_accession",
"assembly",
"product_accession",
"tmpcol",
"product_accession" IN (SELECT product_accession
FROM "cachedb")
OR "tmpcol" IN (SELECT product_accession
FROM "cachedb") AS "CACHE",
"product_accession" IN (SELECT product_accession
FROM "sbpdb")
OR "tmpcol" IN (SELECT product_accession
FROM "sbpdb") AS "SBP"
FROM (SELECT *
FROM "pairtable2"
LIMIT 500000) "dbplyr_031";
(查看解释)
QUERY PLAN
----------------------------------------------------------------------------------------
Subquery Scan on dbplyr_031 (cost=3242.27..3846856408.45 rows=500000 width=59)
-> Limit (cost=0.00..10666.17 rows=500000 width=57)
-> Seq Scan on "pairTable2" (cost=0.00..781515.16 rows=36635216 width=57)
SubPlan 1
-> Seq Scan on "cacheDB" (cost=0.00..1394.91 rows=90491 width=14)
SubPlan 2
-> Seq Scan on "cacheDB" "cacheDB_1" (cost=0.00..1394.91 rows=90491 width=14)
SubPlan 3
-> Materialize (cost=0.00..7001.57 rows=276838 width=14)
-> Seq Scan on "SBPDB" (cost=0.00..4265.38 rows=276838 width=14)
SubPlan 4
-> Materialize (cost=0.00..7001.57 rows=276838 width=14)
-> Seq Scan on "SBPDB" "SBPDB_1" (cost=0.00..4265.38 rows=276838 width=14)
(13 rows)
所以，这只是50万行的样本，1小时后还在运行。总的行数是。
genomes=> select count(*) from "pairTable2";
count
----------
36633962
(1 row)
我至少需要一些建议来找出一个更好的查询来加快我的需求。
表的例子。(1)
genomic_accession | assembly | product_accession | tmpcol
-------------------+-----------------+-------------------+----------------
NC_007777.1 | GCF_000013345.1 | WP_011437108.1 | WP_011437109.1
NC_007777.1 | GCF_000013345.1 | WP_011437109.1 | WP_011437110.1
NC_007777.1 | GCF_000013345.1 | WP_011437110.1 | WP_011437113.1
NC_007777.1 | GCF_000013345.1 | WP_011437113.1 | WP_011437114.1
NC_007777.1 | GCF_000013345.1 | WP_011437114.1 | WP_011437116.1
NC_007777.1 | GCF_000013345.1 | WP_011437116.1 | WP_011437117.1
NC_007777.1 | GCF_000013345.1 | WP_011437117.1 | WP_011437118.1
NC_007777.1 | GCF_000013345.1 | WP_011437118.1 | WP_011437120.1
NC_007777.1 | GCF_000013345.1 | WP_011437120.1 | WP_011437121.1
NC_007777.1 | GCF_000013345.1 | WP_011437121.1 | WP_011437123.1
(10 rows)
(2)
product_accession
-------------------
WP_005887071.1
WP_005913801.1
WP_002804432.1
WP_010366489.1
WP_012444785.1
NP_636898.1
WP_046342269.1
WP_074057745.1
WP_039420813.1
WP_005932253.1
(10 rows)
(3)
product_accession
-------------------
ABG85315.1
ABG85570.1
ABG86033.1
ABG86301.1
ABG87594.1
ACX82524.1
ACX83274.1
ACX83416.1
ADX79866.1
ADX79880.1
(10 rows)
输出的例子。
genomic_accession | assembly | product_accession | tmpcol | CACHE | SBP
-------------------+-----------------+-------------------+----------------+-------+-----
NC_007899.1 | GCF_000009945.1 | WP_011457581.1 | WP_011457582.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457582.1 | WP_011457583.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457583.1 | WP_011457584.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457584.1 | WP_011457585.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457585.1 | WP_011457586.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457586.1 | WP_011457587.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457587.1 | WP_011457588.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457588.1 | WP_011457589.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457589.1 | WP_011457590.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457590.1 | WP_011457592.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457592.1 | WP_011457593.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457593.1 | WP_011457594.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457594.1 | WP_011457596.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457596.1 | WP_011457597.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457597.1 | WP_011457598.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457598.1 | WP_011457600.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457600.1 | WP_011457601.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457601.1 | WP_011457602.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457602.1 | WP_011457603.1 | f | f
NC_007899.1 | GCF_000009945.1 | WP_011457603.1 | WP_011457604.1 | f | f
先谢谢你
这是你的查询。
SELECT "genomic_accession", "assembly", "product_accession", "tmpcol",
("product_accession" IN ( SELECT product_accession FROM "cacheDB" ) OR
"tmpcol" IN ( SELECT product_accession FROM "cacheDB")
) AS "CACHE",
("product_accession" IN ( SELECT product_accession FROM "SBPDB" ) OR
"tmpcol" IN ( SELECT product_accession FROM "SBPDB" ) AS "SBP"
FROM (SELECT * FROM "pairTable2" LIMIT 500000) "dbplyr_031";
我会把所有的双引号去掉。 不要创建需要转义的列名和表名。 然后。EXISTS 用正确的索引往往会有更好的表现。
SELECT "genomic_accession", "assembly", "product_accession", "tmpcol",
(EXISTS (SELECT 1
FROM "cacheDB" c
*WHERE c.product_accession IN (pt.product_accession, pt.tmpcol )
)
) AS CACHE,
(EXISTS (SELECT 1
FROM "SBPDB" s
WHERE s.product_accession IN (pt.product_accession, pt.tmpcol )
)
) AS SBP
FROM (SELECT * FROM "pairTable2" LIMIT 500000) pt;
那么，为了保证性能，你希望索引能用在 cachedb(product_accession) 和 sbpdb(product_accession).