MariaDB / MySQL 上的计数(*)极其缓慢

问题描述 投票:0回答:1

运行

EXPLAIN SELECT COUNT(*) FROM activities
显示它计划使用二级索引,并且只有一列 int 和 keylen 为 5。操作需要 2 分钟到 6 分钟不等,偶尔

背景

  • MariaDB - 版本 10.11.7
  • 按日期范围分区 InnoDB 表
  • 表中有600M条记录
  • 桌子尺寸
  • 常规操作需要合理的时间,即最多 2-3 分钟
  • 定义了 3 个索引,文本上没有,全部为 BTREE -
    • 主要 - int(11) + datetime,索引大小约为表大小的 30%
    • Index-1,可为空,非唯一 - 索引 int(11),大小约为表大小的 15%
    • Index-1,可为空,非唯一 - 索引 int(11),大小约为表大小的 15%
  • 按月分区,即按范围列分区(
    activity_datetime
    )
  • 隔离级别设置为可重复读取(问题发生后更改,因此这不是原因,但也不是解决方案)
  • 具有 64GB RAM 的独立服务器,仅服务于数据库
  • innodb_buffer_pool_size 设置为 50GB
  • innodb_buffer_pool_chunk_size 设置为 1GB
  • innodb_read_io_threads 设置为 8(从默认 4 更改)
  • SHOW ENGINE INNODB STATUS - 显示没有什么特别的,死锁很少发生,而且肯定不是原因

尝试的改变:

  • 将隔离级别设置为可重复读取
  • count(id) 而不是 count(*)
  • 调整了系统变量,主要是——
    • 将 innodb_buffer_pool_size 增加到 50GB
    • 将 innodb_buffer_pool_chunk_size 增加到 2GB
    • 将 innodb_read_io_threads 增加到 8
  • 在带有索引的表副本上进行相同的测试
  • 在没有分区使用的表副本上进行相同的测试

恐怕没有什么帮助,count(*) 超级慢,99.99% 都花在了“发送数据”上

1   Starting           28 µs
2   Checking Permissions   6 µs
3   Opening Tables         23 µs
4   After Opening Tables.  15 µs
5   System Lock            18 µs
6   Table Lock             4 µs
7   Init                   7 µs
8   Optimizing             4 µs
9   Statistics             19 µs
10  Preparing              26 µs
11  Executing              2 µs
**12    Sending Data           358.4 s**
13  End Of Update Loop.    20 µs
14  Query End              3 µs
15  Commit                 6 µs
16  Closing Tables         4 µs
17  Unlocking Tables       2 µs
18  Closing Tables         15 µs
19  Starting Cleanup       3 µs
20  Freeing Items          6 µs
21  Updating Status        14 µs
22  Reset For Next Command 4 µs

创建表

CREATE TABLE `contact_activity` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `id_list` int(11) DEFAULT NULL,
  `activity` varchar(50) DEFAULT NULL,
  `id_contact` int(11) DEFAULT NULL,
  `id_isp` int(11) DEFAULT NULL,
  `id_esp` int(11) DEFAULT NULL,
  `id_esp_connection` int(11) DEFAULT NULL,
  `id_message` int(11) DEFAULT NULL,
  `id_campaign` int(11) DEFAULT NULL,
  `activity_datetime` datetime NOT NULL,
  `ip` varchar(150) DEFAULT NULL,
  `country_code` varchar(50) DEFAULT NULL,
  `browser` varchar(50) DEFAULT NULL,
  `os` varchar(50) DEFAULT NULL,
  `data` text DEFAULT NULL,
  `id_platform` int(11) DEFAULT NULL,
  `created_at` timestamp NOT NULL DEFAULT current_timestamp(),
  `updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  PRIMARY KEY (`id`,`activity_datetime`),
  KEY `id_contact` (`id_contact`),
  KEY `id_list` (`id_list`)
) ENGINE=InnoDB AUTO_INCREMENT=629997644 DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
 PARTITION BY RANGE  COLUMNS(`activity_datetime`)
(PARTITION `pMonth_231` VALUES LESS THAN ('2023-01-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_232` VALUES LESS THAN ('2023-02-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_233` VALUES LESS THAN ('2023-03-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_234` VALUES LESS THAN ('2023-04-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_235` VALUES LESS THAN ('2023-05-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_236` VALUES LESS THAN ('2023-06-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_237` VALUES LESS THAN ('2023-07-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_238` VALUES LESS THAN ('2023-08-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_239` VALUES LESS THAN ('2023-09-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2310` VALUES LESS THAN ('2023-10-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2311` VALUES LESS THAN ('2023-11-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2312` VALUES LESS THAN ('2023-12-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_241` VALUES LESS THAN ('2024-01-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_242` VALUES LESS THAN ('2024-02-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_243` VALUES LESS THAN ('2024-03-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_244` VALUES LESS THAN ('2024-04-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_245` VALUES LESS THAN ('2024-05-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_246` VALUES LESS THAN ('2024-06-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_247` VALUES LESS THAN ('2024-07-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_248` VALUES LESS THAN ('2024-08-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_249` VALUES LESS THAN ('2024-09-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2410` VALUES LESS THAN ('2024-10-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2411` VALUES LESS THAN ('2024-11-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMonth_2412` VALUES LESS THAN ('2024-12-01 00:00:00') ENGINE = InnoDB,
 PARTITION `pMaxMonth` VALUES LESS THAN (MAXVALUE) ENGINE = InnoDB) 
count bigdata innodb partition
1个回答
0
投票

key = id_contact key_len = 5
——这是可以预料的。这意味着正在对
KEY(id_contact)
进行完整索引扫描,这是获取
COUNT(*)
所能做的最好的事情。

请求

COUNT(id)
将检查每个 id 是否为非空并且速度稍快一些。

© www.soinside.com 2019 - 2024. All rights reserved.