Postgres死锁,带有(选择用于共享+插入)和(选择用于更新+更新)

问题描述 投票:0回答:1

我有下表(所有示例都在psycopg2 python库中:]

CREATE TABLE IF NOT EXISTS TeamTasks
        (
            id              SERIAL PRIMARY KEY,
            round           INTEGER,
            task_id         INTEGER,
            team_id         INTEGER,
            score           FLOAT         DEFAULT 0,
            UNIQUE (round, task_id, team_id)
        )

我有两个功能:

  1. 从上一回合中取TeamTasks并将它们复制到新的回合中,而SELECTINSERT之间没有更新,并且该功能的实现如下:
query = """
    WITH prev_table AS (
        SELECT score FROM teamtasks 
        WHERE task_id = %(task_id)s AND team_id = %(team_id)s AND round <= %(round)s - 1
        ORDER BY round DESC LIMIT 1 
        FOR SHARE
    )
    INSERT INTO TeamTasks (task_id, team_id, round, score) 
    SELECT %(task_id)s, %(team_id)s, %(round)s, score
    FROM prev_table;
"""

with aux.db_cursor() as (conn, curs):    
    for team_id in range(team_count):
        for task_id in range(task_count): 
            curs.execute(
                query, 
                {
                    'task_id': task_id,
                    'team_id': team_id,
                    'round': cur_round + 1,
                },
            )
    conn.commit()

aux.db_cursor只是获得psycopg2连接和光标的便捷包装。

  1. TeamTasks中的更新针对特定的团队对和特定的任务以及多个回合。它是这样实现的:
# I have team1_id, team2_id and task_id

query1 = "SELECT score from teamtasks WHERE team_id=%s AND task_id=%s AND round=%s FOR NO KEY UPDATE"

query2 = "UPDATE teamtasks SET score = %s WHERE team_id=%s AND task_id=%s AND round >= %s"

with aux.db_cursor() as (conn, curs):
    curs.execute(query1, (team1_id, task_id, cur_round))
    score1, = curs.fetchone()

    curs.execute(query1, (team2_id, task_id, cur_round))
    score2, = curs.fetchone()

    sleep(0.1)  # Here happens something time-consuming

    curs.execute(query2, (score1 + 0.1, team1_id, task_id, cur_round))
    curs.execute(query2, (score2 - 0.1, team2_id, task_id, cur_round))

    conn.commit()

我可以保证每个团队只能成为第二个职能中一个更新的主体,因此所有同时更新的团队始终是截然不同的。

此外,第一个函数很少运行,除这两个函数外,没有其他人会更新该表,因此,第一个函数的锁定正好使在TeamTasks复制期间不更改该表。

在上述环境下,我遇到了很多类似以下的死锁:

postgres_1  | 2019-11-17 20:43:08.510 UTC [49] ERROR:  deadlock detected
postgres_1  | 2019-11-17 20:43:08.510 UTC [49] DETAIL:  Process 49 waits for ShareLock on transaction 685; blocked by process 65.
postgres_1  |   Process 65 waits for ShareLock on transaction 658; blocked by process 49.
postgres_1  |   Process 49:
postgres_1  |           WITH prev_table AS (
postgres_1  |               SELECT score FROM teamtasks
postgres_1  |               WHERE task_id = 8 AND team_id = 6 AND round <= 1 - 1
postgres_1  |               ORDER BY round DESC LIMIT 1
postgres_1  |               FOR SHARE
postgres_1  |           )
postgres_1  |           INSERT INTO TeamTasks (task_id, team_id, round, score)
postgres_1  |           SELECT 8, 6, 1, score
postgres_1  |           FROM prev_table;
postgres_1  |
postgres_1  |   Process 65: SELECT score from teamtasks WHERE team_id=0 AND task_id=8 AND round=0 FOR NO KEY UPDATE
postgres_1  | 2019-11-17 20:43:08.510 UTC [49] HINT:  See server log for query details.
postgres_1  | 2019-11-17 20:43:08.510 UTC [49] CONTEXT:  while locking tuple (0,69) in relation "teamtasks"
postgres_1  | 2019-11-17 20:43:08.510 UTC [49] STATEMENT:
postgres_1  |           WITH prev_table AS (
postgres_1  |               SELECT score FROM teamtasks
postgres_1  |               WHERE task_id = 8 AND team_id = 6 AND round <= 1 - 1
postgres_1  |               ORDER BY round DESC LIMIT 1
postgres_1  |               FOR SHARE
postgres_1  |           )
postgres_1  |           INSERT INTO TeamTasks (task_id, team_id, round, score)
postgres_1  |           SELECT 8, 6, 1, score
postgres_1  |           FROM prev_table;

如何解决这些僵局?我没有看到一个整洁的解决方案吗?

python postgresql deadlock
1个回答
0
投票

select for share在这里似乎不必要。该语法用于保留引用完整性。在您的情况下,您是从同一个teamtasks表中进行选择并插入其中,因此您不必要在表上持有会导致两个连接相互阻塞的锁(最终,重构代码会很不错,所以您只需使用一种连接,但我不知道这对您来说是多么可行)。据我所知,select for share语法与对其他表的更新和引用完整性有关的更多,而与对同一表的插入无关。

问题在于,在第一个aux_db_cursor()调用中,当您循环遍历FOR SHAREteamtasks时,您正在range(team_count)中锁定range(task_count)中的几行-然后在第二个aux_db_cursor()中循环调用您在执行某些行上的UPDATE之前正在执行一项耗时的任务-这些UPDATE锁定请求将与那些FOR SHARE锁定冲突。我会放弃FOR SHARE锁,除非您确实需要它们(此时,如果可能,我会寻找将其整合到一个DB连接中的方法)。

© www.soinside.com 2019 - 2024. All rights reserved.