简单的Python方法永远需要执行/运行

Question

我的代码被卡在clean_up()中的MyClass()方法上>

my_class.py：

import os
import pandas as pd
import psycopg2, pymysql, pyodbc
from db_credentials_dict import db_credentials

class MyClass():

    def __init__(self, from_database, to_table_name, report_name):
        ...

    def get_sql(self):
        ...

    def get_connection(self):
        ...

    def query_to_csv(self):
        ...

    def csv_to_postgres(self):
        ...

    def extract_and_load(self):
        self.query_to_csv()
        self.csv_to_postgres()

    def get_final_sql(self):
        ...

    def postgres_to_csv(self):
        ...

    def clean_up(self):
        print('\nTruncating {}...'.format(self.to_table_name), end='')
        with self.postgres_connection.cursor() as cursor:
            cursor.execute("SELECT NOT EXISTS (SELECT 1 FROM %s)" % self.to_table_name)
            empty = cursor.fetchone()[0]
            if not empty:
                cursor.execute("TRUNCATE TABLE %s" % self.to_table_name)
        self.postgres_connection.commit()
        print('DONE')

        print('Removing {}...'.format(self.filename), end='')
        if os.path.exists(self.filepath):
            os.remove(self.filepath)
            print('DONE')
        else:
            print('{} does not exist'.format(self.filename))
main.py：

from my_class import MyClass
from time import sleep

bookings = MyClass(from_database='...',to_table_name='...',report_name='...')
bookings2 = MyClass(from_database='...',to_table_name='...',report_name='...')
channel = MyClass(from_database='...',to_table_name='...',report_name='...')
cost = MyClass(from_database='...',to_table_name='...',report_name='...')

tables = [bookings, bookings2, channel, cost]
for table in tables:
    table.extract_and_load()

daily_report = MyClass(from_database='...',to_table_name='...',report_name='...')
daily_source_report.postgres_to_csv()
sleep(10)

for table in tables:
    table.clean_up()
[当我运行main.py时，它将运行所有内容，直到最后一个循环，即for table in tables: table.clean_up()。它只是卡在那里，没有错误或警告。

[单独运行该方法时，它可以正常工作，即它会截断postgres表。需要帮助使此方法正常工作，并理解为什么在执行所有其他方法时最终方法不执行。

<<

TRUNCATE子句需要在关系上锁定ACCESS EXCLUSIVE，并且可能会触发某些BEFORE TRUNCATE触发器。

我的猜测是问题在Postgres一方，例如TRUNCATE试图获取对该关系的ACCESS EXCLUSIVE锁定，但该锁定已被其他人锁定。

首先，检查您的Postgres日志。

接下来，用以下方法挂起TRUNCATE来检查Postgres后端在做什么：

SELECT * FROM pg_stat_activity;

接下来，使用以下方法调查锁：

-- View with readable locks info and filtered out locks on system tables
CREATE VIEW active_locks AS
SELECT clock_timestamp(), pg_class.relname, pg_locks.locktype, pg_locks.database,
       pg_locks.relation, pg_locks.page, pg_locks.tuple, pg_locks.virtualtransaction,
       pg_locks.pid, pg_locks.mode, pg_locks.granted
FROM pg_locks JOIN pg_class ON pg_locks.relation = pg_class.oid
WHERE relname !~ '^pg_' and relname <> 'active_locks';

-- Now when we want to see locks just type
SELECT * FROM active_locks;

Answer 1

首先，检查您的Postgres日志。

Answer 2

因为被截断的表已经为空，所以脚本保持静止（不确定原因）。

我像这样修改了我的方法，效果很好：

def clean_up(self):
    print('\nTruncating {}...'.format(self.to_table_name), end='')
    with self.postgres_connection.cursor() as cursor:
        cursor.execute("SELECT NOT EXISTS (SELECT 1 FROM %s)" % self.to_table_name)
        empty = cursor.fetchone()[0]
        if not empty:
            cursor.execute("TRUNCATE TABLE %s" % self.to_table_name)
    self.postgres_connection.commit()
    print('DONE')

    print('Removing {}...'.format(self.filename), end='')
    if os.path.exists(self.filepath):
        os.remove(self.filepath)
        print('DONE')
    else:
        print('{} does not exist'.format(self.filename))