循环遍历Django查询集的末尾

Question

我试图循环一个Django查询集，从最后一条记录开始，一次返回1,000条记录。我可以使用以下查询获取最后1,000条记录：

employees = Employee.objects.all().order_by('-id')[:1000]

假设我的查询集是10,0000个结果。我如何从8,000到9,000？我是否必须使用.count（）来获取总记录数？我的完整查询集是1200万条记录，所以我尽量避免这种情况。

Answer 1

这是我提出的解决方案。没有使用count（），我无法做到。但我确实切换到原始SQL，这应该更快。

from django.db import connection
from employee.models import Employee

# count records using raw SQL (best for larger datasets)
cursor = connection.cursor()
cursor.execute('SELECT COUNT(*) FROM employee_employee;')
(count,) = cursor.fetchone()

records_to_index = 100000
batch_size = 3000

# loop through batch starting at end of queryset going backwards
# continue until records_to_index is met
for end in range(count, count - records_to_index, -batch_size):
    # add 1 to ensure records do not overlap
    start = end - batch_size + 1

    employees = Employee.objects.all().order_by('id')[start:end]

    for employee in employees:
        # work with object