如何从数据库中获取数据后多次提高排序列表的性能

问题描述 投票:1回答:2

我刚刚遇到了排序列表性能非常慢的问题。在对数据库运行查询后,我得到了结果,需要在满足要求之前对其进行4次订购。查询运行速度非常快,我几乎立即获得了所有结果,但是对记录进行排序大约需要8秒。

我也使用分页,因此每次我每页只选择50条记录,但我必须一次又一次地重新排序整个列表,这是一场噩梦。你们有没有办法让它跑得更快?

var studentMessages = context.Students
    .Where(s => s.SchoolId == SchoolId).ToList();
var sSorted = studentMessages
    .Where(x => x.message == null && x.student.Status != (int)StudentStatusEnum.NotActive)
    .OrderByDescending(x => x.student.UserId)
    .ToList();

sSorted = sSorted
    .Concat(studentMessages
        .Where(x => x.message != null && x.student.Status != (int)StudentStatusEnum.NotActive)
        .OrderBy(x => x.message.NextFollowUpDate)
        .ToList()
    ).ToList();

sSorted = sSorted
    .Concat(studentMessages
        .Where(x => x.message != null && x.student.Status == (int)StudentStatusEnum.NotActive)
        .OrderByDescending(x => x.message.NextFollowUpDate)
        .ToList()
    ).ToList();

sSorted = sSorted
    .Concat(studentMessages
    .Where(x => x.message == null && x.student.Status == (int)StudentStatusEnum.NotActive)
    .OrderByDescending(x => x.user.Id)
    .ToList()
    ).ToList();

var allStudents = (isSelectAll == true ? sSorted  : sSorted .Skip(skipNumber).Take(query.AmountEachPage)).ToList();
c# performance entity-framework linq entity-framework-6
2个回答
1
投票

我认为你的问题的原因是你得到子集或你的序列并订购这个子集。您这样做了几次,并决定列出所有中间结果。

我们先来看看你想如何订购你的学生。

所以你有一个schoolId和一系列Students。每个Student都有属性SchoolIdMessageStatus

你从学校拿Students带来所有schoolId,由于某种原因你决定称这些学生为studentMessages

然后你想按以下顺序订购这些Students(studentmessages):

  • 首先所有留空消息且状态不等于notActive的学生,按UserId降序排序
  • 然后所有具有非空消息且状态不等于notActive的学生,按Message.NextFollowUpdate排序
  • 然后所有具有非空消息和状态等于notActive的学生,按Message.NextFollowUpdate排序
  • 最后所有留空信息和状态等于notActive的学生,按User.Id降序排序(肯定你不是指UserId?我认为这将是相同的)

在表格中:

group | Message |  Status      | Order by
  1   | == null | != notActive | descending UserId
  2   | != null | != notActive | ascending  message.NextFollowUpdate
  3   | != null | == notActive | descending message.NextFollowUpdate
  4   | == null | == notActive | ascending  UserId

其中一种方法是让您的数据库管理系统执行此操作(AsQueryable)。排序算法似乎相当复杂。我不确定DBMS是否可以比您的流程更高效地执行此操作。

另一种方法是仅获取您实际需要的学生并让您的过程进行排序(AsE​​numerable)。提供一个实现IComparer<Student>的类来决定订单。

int schoolId = ...
IComparer<Student> mySpecialStudentComparer = ...
var orderedStudents = dbContext.Students
    .Where(student => student.SchoolId == schoolId)
    .AsEnumerable()         // move the selected data to local process

    // now that the data is local, we can use our local Student Comparer
    .OrderBy(mySpecialStudentComparer);

如果您的学生拥有许多在获取数据后将不会使用的属性,请考虑创建仅包含所需属性的本地类,并将所选数据限制为此本地类,例如FetchedStudent

    .Select(student => new FetchedStudent
    {
        // select only the properties you actually plan to use,

        // for the sorting we need at least the following:
        Message = student.Message,
        Status = student.Status
        UserId = student.UserId,

        // Select the other Student properties you plan to use, for example:
        Id = student.Id,
        Name = student.Name, 
        ...
    }

当然,在这种情况下,您的比较器需要实现IComparer<FetchedStudent>

所以,让我们创建一个StudentComparer,根据您的要求对学生进行排序!

class StudentComparer : IComparer<FetchedStudent>
{
    private readonly IComparer<int> UserIdComparer = Comparer<int>.Default;
    private readonly IComparer<DateTime> nextFollowUpdateComparer =
                     Comparer<DateTime>.Default;

    public int CompareTo(FetchedStudent x, FetchedStudent y)
    {
        // TODO: decide what to do with null students: exception?
        // or return as smallest or largest

        // Case 1: check if x is in sorting group 1
        if (x.Message == null && x.Status == notActive)
        {
            // x is in sorting group 1
            if (y.Message == null && y.Status == notActive)
            {
                // x and y are in sorting group 1.
                // order by descending UserId
                return -UserIdComparer.CompareTo(x.UserId, y.UserId);
                // the minus sign is because of the descending
            }
            else
            {   // x is in group 1, y in group 2 / 3 / 4: x comes first
                return -1;
            }
        }

        // case 2: check if X is in sorting group 2
        else if (x.Message != null && x.Status != notActive)
        {   // x is in sorting group 2
            if (y.Message == null && y.Status != notActive)
            {   // x is in group 2; y is in group 1: x is larger than y
                return +1;
            }
            else if (y.Message == null && y.Status != notActive)
            {   // x and y both in group 2: order by descending nextFollowUpDate
                // minus sign is because descending
                return -nextFollowUpdateComparer.CompareTo(
                       x.Message.NextFollowUpdate,
                       y.Message.NextFollowUpdate);
            }
            else
            {   // x in group 2, y in 3 or 4: x comes first
                return -1;
            }
        }

        // case 3: check if X in sorting group 3
        else if (x.Message == null && x.Status != notActive)
        {
           ... etc, you'll know the drill by know
    }
}    

可能的改善

您会看到比较器不断地比较x.Message是否等于null以及x.Status是否等于notActive,以检测x和y属于哪个排序组。

考虑创建一个只计算一次Student所属的排序组并记住排序组的函数:

.Select(student => new FetchedStudent
{
    SortingGroup = student.ToSortingGroup(),

    ... // other properties you need
}

public int CompareTo(FetchedStudent x, FetchedStudent y)
{
    switch (x.SortingGroup)
    {
        case 1:
           switch y.SortingGroup:
           {
               case 1: // x and y both in sorting group 1
                  return -UserIdComparer.CompareTo(x.UserId, y.UserId);

               default: // x in sorting group 1, y in 2 / 3 / 4: x smaller
                  return -1;
           }
        case 2:
           switch y.SortingGroup:
           {
               case 1: // x in sorting group 2; y in sorting group 1: x larger
                  return +1;
               case 2: // x and y both in sorting group 2
                  return -nextFollowUpdateComparer.CompareTo(
                       x.Message.NextFollowUpdate,
                       y.Message.NextFollowUpdate);
           }    

这样,与消息和状态的比较只进行一次


1
投票

代码的性能问题很可能是延迟加载的结果。通过使用studentmessage属性(并且在第四个查询的情况下也是user属性),再次查询每一行的数据库。 studentMessage包含的行越多,代码执行的速度就越慢。这是一个所谓的“n + 1 SELECT”问题。有关详细信息,请参阅此link

如果要快速解决问题,则需要声明相关的子实体也加载了第一个请求。为此,您需要更改以下行并包含所有相关实体:

var studentMessages = context.Students
  .Where(s => s.SchoolId == SchoolId)
  .ToList();    

应该改变,以便实体messageuserstudent也包括在内:

var studentMessages = context.Students
  .Include(x => x.message)
  .Include(x => x.student)
  .Include(x => x.user)
  .Where(s => s.SchoolId == SchoolId)
  .ToList();    

这样,数据就会向数据库加载一个请求,而不是以后加载。

© www.soinside.com 2019 - 2024. All rights reserved.