Spring Boot Elasticsearch 分页和排序不会对 ElasticSearch 中的“文本”类型字段进行排序

Question

在 Spring Boot Elasticsearch 存储库中，我使用 PaginationAndSorting Spring 功能 (Pagination)，该功能对于“日期”字段非常有效，但当我尝试按“文本”字段排序时会出现错误。这种行为是预期的吗？或者我可以做些什么来修复它，以便我可以按所有字段类型进行排序。

LogstashLogRepository.java：

public interface LogstashLogRepository extends ElasticsearchRepository<LogstashLog, String> {
   
    /**
     * Repository's findByLevel does a query for all LogstashLogs with given level.
     * @param logstashLevel level to match, choices are SECURITY and ACTIVITY.
     * @return LogstashLogs with matching level.
     */
    @Query("{\"bool\": {\"must\": [{\"match\": {\"level\": \"?0\"}}]}}")
    Page<LogstashLog> findByLevel(LogstashLog.LogstashLevel logstashLevel, Pageable pageable);
}

LogstashLogController.java：

@RestController
@RequestMapping(value = "/")
public class LogstashLogController {

    /**
     * Object mapper.
     */
    @Autowired
    private ObjectMapper objectMapper;

    /**
     * Spring Boot ES Repository.
     */
    @Autowired
    private LogstashLogRepository logstashLogRepository;

   

    /**
     * Retrieves all Logstash logs by level.
     *
     * @param levelRequest level requested.
     * @return ResponseEntity.
     */
    @Operation(summary = "Retrieve all Logstash logs specified by a level.")
    @GetMapping("/level")
    public ResponseEntity<List<LogstashLog>> findLogstashLogsByLevel(@RequestBody final LogstashLog.LogstashLevel levelRequest,
                                                               @PageableDefault(size = 20) final Pageable pageable) {
        Page<LogstashLog> pagedLogstashLogs = logstashLogRepository.findByLevel(levelRequest, pageable);
        List<LogstashLog> logstashLogs = pagedLogstashLogs.stream().toList();
        return new ResponseEntity<>(logstashLogs, HttpStatus.OK);
    }
}

LogstashLog.java（模型/dto）：

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
@Document(indexName = "logstash-log")
public class LogstashLog {

    @Id
    private String id;

    @JsonAlias("@timestamp")
    @Field(name = "@timestamp")
    private Instant timestamp;

    @JsonAlias("Class")
    @JsonProperty("class")
    @Field(name = "class")
    private String loggingClass;

    @JsonAlias("Level")
    private LogstashLevel level;

    @JsonAlias("Date")
    private Instant date;

    @JsonAlias("Data")
    private LogstashLogData data;

    public enum LogstashLevel {
        SECURITY,
        ACTIVITY
    }
}

弹性搜索中元数据的映射（我使用elk堆栈，因此该数据通过logstash放入）


"logstash-log": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "class": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "data": {
          "properties": {
            "message": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "user": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "date": {
          "type": "date"
        },
        "event": {
          "properties": {
            "original": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "level": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }

如果我使用时间戳（或日期），我可以很好地排序，并返回如下内容：

[
    {
        "id": "aafC544C-o-vp9msel90",
        "timestamp": "2024-04-16T16:34:25.916453227Z",
        "level": "SECURITY",
        "date": "2024-04-16T16:34:25.914062Z",
        "data": {
            "user": "John Smith",
            "message": "Title Security Chicken"
        },
        "class": "com.example.ExampleController"
    },
    {
        "id": "ZqfC544C-o-vp9msel9c",
        "timestamp": "2024-04-16T16:34:25.908138479Z",
...

但是如果我尝试按类别（或 ID、级别等）排序，我会得到： 请求处理失败：org.springframework.data.elasticsearch.UncategorizedElasticsearchException：[es/search]失败：[search_phase_execution_exception]所有分片失败]根本原因“，”异常“：”co.elastic.clients.elasticsearch._types.ElasticsearchException： [es/search] 失败：[search_phase_execution_exception] 所有分片失败 ....

我已经修改了 LogstashRepository，使其不使用分页返回任何内容，而是尝试在结果上执行此操作。但由于我使用 @Query 注释而不是 NativeSearchQuery，我担心我对结果无能为力，因为它们不被识别为查询类型。我是否必须切换到使用 Query 本身而不是注释，以便我可以使用 ES 的排序而不是 Pagination 的排序？

Answer 1

这个问题的答案非常简单。 Elasticsearch 无法对文本进行排序，除非您通过 fielddata=true 告诉它（尽管他们不推荐此方法）或将其更改为类型关键字而不是文本 - 两者都需要修复 Logstash 输入数据的方式。 或者，搜索本身可以只说“*.keyword”，所以在我的例子中“class.keyword”将正确执行排序。