使用给定的ID在行中执行JPA搜索

Question

我正在努力使我的应用程序中的搜索更有效。我试图解决的用例是给定一组ID，只在那些给定的行中执行搜索。问题是这个id的大小超过1024.当我使用BooleanQuery.setMaxClauseCount(Ids.size()时，我没有得到tooManyClauses异常但由于timeOut它没有返回任何结果。

理想情况下我想这样做：

public ResponseEntity search(
            @RequestParam(Param.SEARCH) String query,
            @RequestParam(value = Param.PAGE, defaultValue = Pagination.DEFAULT_PAGE) int page,
            @RequestParam(value = Param.SIZE, defaultValue = Pagination.DEFAULT_SIZE) int size)
            throws Exception {
        log.info("Started searching for query : " + query);

        final Set<Long> accessibleIds = projectsServiceUtils
                .filterIdsAccessibleByLoggedInPerson(null);

        final FullTextQuery fullTextQuery = searchHelper
                .getPrefixSearchQuery(Project.class, SearchFields.PROJECT_BOOST_MAP, query, accessibleRadarIds);

        final List<Project> projects = fullTextQuery.setFirstResult(page * size)
                .setMaxResults(size).getResultList();

        return ResponseEntity.ok()
                .header(Pagination.PAGINATION_PAGE, String.valueOf(page))
                .header(Pagination.PAGINATION_SIZE, String.valueOf(size))
                .header(Pagination.PAGINATION_COUNT, Long.toString(fullTextQuery.getResultSize()))
                .body(projectSearchObjectWriter.writeValueAsString(projects));
    }

用这个作为我的getPrefixSearchQuery方法：

public <T> FullTextQuery getPrefixSearchQuery(
            Class<T> typeClass, Map<String, Float> boostMap, String searchTerms, Set<Long> ids) {
        FullTextEntityManager fullTextEntityManager = Search
                .getFullTextEntityManager(entityManager);
        QueryBuilder qb = fullTextEntityManager
                .getSearchFactory()
                .buildQueryBuilder()
                .forEntity(typeClass).get();

        BooleanQuery.Builder luceneQuery = new BooleanQuery.Builder();

        String[] tokens = searchTerms.split("\\s+");

        for (String token : tokens) {
            if (!StopAnalyzer.ENGLISH_STOP_WORDS_SET.contains(token) || tokens.length == 1) {
                // If search term contains only digits then search substring (possibly phone number)
                final String matcher = token.toLowerCase() + "*";

                final WildcardContext wildcardContext = qb.keyword().wildcard();
                TermMatchingContext termMatchingContext = null;
                for (String field : boostMap.keySet()) {
                    if (termMatchingContext != null) {
                        termMatchingContext = termMatchingContext.andField(field).boostedTo(boostMap.get(field));
                    } else {
                        termMatchingContext = wildcardContext.onField(field).boostedTo(boostMap.get(field));
                    }
                }
                final Query subQuery = termMatchingContext.matching(matcher).createQuery();
                luceneQuery.add(subQuery, BooleanClause.Occur.MUST);
            }
        }
        // NEW CODE TO SUPPORT FILTERING
        if (ids != null) {
            BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));
            TermMatchingContext termMatchingContext2 = qb.keyword().wildcard().onField("id");

            for (Long id : ids) {
                luceneQuery.add(termMatchingContext2.matching(id).createQuery(), BooleanClause.Occur.FILTER);
            }
        }

        FullTextQuery jpaQuery = fullTextEntityManager
                .createFullTextQuery(luceneQuery.build(), typeClass);

        return jpaQuery;
    }

由于我没有使用上述配置获得任何结果，因此我必须在获取查询结果后过滤我的结果。这会导致更多问题，因为我必须确保结果的大小等于传递的大小，因此我不必过滤整个结果集，而是对整个结果集进行筛选，然后对其进行分页以获得结果期望的大小。这是我现在非常低效的工作：

public ResponseEntity search(
            @RequestParam(Param.SEARCH) String query,
            @RequestParam(value = Param.PAGE, defaultValue = Pagination.DEFAULT_PAGE) int page,
            @RequestParam(value = Param.SIZE, defaultValue = Pagination.DEFAULT_SIZE) int size,
            @RequestParam(value = Param.SORT, defaultValue = SortDefault.BY_NAME) String sort)
            throws IOException, IdmsException {
        log.info("Started searching for query : " + query);

        final FullTextQuery fullTextQuery = searchHelper
                .getPrefixSearchQuery(Project.class, SearchFields.PROJECT, query);
        final List<Project> projects = fullTextQuery.getResultList();

        final ImmutableSet<Long> Ids = projects
                .stream()
                .map(Project::getId)
                .collect(collectingAndThen(Collectors.toSet(),
                        ImmutableSet::copyOf));

        final Set<Long> accessibleIds = projectsServiceUtils
                .filterIdsAccessibleByLoggedInPerson(Ids);

        PageRequest pageRequest = new PageRequest(
                page, size);

        final Page<Project> projectsFiltered = projectRepository.findByIdIn(
                accessibleIds, pageRequest);

        return ResponseEntity.ok()
                .header(Pagination.PAGINATION_PAGE, String.valueOf(page))
                .header(Pagination.PAGINATION_SIZE, String.valueOf(size))
                .header(Pagination.PAGINATION_COUNT, String.valueOf(accessibleIds.size()))
                .body(projectSearchObjectWriter.writeValueAsString(projectsFiltered.getContent()));
    }

有没有什么办法可以在给定Id的行中执行搜索并将其分页？

Answer 1

    if (ids != null) {
       BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));
       TermMatchingContext termMatchingContext2 = qb.keyword().wildcard().onField("id");

       for (Long id : ids) {
           luceneQuery.add(termMatchingContext2.matching(id).createQuery(), BooleanClause.Occur.FILTER);
       }
   }

您正在使用通配符查询，但您不需要。根据您的ID列表的大小，这可能是您获得超时的原因。

试试这个：

     if (ids != null) {
        BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));

        for (Long id : ids) {
            luceneQuery.add(qb.keyword().onField("id").matching(id).createQuery(), BooleanClause.Occur.FILTER);
        }
    }

此外，每次执行查询时都不需要设置max子句计数。实际上，你真的不应该，因为可以并行执行多个查询。

只需在应用程序启动时执行类似的操作：

BooleanQuery.setMaxClauseCount(<some large number>);