如何限制Sphinx使用内存？

Question

我的系统是

% uname -or
FreeBSD 11.0-RELEASE-p2

狮身人面像版本是

% searchd --help
Sphinx 2.2.11-id64-release (95ae9a6)

狮身人面像配置：

index content_rt_template : common_template
{
    type              = rt
    rt_mem_limit      = 128M              # 128M only...

    rt_field          = text
    rt_attr_string    = text
    rt_field          = title
    rt_attr_string    = title
    rt_field          = url
    rt_attr_string    = url
    rt_attr_bigint    = item_id
    rt_attr_uint      = source_id
    rt_attr_timestamp = published_date
    rt_attr_timestamp = created_date
}

common {
    lemmatizer_base   = /path/to/sphinx/
}

indexer
{
    mem_limit         = 128M              # 128M only...
}


index content_rt_from_20170501_to_20170601 : content_rt_template
{
    path              = /path/to/sphinx/data/2017/content_rt_from_20170501_to_20170601
}

index content_rt_from_20170601_to_20170701 : content_rt_template
{
    path              = /path/to/sphinx/data/2017/content_rt_from_20170601_to_20170701
}

index content_rt_from_20171201_to_20180101 : content_rt_template
{
    path              = /path/to/sphinx/data/2017/content_rt_from_20171201_to_20180101
}

index content2017
{
    type              = distributed
    local             = content_rt_from_20170501_to_20170601
    local             = content_rt_from_20170601_to_20170701
    local             = content_rt_from_20171201_to_20180101
}

searchd
{
    listen            = 127.0.0.1:9417
    listen            = 9317:mysql41
    log               = /path/to/sphinx/log/searchd_2017.log
    query_log         = /path/to/sphinx/log/query_2017.log
    read_timeout      = 60
    max_children      = 30
    pid_file          = /path/to/sphinx/pid/searchd2017.pid
    seamless_rotate   = 0
    preopen_indexes   = 0
    unlink_old        = 1
    workers           = threads # for RT to work
    binlog_path       = /path/to/sphinx/data/2017/
}

启动Sphinx之前使用的内存：

Mem[|||||||||                     5.33G/40.0G]
Swp[||||||||||||||||||||||||||||||3.35G/4.00G]

登录Sphinx开始：

% ./start.sh
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/path/to/sphinx/conf/content2017.conf'...
listening on 127.0.0.1:9417
listening on all interfaces, port=9317
WARNING: index 'common_template': key 'path' not found - NOT SERVING
WARNING: index 'content_rt_template': path must be specified - NOT SERVING
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20170501_to_20170601'
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20170601_to_20170701'
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20171201_to_20180101'
precached 3 indexes in 6.520 sec
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/path/to/sphinx/conf/content_dist.conf'...
listening on 127.0.0.1:9312
listening on all interfaces, port=9306
WARNING: index 'common_template': key 'path' not found - NOT SERVING
WARNING: index 'content_rt_template': path must be specified - NOT SERVING
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_snippet'
precached 1 indexes in 0.064 sec

Sphinx启动后使用的内存：

Mem[|||||||||||||||               11.6G/40.0G]
Swp[||||||||||||||||||||||||||||||3.34G/4.00G]

狮身人面像使用约6G。但根据mem_limit和rt_mem_limit必须使用不超过128 * 3 = 384M。

使用大量内存可能是什么原因？

也许这个警告的理由？

警告：无法初始化进程共享rwlock：进程共享rwlock

UPD

我曾在Ubuntu 16.04上尝试过，情况也一样。

Answer 1

rt_mem_limit仅限制ram块本身的大小。任何磁盘块都将使用自己的内存。

通常，构成最大部分的属性（默认情况下）都保存在内存中。可以用http://sphinxsearch.com/docs/current.html#conf-ondisk-attrs减少记忆

Answer 2

有选择权

ondisk_attrs = pool

RT索引使用的RAM不多。