使用字符串对用户定义的数组进行排序会给出错误的顺序，即使文件内容在磁盘上完全可用

问题描述 投票：0回答：1

我正在查询 ElasticSearch 并使用 

jq
在 Bash 中对文档进行本地排序，因为在 ES 中排序对我来说太慢了。

最初的目的是创建一个CSV文件。

但是我发现排序无法正常工作，似乎

sort
步骤没有任何作用。

当我启动 

cURL
请求时，我认为错误的顺序是由于内容被分块所致，因此我将一些结果保存到本地 
test.json
文件中并再次尝试，但它仍然不起作用。

test.json
：

{
    "took": 680,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "max_score": 1.0,
        "hits": [
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113584925",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113584925"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111121254059",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111121254059"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116879444",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116879444"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116879484",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116879484"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114472530",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114472530"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113372966",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113372966"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113046053",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113046053"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113034864",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113034864"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116770197",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116770197"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111123578950",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111123578950"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114472544",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114472544"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111123578971",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111123578971"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124882870",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124882870"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124178732",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124178732"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113649568",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113649568"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113034877",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113034877"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114049560",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114049560"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113034894",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113034894"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116879498",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116879498"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115634604",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115634604"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117550495",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117550495"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117964387",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117964387"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111120647956",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111120647956"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124178784",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124178784"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111123579003",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111123579003"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114049597",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114049597"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113922927",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113922927"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113649622",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113649622"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113922948",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113922948"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124178797",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124178797"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114775147",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114775147"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115634973",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115634973"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115634987",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115634987"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124882959",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124882959"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114049664",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114049664"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117964419",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117964419"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117069687",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117069687"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117453564",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117453564"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113046124",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113046124"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113586875",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113586875"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115735339",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115735339"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111119990227",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111119990227"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111120648051",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111120648051"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117453835",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117453835"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113899778",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113899778"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111120648071",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111120648071"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111117964461",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111117964461"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114049666",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114049666"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114049678",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114049678"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113923053",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113923053"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111119990248",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111119990248"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115735378",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115735378"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111115735384",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111115735384"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116770587",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116770587"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124883047",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124883047"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111114775219",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111114775219"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113373129",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113373129"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113046216",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113046216"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111124883526",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111124883526"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113046278",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113046278"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111113587608",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111113587608"
                    ]
                }
            },
            {
                "_index": "my-index",
                "_type": "_doc",
                "_id": "111111116770696",
                "_score": 1.0,
                "fields": {
                    "field2": [
                        "FOO"
                    ],
                    "field1": [
                        "111111116770696"
                    ]
                }
            }
        ]
    }
}

我使用的命令：

jq '.hits.hits[].fields | [.field1[0] + "," + .field2[0]] | sort | .[0]' -r test.json

结果：

111111113584925,FOO
111111121254059,FOO
111111116879444,FOO
111111116879484,FOO
111111114472530,FOO
111111113372966,FOO
111111113046053,FOO
111111113034864,FOO
111111116770197,FOO
111111123578950,FOO
111111114472544,FOO
111111123578971,FOO
111111124882870,FOO
111111124178732,FOO
111111113649568,FOO
111111113034877,FOO
111111114049560,FOO
111111113034894,FOO
111111116879498,FOO
111111115634604,FOO
111111117550495,FOO
111111117964387,FOO
111111120647956,FOO
111111124178784,FOO
111111123579003,FOO
111111114049597,FOO
111111113922927,FOO
111111113649622,FOO
111111113922948,FOO
111111124178797,FOO
111111114775147,FOO
111111115634973,FOO
111111115634987,FOO
111111124882959,FOO
111111114049664,FOO
111111117964419,FOO
111111117069687,FOO
111111117453564,FOO
111111113046124,FOO
111111113586875,FOO
111111115735339,FOO
111111119990227,FOO
111111120648051,FOO
111111117453835,FOO
111111113899778,FOO
111111120648071,FOO
111111117964461,FOO
111111114049666,FOO
111111114049678,FOO
111111113923053,FOO
111111119990248,FOO
111111115735378,FOO
111111115735384,FOO
111111116770587,FOO
111111124883047,FOO
111111114775219,FOO
111111113373129,FOO
111111113046216,FOO
111111124883526,FOO
111111113046278,FOO
111111113587608,FOO
111111116770696,FOO

为什么？

我应该依赖 

jq
排序吗？我使用正确吗？我的意思是我想按字母顺序进行字符串比较，并且 
field1
都有唯一的值，所以它永远不会是平局并开始比较 
field2
的值（它也可以有各种值，但我只想排序通过 
field1
）

我应该使用 Bash 

sort -k 1
来代替吗？当涉及 100K 行时，哪个更快？

bash sorting jq
1个回答
0
投票

要按 

.fields.field1[0]
排序，请使用 
sort_by(.fields.field1[0])
。但是，当您想要输出用作条件的相同值时，请预先使用 
map
进行减少，然后在结果数组上使用 
sort
：

jq -r '.hits.hits | map(.fields.field1[0]) | sort[]' file.json

111111113034864
111111113034877
111111113034894
111111113046053
111111113046124
111111113046216
111111113046278
111111113372966
111111113373129
111111113584925
:

演示

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.