非ASCII字符的字典排序

Question

我已经通过以下代码对ascii字符进行了字典排序：

std::ifstream infile;
std::string line, new_line;
std::vector<std::string> v;
while(std::getline(infile, line))
            {
                // If line is empty, ignore it
                if(line.empty())
                    continue;
                new_line = line + "\n";
                // Line contains string of length > 0 then save it in vector
                if(new_line.size() > 0)
                    v.push_back(new_line);
            }   
sort(v.begin(), v.end());

结果应为：一种ah阿卜杜鲁bbhecgh割草机....

但是我不知道如何按以下顺序对ascii和非ascii字符进行字典分类：AÁÃbrg Baq ckrwg CkfgF d Dgrn ...请告诉我如何编写代码。谢谢！

Answer 1

OP没有，但是我发现值得一提：说到非ASCII字符，也应该考虑编码。

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

[，Á和Â等字符不是7 bit ASCII的一部分，但以多种8位编码（例如Windows 1252。因此，不能保证某个字符（不是ASCII的一部分）在任何编码中都具有相同的代码点（即数字）。（大多数字符在大多数编码中都没有数字。）

但是，Unicode提供了一个唯一的编码表，其中包含任何其他编码的所有字符（我相信）。有实现为

[UTF-8，其中，代码点由1个或更多8位值表示（用char存储）
[UTF-16，其中代码点用1或2个16位值表示（用std::char16_t或也许是wchar_t存储）]
[UTF-32，其中的代码点用1 32位值表示（用std::char32_t进行存储，或者，如果具有足够的大小，则可以用wchar_t存储）。]]

关于wchar_t的大小：Character types。

话虽如此，我在示例中使用wchar_t和std::wstring来使变音符号语言环境和平台的使用独立。

std::sort()中用于排序std::sort()元素范围的顺序默认情况下是通过T和bool < operator(const T&, const T&)的<运算符定义的。但是，有T的样式来定义自定义谓词。

自定义谓词必须与签名匹配，并且必须提供std::sort()。

因此，我建议使用strict weak ordering relation将字符映射到按预期顺序生成的索引。

这是我在示例中使用的谓词：

std::map

从下至上：

[// sort words auto charIndex = [&mapChars](wchar_t chr) { const CharMap::const_iterator iter = mapChars.find(chr); return iter != mapChars.end() ? iter->second : (CharMap::mapped_type)mapChars.size(); }; auto pred = [&mapChars, &charIndex](const std::wstring &word1, const std::wstring &word2) { const size_t len = std::min(word1.size(), word2.size()); // + 1 to include zero terminator for (size_t i = 0; i < len; ++i) { const wchar_t chr1 = word1[i], chr2 = word2[i]; const unsigned i1 = charIndex(chr1), i2 = charIndex(chr2); if (i1 != i2) return i1 < i2; } return word1.size() < word2.size(); }; std::sort(words.begin(), words.end(), pred);用第三个参数调用，该参数为我的自定义订单提供谓词std::sort(words.begin(), words.end(), pred);。
lambda pred，逐个字符比较两个pred()。因此，比较是使用std::wstring std::map完成的，该mapChars wchar_t将unsigned映射到mapChars，即按照我的顺序将字符映射到其等级。
mapChars仅存储所有字符值的选择。因此，可能无法在charIndex()中找到任务中的角色。为了解决这个问题，使用了一个助手lambda mapChars.size()，在这种情况下，它返回CharMap –它被授予为高于所有出现的索引。

类型typedef只是一个typedef std::map<wchar_t, unsigned> CharMap;：

CharMap要初始化CharMap makeCharMap(const wchar_t *table[], size_t size)
{
  CharMap mapChars;
  unsigned rank = 0;
  for (const wchar_t **chars = table; chars != table + size; ++chars) {
    for (const wchar_t *chr = *chars; *chr; ++chr) mapChars[*chr] = rank;
    ++rank;
  }
  return mapChars;
}
，请使用函数：

const wchar_t *table[] = {
  L"aA", L"äÄ", L"bB", L"cC", L"dD", L"eE", L"fF", L"gG", L"hH", L"iI", L"jJ", L"kK", L"lL", L"mM", L"nN",
  L"oO", L"öÖ", L"pP", L"qQ", L"rR", L"sS", L"tT", L"uU", L"üÜ", L"vV", L"wW", L"xX", L"yY", L"zZ"
};
必须用一个字符串数组来调用，该字符串数组按预期顺序包含所有字符组：

#include <string>
#include <sstream>
#include <vector>

static const wchar_t *table[] = {
  L"aA", L"äÄ", L"bB", L"cC", L"dD", L"eE", L"fF", L"gG", L"hH", L"iI", L"jJ", L"kK", L"lL", L"mM", L"nN",
  L"oO", L"öÖ", L"pP", L"qQ", L"rR", L"sS", L"tT", L"uU", L"üÜ", L"vV", L"wW", L"xX", L"yY", L"zZ"
};

static const wchar_t *tableGerman[] = {
  L"aAäÄ", L"bB", L"cC", L"dD", L"eE", L"fF", L"gG", L"hH", L"iI", L"jJ", L"kK", L"lL", L"mM", L"nN",
  L"oOöÖ", L"pP", L"qQ", L"rR", L"sS", L"tT", L"uUüÜ", L"vV", L"wW", L"xX", L"yY", L"zZ"
};

typedef std::map<wchar_t, unsigned> CharMap;

// fill a look-up table to map characters to the corresponding rank
CharMap makeCharMap(const wchar_t *table[], size_t size)
{
  CharMap mapChars;
  unsigned rank = 0;
  for (const wchar_t **chars = table; chars != table + size; ++chars) {
    for (const wchar_t *chr = *chars; *chr; ++chr) mapChars[*chr] = rank;
    ++rank;
  }
  return mapChars;
}

// conversion to UTF-8 found in https://stackoverflow.com/a/7561991/7478597
// needed to print to console
// Please, note: std::codecvt_utf8() is deprecated in C++17. :-(
std::wstring_convert<std::codecvt_utf8<wchar_t>> utf8_conv;

// collect words and sort accoring to table
void printWordsSorted(
  const std::wstring &text, const wchar_t *table[], const size_t size)
{
  // make look-up table
  const CharMap mapChars = makeCharMap(table, size);
  // strip punctuation and other noise
  std::wstring textClean;
  for (const wchar_t chr : text) {
    if (chr == ' ' || mapChars.find(chr) != mapChars.end()) {
      textClean += chr;
    }
  }
  // fill word list with sample text
  std::vector<std::wstring> words;
  for (std::wistringstream in(textClean);;) {
    std::wstring word;
    if (!(in >> word)) break; // bail out
    // store word
    words.push_back(word);
  }
  // sort words
  auto charIndex = [&mapChars](wchar_t chr)
  {
    const CharMap::const_iterator iter = mapChars.find(chr);
    return iter != mapChars.end()
      ? iter->second
      : (CharMap::mapped_type)mapChars.size();
  };
  auto pred
    = [&mapChars, &charIndex](const std::wstring &word1, const std::wstring &word2)
  {
    const size_t len = std::min(word1.size(), word2.size());
    // + 1 to include zero terminator
    for (size_t i = 0; i < len; ++i) {
      const wchar_t chr1 = word1[i], chr2 = word2[i];
      const unsigned i1 = charIndex(chr1), i2 = charIndex(chr2);
      if (i1 != i2) return i1 < i2;
    }
    return word1.size() < word2.size();
  };
  std::sort(words.begin(), words.end(), pred);
  // remove duplicates
  std::vector<std::wstring>::iterator last = std::unique(words.begin(), words.end());
  words.erase(last, words.end());
  // print result
  for (const std::wstring &word : words) {
    std::cout << utf8_conv.to_bytes(word) << '\n';
  }
}

template<typename T, size_t N>
size_t size(const T (&arr)[N]) { return sizeof arr / sizeof *arr; }

int main()
{
  // a sample string
  std::wstring sampleText
    = L"In the German language the ä (a umlaut), ö (o umlaut) and ü (u umlaut)"
      L" have the same lexicographical rank as their counterparts a, o, and u.\n";
  std::cout << "Sample text:\n"
    << utf8_conv.to_bytes(sampleText) << '\n';
  // sort like requested by OP
  std::cout << "Words of text sorted as requested by OP:\n";
  printWordsSorted(sampleText, table, size(table));
  // sort like correct in German
  std::cout << "Words of text sorted as usual in German language:\n";
  printWordsSorted(sampleText, tableGerman, size(tableGerman));
}
完整示例：

Words of text sorted as requested by OP: a and as ä counterparts German have In language lexicographical o ö rank same the their u umlaut ü Words of text sorted as usual in German language: ä a and as counterparts German have In language lexicographical o ö rank same the their u ü umlaut

输出：

Live Demo on coliru

std::wcout

注意：

我的初衷是用simple way to convert wstrings to UTF-8进行输出。这对于ä，ö，ü不能正常工作。因此，我查找了wstring。我已经知道coliru支持UTF-8。

非ASCII字符的字典排序

问题描述投票：0回答：1

1个回答

最新问题

非ASCII字符的字典排序

问题描述 投票：0回答：1

1个回答

最新问题

问题描述投票：0回答：1