如何在Android 9上保持UTF-8的向后兼容性?

问题描述 投票:1回答:1

Android 9中引入的行为更改之一是更严格的UTF-8 decoder。如果我们有一个不正确的字节数组UTF-8字符串(例如随机字节或一些二进制数据)并尝试从中创建一个字符串:

return new String(bytes)

Android将选择UTF-8作为首选编码(这很好)但在Android 9上返回的结果与旧版Android相比略有不同。

我知道将随机字节转换为UTF-8字符串首先听起来不是一个好主意,但我现在需要向后兼容。

是否有选项可以在所有Android版本上获得完全相同的String结果?

编辑:

重现步骤:

    byte[] bytes =  new byte[]{25, 17, 113, 18, 62, 121, -6, -71, 45, -126, -113, 122, 58, 49, -30, -53, -66, -7, 0, -41};
    char[] password = new String(bytes).toCharArray();
    byte[] passKey = PBEParametersGenerator.PKCS12PasswordToBytes(password);
    Log.d("TEST", "Bytes: ".concat(Arrays.toString(passKey)));

Android <9.0的输出:

[0, 25, 0, 17, 0, 113, 0, 18, 0, 62, 0, 121, -1, -3, 0, 45, -1, -3, -1, -3, 0, 122, 0, 58, 0, 49, -1, -3, 2, -2, -1, -3, 0, 0, -1, -3, 0, 0]

Android 9.0的输出:

[0, 25, 0, 17, 0, 113, 0, 18, 0, 62, 0, 121, -1, -3, -1, -3, 0, 45, -1, -3, -1, -3, 0, 122, 0, 58, 0, 49, -1, -3, 2, -2, -1, -3, 0, 0, -1, -3, 0, 0]
java android encoding utf-8 android-9.0-pie
1个回答
1
投票

您可以使用此代码,该代码是从以前Android版本的UTF-8解码器移植而来的。

private static final char REPLACEMENT_CHAR = (char) 0xfffd;

public static char[] byteArrayToCharArray(byte[] data) {
    char[] value;
    final int offset = 0;
    final int byteCount = data.length;

    char[] v = new char[byteCount];
    int idx = offset;
    int last = offset + byteCount;
    int s = 0;
    outer:
    while (idx < last) {
        byte b0 = data[idx++];
        if ((b0 & 0x80) == 0) {
            // 0xxxxxxx
            // Range:  U-00000000 - U-0000007F
            int val = b0 & 0xff;
            v[s++] = (char) val;
        } else if (((b0 & 0xe0) == 0xc0) || ((b0 & 0xf0) == 0xe0) ||
                ((b0 & 0xf8) == 0xf0) || ((b0 & 0xfc) == 0xf8) || ((b0 & 0xfe) == 0xfc)) {
            int utfCount = 1;
            if ((b0 & 0xf0) == 0xe0) utfCount = 2;
            else if ((b0 & 0xf8) == 0xf0) utfCount = 3;
            else if ((b0 & 0xfc) == 0xf8) utfCount = 4;
            else if ((b0 & 0xfe) == 0xfc) utfCount = 5;
            // 110xxxxx (10xxxxxx)+
            // Range:  U-00000080 - U-000007FF (count == 1)
            // Range:  U-00000800 - U-0000FFFF (count == 2)
            // Range:  U-00010000 - U-001FFFFF (count == 3)
            // Range:  U-00200000 - U-03FFFFFF (count == 4)
            // Range:  U-04000000 - U-7FFFFFFF (count == 5)
            if (idx + utfCount > last) {
                v[s++] = REPLACEMENT_CHAR;
                continue;
            }
            // Extract usable bits from b0
            int val = b0 & (0x1f >> (utfCount - 1));
            for (int i = 0; i < utfCount; ++i) {
                byte b = data[idx++];
                if ((b & 0xc0) != 0x80) {
                    v[s++] = REPLACEMENT_CHAR;
                    idx--; // Put the input char back
                    continue outer;
                }
                // Push new bits in from the right side
                val <<= 6;
                val |= b & 0x3f;
            }
            // Note: Java allows overlong char
            // specifications To disallow, check that val
            // is greater than or equal to the minimum
            // value for each count:
            //
            // count    min value
            // -----   ----------
            //   1           0x80
            //   2          0x800
            //   3        0x10000
            //   4       0x200000
            //   5      0x4000000
            // Allow surrogate values (0xD800 - 0xDFFF) to
            // be specified using 3-byte UTF values only
            if ((utfCount != 2) && (val >= 0xD800) && (val <= 0xDFFF)) {
                v[s++] = REPLACEMENT_CHAR;
                continue;
            }
            // Reject chars greater than the Unicode maximum of U+10FFFF.
            if (val > 0x10FFFF) {
                v[s++] = REPLACEMENT_CHAR;
                continue;
            }
            // Encode chars from U+10000 up as surrogate pairs
            if (val < 0x10000) {
                v[s++] = (char) val;
            } else {
                int x = val & 0xffff;
                int u = (val >> 16) & 0x1f;
                int w = (u - 1) & 0xffff;
                int hi = 0xd800 | (w << 6) | (x >> 10);
                int lo = 0xdc00 | (x & 0x3ff);
                v[s++] = (char) hi;
                v[s++] = (char) lo;
            }
        } else {
            // Illegal values 0x8*, 0x9*, 0xa*, 0xb*, 0xfd-0xff
            v[s++] = REPLACEMENT_CHAR;
        }
    }
    if (s == byteCount) {
        // We guessed right, so we can use our temporary array as-is.
        value = v;
    } else {
        // Our temporary array was too big, so reallocate and copy.
        value = new char[s];
        System.arraycopy(v, 0, value, 0, s);
    }
    return value;
}

我通过创建一个20字节的随机数组进行单元测试,并与之前的Android版本上的原始new String(bytes).toCharArray()实现进行比较,然后重复这一百万次。我没有看到多个较旧的Android版本有任何差异。

原始源代码来自这里:https://android.googlesource.com/platform/libcore/+/a7752f4d22097346dd7849b92b9f36d0a0a7a8f3/libdvm/src/main/java/java/lang/String.java#245为了简化它,我删除了处理非UTF8字符集的部分,如果你使用new String(),那么UTF-8是使用的默认字符集,所以你应该没问题。

此代码将为您提供所需的向后兼容性。但正如其他人会建议你的那样,我建议在可能的情况下寻找更简单的解决方案,这不依赖于Android版本(或其他不受你控制的组件)

© www.soinside.com 2019 - 2024. All rights reserved.