在我之前的问题中（在处理文件后，C# 中的 RAM 没有被释放）我询问了一种清除 RAM 的方法。有人建议使用流而不是将其读入变量。 我发现加密/解密大文件（.NET）使用流，但没有使用AesGcm。 问题是我找不到如何将 AesGcm 与流一起使用。 AesGcm.decrypt 仅接受密文字段中的 Byte[]， 并且 AesManaged 没有 CihperMode.GCM。
目前，解密 800MB 的文件需要 4GB 的 RAM。 如何在不填充 RAM 的情况下使用 AesGcm 解密文件？
谢谢。
我会说.NET中的
AesGcmAesCcmAesGcm如果可能的话你应该改变算法。否则可以找到其他解决方案。 Bouncycastle 库支持 AesGcm。
我发布一个非流式答案，因为我有一个相当不错的 AesGcm 低分配实现，可以满足您的需求。您可以将
ArraySegment<byte>如果您需要使用替代解密机制，字节结构是直接的。
12 bytes 16 bytes n bytes up to int.IntMax - 28 bytes.
[ Nonce / IV ][ Tag / MAC ][ Ciphertext ]
链接到我的 Github 存储库。
使用示例）
// HashKey or PassKey or Passphrase in 16/24/32 byte format.
var encryptionProvider = new AesGcmEncryptionProvider(hashKey, "ARGON2ID");
// This is an ArraySegment<byte>, this allows a defer allocation of byte[]
var encryptedData = encryptionProvider.Encrypt(_data);
// When you are ready for a byte[]
encryptedData.ToArray()
// You can also use
encryptedData.Array
// but this is a buffer and often exceeds the actual size of your bytes.
// Use conscientiously but does prevent a copy / allocation of the bytes.
// To Decrypt - same return type in case you need to serialize / decompress etc.
var decryptedData = encryptionProvider.Decrypt(encryptedData);
decrypted.ToArray() // the proper decrypted bytes of data.
decrypted.Array // the buffer used.
// Convert to a string
Encoding.UTF8.GetString(_encryptedData.ToArray())
如果您发现任何问题，请告诉我，很乐意进行更改/修复 - 或者更好地在 Github 上提交问题/PR，以便我可以保持真实代码最新。
基准
// * Summary *
BenchmarkDotNet=v0.13.0, OS=Windows 10.0.18363.1556 (1909/November2019Update/19H2)
Intel Core i7-9850H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=5.0.203
[Host] : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
Job-ADZLQM : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
.NET 5.0 : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
Runtime=.NET 5.0
| Method | Job | IterationCount | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------------ |----------- |--------------- |--------------:|--------------:|--------------:|--------------:|------:|--------:|----------:|---------:|---------:|----------:|
| Encrypt1KBytes | .NET 5.0 | Default | 1.512 us | 0.0298 us | 0.0398 us | 1.504 us | 1.00 | 0.00 | 0.1926 | - | - | 1 KB |
| Encrypt2KBytes | .NET 5.0 | Default | 1.965 us | 0.0382 us | 0.0408 us | 1.951 us | 1.30 | 0.04 | 0.3548 | - | - | 2 KB |
| Encrypt4kBytes | .NET 5.0 | Default | 2.946 us | 0.0583 us | 0.0942 us | 2.948 us | 1.96 | 0.07 | 0.6828 | - | - | 4 KB |
| Encrypt8KBytes | .NET 5.0 | Default | 4.630 us | 0.0826 us | 0.0733 us | 4.631 us | 3.09 | 0.08 | 1.3351 | - | - | 8 KB |
| Decrypt1KBytes | .NET 5.0 | Default | 1.234 us | 0.0247 us | 0.0338 us | 1.216 us | 0.82 | 0.03 | 0.1869 | - | - | 1 KB |
| Decrypt2KBytes | .NET 5.0 | Default | 1.644 us | 0.0328 us | 0.0378 us | 1.630 us | 1.09 | 0.04 | 0.3510 | - | - | 2 KB |
| Decrypt4kBytes | .NET 5.0 | Default | 2.462 us | 0.0274 us | 0.0214 us | 2.460 us | 1.64 | 0.04 | 0.6752 | - | - | 4 KB |
| Decrypt8KBytes | .NET 5.0 | Default | 4.167 us | 0.0828 us | 0.1016 us | 4.179 us | 2.76 | 0.12 | 1.3275 | - | - | 8 KB |
代码及时快照。
public class AesGcmEncryptionProvider : IEncryptionProvider
{
/// <summary>
/// Safer way of generating random bytes.
/// https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.rngcryptoserviceprovider?redirectedfrom=MSDN&view=net-5.0
/// </summary>
private readonly RNGCryptoServiceProvider _rng = new RNGCryptoServiceProvider();
private readonly ArrayPool<byte> _pool = ArrayPool<byte>.Create();
private readonly byte[] _key;
public string Type { get; private set; }
public AesGcmEncryptionProvider(byte[] key, string hashType)
{
if (!Constants.Aes.ValidKeySizes.Contains(key.Length)) throw new ArgumentException("Keysize is an invalid length.");
_key = key;
switch (_key.Length)
{
case 16: Type = "AES128"; break;
case 24: Type = "AES192"; break;
case 32: Type = "AES256"; break;
}
if (!string.IsNullOrWhiteSpace(hashType)) { Type = $"{hashType}-{Type}"; }
}
public ArraySegment<byte> Encrypt(ReadOnlyMemory<byte> data)
{
using var aes = new AesGcm(_key);
// Slicing Version
// Rented arrays sizes are minimums, not guarantees.
// Need to perform extra work managing slices to keep the byte sizes correct but the memory allocations are lower by 200%
var encryptedBytes = _pool.Rent(data.Length);
var tag = _pool.Rent(AesGcm.TagByteSizes.MaxSize); // MaxSize = 16
var nonce = _pool.Rent(AesGcm.NonceByteSizes.MaxSize); // MaxSize = 12
_rng.GetBytes(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
aes.Encrypt(
nonce.AsSpan().Slice(0, AesGcm.NonceByteSizes.MaxSize),
data.Span,
encryptedBytes.AsSpan().Slice(0, data.Length),
tag.AsSpan().Slice(0, AesGcm.TagByteSizes.MaxSize));
// Prefix ciphertext with nonce and tag, since they are fixed length and it will simplify decryption.
// Our pattern: Nonce Tag Cipher
// Other patterns people use: Nonce Cipher Tag // couldn't find a solid source.
var encryptedData = new byte[AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize + data.Length];
Buffer.BlockCopy(nonce, 0, encryptedData, 0, AesGcm.NonceByteSizes.MaxSize);
Buffer.BlockCopy(tag, 0, encryptedData, AesGcm.NonceByteSizes.MaxSize, AesGcm.TagByteSizes.MaxSize);
Buffer.BlockCopy(encryptedBytes, 0, encryptedData, AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize, data.Length);
_pool.Return(encryptedBytes);
_pool.Return(tag);
_pool.Return(nonce);
return encryptedData;
}
public async Task<MemoryStream> EncryptAsync(Stream data)
{
using var aes = new AesGcm(_key);
var buffer = _pool.Rent((int)data.Length);
var bytesRead = await data
.ReadAsync(buffer.AsMemory(0, (int)data.Length))
.ConfigureAwait(false);
if (bytesRead == 0) throw new InvalidDataException();
// Slicing Version
// Rented arrays sizes are minimums, not guarantees.
// Need to perform extra work managing slices to keep the byte sizes correct but the memory allocations are lower by 200%
var encryptedBytes = _pool.Rent((int)data.Length);
var tag = _pool.Rent(AesGcm.TagByteSizes.MaxSize); // MaxSize = 16
var nonce = _pool.Rent(AesGcm.NonceByteSizes.MaxSize); // MaxSize = 12
_rng.GetBytes(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
aes.Encrypt(
nonce.AsSpan().Slice(0, AesGcm.NonceByteSizes.MaxSize),
buffer.AsSpan().Slice(0, (int)data.Length),
encryptedBytes.AsSpan().Slice(0, (int)data.Length),
tag.AsSpan().Slice(0, AesGcm.TagByteSizes.MaxSize));
// Prefix ciphertext with nonce and tag, since they are fixed length and it will simplify decryption.
// Our pattern: Nonce Tag Cipher
// Other patterns people use: Nonce Cipher Tag // couldn't find a solid source.
var encryptedStream = new MemoryStream(new byte[AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize + (int)data.Length]);
using (var binaryWriter = new BinaryWriter(encryptedStream, Encoding.UTF8, true))
{
binaryWriter.Write(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
binaryWriter.Write(tag, 0, AesGcm.TagByteSizes.MaxSize);
binaryWriter.Write(encryptedBytes, 0, (int)data.Length);
}
_pool.Return(buffer);
_pool.Return(encryptedBytes);
_pool.Return(tag);
_pool.Return(nonce);
encryptedStream.Seek(0, SeekOrigin.Begin);
return encryptedStream;
}
public MemoryStream EncryptToStream(ReadOnlyMemory<byte> data)
{
return new MemoryStream(Encrypt(data).ToArray());
}
public ArraySegment<byte> Decrypt(ReadOnlyMemory<byte> encryptedData)
{
using var aes = new AesGcm(_key);
// Slicing Version
var nonce = encryptedData
.Slice(0, AesGcm.NonceByteSizes.MaxSize)
.Span;
var tag = encryptedData
.Slice(AesGcm.NonceByteSizes.MaxSize, AesGcm.TagByteSizes.MaxSize)
.Span;
var encryptedBytes = encryptedData
.Slice(AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize)
.Span;
var decryptedBytes = new byte[encryptedBytes.Length];
aes.Decrypt(nonce, encryptedBytes, tag, decryptedBytes);
return decryptedBytes;
}
public MemoryStream Decrypt(Stream stream)
{
using var aes = new AesGcm(_key);
using var binaryReader = new BinaryReader(stream);
var nonce = binaryReader.ReadBytes(AesGcm.NonceByteSizes.MaxSize);
var tag = binaryReader.ReadBytes(AesGcm.TagByteSizes.MaxSize);
var encryptedBytes = binaryReader.ReadBytes((int)binaryReader.BaseStream.Length - AesGcm.NonceByteSizes.MaxSize - AesGcm.TagByteSizes.MaxSize);
var decryptedBytes = new byte[encryptedBytes.Length];
aes.Decrypt(nonce, encryptedBytes, tag, decryptedBytes);
return new MemoryStream(decryptedBytes);
}
public MemoryStream DecryptToStream(ReadOnlyMemory<byte> data)
{
return new MemoryStream(Decrypt(data).ToArray());
}
}
事情是这样的：
using var aes = new AesGcm(_key);
using FileStream fs = new(<path to file>, FileMode.Open);
int bytesRead;
while ((bytesRead = fs.Read(buffer)) > 0)
{
aes.Encrypt(nonce, buffer[..bytesRead], buffer[..bytesRead], tag);
using var encfs = new FileStream($@"{path to output file}.enc", FileMode.Append);
encfs.Write(_salt);
encfs.Write(nonce);
encfs.Write(buffer[..bytesRead]);
encfs.Write(tag);
}
这将打开文件流，根据缓冲区大小通过缓冲区流式传输文件，并将生成的密码写入同一文件。此阶段的内存应该采用缓冲区大小+来自程序中当时处于活动状态的各种对象的非常小的东西。
我在加载例如 3 GB 文件、对其进行加密并使内存良好且清晰的过程中遇到问题，上面的代码正好提供了这一点，没有任何问题。
有一个 .NET 库 StreamingAead
dotnet add package StreamingAead
从描述来看：
用于对大型数据集进行流式加密和解密的库，基于 Google 的 Streaming AEAD 的 Tink 协议。它采用 AES256_GCM_HKDF_1MB 进行分段 AES-256 GCM 加密，确保数据安全，重点关注效率和与 Tink 的互操作性。这是一个例子：
byte[] key = File.ReadAllBytes(args.KeyPath);
byte[] associatedDataBytes = Encoding.UTF8.GetBytes(args.AssociatedData);
using (var inputStream = File.OpenRead(args.InputPath))
using (var outputStream = File.OpenWrite(args.OutputPath))
{
if (args.Mode == "encrypt")
{
AES256_GCM_HKDF_1MB.Encrypt(key, inputStream, outputStream, associatedDataBytes);
}
else if (args.Mode == "decrypt")
{
AES256_GCM_HKDF_1MB.Decrypt(key, inputStream, outputStream, associatedDataBytes);
}
else
{
Console.WriteLine($"Invalid mode: {args.Mode}");
return;
}
}