从 Blob 存储中提取 tar.gz 文件并将提取的结果上传到另一个 Blob 存储

Question

我有这段代码可以获取 blob 存储文件并提取它，当我测试将结果保存在我的计算机上时它可以工作。

现在我不想将文件保存在本地计算机上，而是想将提取的结果上传到另一个 Blob 存储。我的挑战取决于如何将结果提取为字节或流，以便我可以上传，我确实尝试过这个，并且没有收到错误，但 blob 是 0 字节。任何建议表示赞赏。

var blobEndPointSrc = "https://storageaccountname.blob.core.windows.net/data-source-raw?etc...";
var blobContainerClientSrc = GetBlobServiceClientSAS(blobEndPointSrc);
var blobClientSrc = blobContainerClientSrc.GetBlobClient("file.tar.gz");
var blobDownloadInfo = blobClientSrc.OpenRead();
using Stream gzipStream = new GZipInputStream(blobDownloadInfo);

// test on local drive and works.
using var tarArchive = TarArchive.CreateInputTarArchive(gzipStream);
tarArchive.ExtractContents(@"D:\file");

现在尝试获取结果并将其上传到 Blob 存储：

// using var tarArchive = TarArchive.CreateInputTarArchive(gzipStream);
// tarArchive.ExtractContents(@"D:\file");

var tarBuffer = TarBuffer.CreateInputTarBuffer(gzipStream);
Stream result = new MemoryStream();
TarBuffer.CreateOutputTarBuffer(result);

var blobEndPointDst = = "https://storageaccountname2.blob.core.windows.net/data-source-raw?etc...";
var blobContainerClientDst = GetBlobServiceClientSAS(blobEndPointDst);
var blobClientDst = blobContainerClient1.GetBlobClient("extracted_file");
blobClientDst.Upload(result, true);

所有 blob 代码都可以正常上传和下载。但提取结果的转换最终在上传的文件中给出了 0 字节。

如果建议另一个提取库或其他解决方案也可以。

Answer 1

我尝试使用以下代码从Blob存储中提取tar.gz文件并将提取的结果上传到另一个Blob存储。

代码：

using System;
using System.IO;
using ICSharpCode.SharpZipLib.GZip;
using ICSharpCode.SharpZipLib.Tar;
using Azure.Storage.Blobs;

namespace ExtractAndUpload
{
    class Program
    {
        static void Main(string[] args)
        {
            string sourceBlobConnectionString = "<sourceblobconne_string>";
            string destinationBlobConnectionString = "<destinationblobconne_string>";

            string sourceContainerName = "<sourceContainer_name>";
            string sourceBlobName = "<blobname.tar.gz>";
            string destinationContainerName = "<destinationContainer_name>";
            string destinationBlobName = "<destinationBlob_name>";

            var sourceBlobServiceClient = new BlobServiceClient(sourceBlobConnectionString);
            var sourceBlobContainerClient = sourceBlobServiceClient.GetBlobContainerClient(sourceContainerName);
            var sourceBlobClient = sourceBlobContainerClient.GetBlobClient(sourceBlobName);

            var downloadResponse = sourceBlobClient.Download();
            using (Stream sourceStream = downloadResponse.Value.Content)
            {
                using (var gzipStream = new GZipInputStream(sourceStream))
                {
                    using (var tarInputStream = new TarInputStream(gzipStream))
                    {
                        TarEntry entry;
                        while ((entry = tarInputStream.GetNextEntry()) != null)
                        {
                            if (entry.IsDirectory)
                                continue;
                            using (var result = new MemoryStream())
                            {
                                tarInputStream.CopyEntryContents(result);
                                result.Position = 0;
                                var destinationBlobServiceClient = new BlobServiceClient(destinationBlobConnectionString);
                                var destinationBlobContainerClient = destinationBlobServiceClient.GetBlobContainerClient(destinationContainerName);
                                var destinationBlobClient = destinationBlobContainerClient.GetBlobClient(destinationBlobName);
                                destinationBlobClient.Upload(result, true);
                            }
                        }
                    }
                }
            }
            Console.WriteLine("Extraction and upload completed successfully.");
        }
    }
}

输出：

运行成功，如下图：

Extraction and upload completed successfully.

来源blob：

目的地斑点：

提取的文件数据已成功保存到目标blob，如下所示。