ADLS Gen2上的多库-CSV文件作为源失败,出现'HdfsBridge :: recordReaderFillBuffer'错误

问题描述 投票:0回答:1

这是我第一次使用Polybase,我正尝试从ADLS Gen2将示例CSV文件(第一记录作为标题)加载到Synapse。我之前已经创建了一个万能钥匙,所以没有再创建它。我执行的其余步骤如下:

-- Step 1

CREATE DATABASE SCOPED CREDENTIAL access_cred
WITH
    IDENTITY = 'my_name',
    SECRET = '12345678910****==';

-- Step 2

CREATE EXTERNAL DATA SOURCE CreditCards
WITH
(
    TYPE = HADOOP,
    LOCATION = 'abfss://[email protected]',
    CREDENTIAL = access_cred
);

-- Step 3

CREATE EXTERNAL FILE FORMAT CC_FileFormat
WITH
(
    FORMAT_TYPE = DELIMITEDTEXT,
    FORMAT_OPTIONS
    (
        FIELD_TERMINATOR = ',',
        FIRST_ROW = 2,
        USE_TYPE_DEFAULT = FALSE
    )
);

--Step 4

CREATE SCHEMA ext;
CREATE SCHEMA cc;

--Step 5: Create External Table

CREATE EXTERNAL TABLE ext.creditcards (
    Card_Type_Full_Name varchar(50),
    Issuing_Bank varchar(50),
    Card_Number varchar(50),
    Card_Holder_Name varchar(50),
    CVV_CVV2 varchar(50),
    Issue_Date varchar(50),
    Expiry_Date varchar(50),
    Billing_Date varchar(50),
    Card_PIN varchar(50),
    Credit_Limit varchar(50)
)
WITH (LOCATION='/CreditCards/', --I've a folder 'CreditCards' inside which the 'Creditcards.csv` file sits
    DATA_SOURCE = CreditCards,  
    FILE_FORMAT = CC_FileFormat,
    REJECT_TYPE = VALUE,
    REJECT_VALUE = 0
);


--Step 6

CREATE TABLE cc.creditcards
WITH
(
    DISTRIBUTION = REPLICATE,
    CLUSTERED COLUMNSTORE INDEX
)
AS
SELECT * FROM ext.creditcards
OPTION (LABEL = 'CTAS : Load cc.creditcards');

我不知道我在做什么错。我尝试通过互联网浏览多个帖子,但没有什么能真正解决我面临的问题。尝试执行SELECT * from <external_table>时出现以下错误。 (这也是我在CTAS命令中加载最终表时在步骤6中遇到的错误。)

Msg 107090, Level 16, State 1, Line 74
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 1

我不明白还缺少什么。我还授予了容器级别的访问权限。有人可以帮我解决这个问题吗?

azure-storage-blobs azure-data-lake polybase azure-data-lake-gen2
1个回答
0
投票

已解决:

© www.soinside.com 2019 - 2024. All rights reserved.