这是我第一次使用Polybase,我正尝试从ADLS Gen2将示例CSV文件(第一记录作为标题)加载到Synapse。我之前已经创建了一个万能钥匙,所以没有再创建它。我执行的其余步骤如下:
-- Step 1
CREATE DATABASE SCOPED CREDENTIAL access_cred
WITH
IDENTITY = 'my_name',
SECRET = '12345678910****==';
-- Step 2
CREATE EXTERNAL DATA SOURCE CreditCards
WITH
(
TYPE = HADOOP,
LOCATION = 'abfss://[email protected]',
CREDENTIAL = access_cred
);
-- Step 3
CREATE EXTERNAL FILE FORMAT CC_FileFormat
WITH
(
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS
(
FIELD_TERMINATOR = ',',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = FALSE
)
);
--Step 4
CREATE SCHEMA ext;
CREATE SCHEMA cc;
--Step 5: Create External Table
CREATE EXTERNAL TABLE ext.creditcards (
Card_Type_Full_Name varchar(50),
Issuing_Bank varchar(50),
Card_Number varchar(50),
Card_Holder_Name varchar(50),
CVV_CVV2 varchar(50),
Issue_Date varchar(50),
Expiry_Date varchar(50),
Billing_Date varchar(50),
Card_PIN varchar(50),
Credit_Limit varchar(50)
)
WITH (LOCATION='/CreditCards/', --I've a folder 'CreditCards' inside which the 'Creditcards.csv` file sits
DATA_SOURCE = CreditCards,
FILE_FORMAT = CC_FileFormat,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0
);
--Step 6
CREATE TABLE cc.creditcards
WITH
(
DISTRIBUTION = REPLICATE,
CLUSTERED COLUMNSTORE INDEX
)
AS
SELECT * FROM ext.creditcards
OPTION (LABEL = 'CTAS : Load cc.creditcards');
我不知道我在做什么错。我尝试通过互联网浏览多个帖子,但没有什么能真正解决我面临的问题。尝试执行SELECT * from <external_table>
时出现以下错误。 (这也是我在CTAS命令中加载最终表时在步骤6中遇到的错误。)
Msg 107090, Level 16, State 1, Line 74
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 1
我不明白还缺少什么。我还授予了容器级别的访问权限。有人可以帮我解决这个问题吗?
已解决: