我正在尝试将AWS Glue设置为使用CloudFormation从RDS Postgres读取。为此,我需要使用JdbcTarget选项创建一个搜寻器。 (还是我不?)
Records:
Type: 'AWS::Glue::Crawler'
Properties:
DatabaseName: transact
Targets:
JdbcTargets:
- Path: "jdbc:postgresql://host:5432/database"
Role: !Ref ETLAgent
但是在CloudFormation上创建堆栈将失败:
CREATE_FAILED | AWS::Glue::Crawler | Records | Connection name cannot be equal to null or empty. (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException;
即使the docs说:
ConnectionName
用于JDBC目标的连接名称。
必填:否
使用CloudFormation的正确的AWS Glue设置是什么,这将允许我从RDS进行读取?
您确实缺少ConnectionName
属性,该属性应带有您缺少的连接资源的名称。您正在设置的Path
属性用于选择要爬网的模式/表(dbname/%/%
包括所有)。有关详情,请参阅CloudFormation docs on Crawler JDBCTarget。
您的模板应类似于
MyDbConnection:
Type: "AWS::Glue::Connection"
Properties:
CatalogId: !Ref 'AWS::AccountId'
ConnectionInput:
Description: "JDBC Connection to my RDS DB"
PhysicalConnectionRequirements:
AvailabilityZone: "eu-central-1a"
SecurityGroupIdList:
- my-sec-group-id
SubnetId: my-subnet-id
ConnectionType: "JDBC"
ConnectionProperties:
"JDBC_CONNECTION_URL": "jdbc:postgresql://host:5432/database"
"USERNAME": "my-db-username"
"PASSWORD": "my-password"
Records:
Type: 'AWS::Glue::Crawler'
Properties:
DatabaseName: transact
Targets:
JdbcTargets:
- ConnectionName: !Ref MyDbConnection
Path: "database/%/%"
Role: !Ref ETLAgent