更改AWS Athena查询的输出CSV文件名

问题描述 投票:0回答:1

我想通过AWS Lambda运行Athena查询,但也将输出CSV文件的名称从查询执行ID更改为my-bucket / folder / my-preferred-string.csv

我尝试在Web上搜索结果,但找不到lambda函数的确切代码。

我是AWS的数据科学家和初学者。对我来说这是一次性的事情,因此需要快速解决方案或修补程序。

amazon-athena
1个回答
0
投票

此问题已发布here

这里是您要寻找的lambda function的要点

client = boto3.client('athena')
s3 = boto3.resource("s3")

# Run query
queryStart = client.start_query_execution(
    # PUT_YOUR_QUERY_HERE
    QueryString = '''
        SELECT *
        FROM "db_name"."table_name"
        WHERE value > 50
    ''',
    QueryExecutionContext = {
        # YOUR_ATHENA_DATABASE_NAME
        'Database': "covid_data"
    },
    ResultConfiguration = {
        # query result output location you mentioned in AWS Athena
        "OutputLocation": "s3://bucket-name-X/folder-Y/"
    }
)

# Executes query and waits 3 seconds
queryId = queryStart['QueryExecutionId']
time.sleep(3)

# Copies newly generated csv file with appropriate name
# query result output location you mentioned in AWS Athena
queryLoc = "bucket-name-X/folder-Y/" + queryId + ".csv"

# Destination location and file name
s3.Object("bucket-name-A", "report-2018.csv").copy_from(CopySource = queryLoc)

# Deletes Athena generated csv and it's metadata file
response = s3.delete_object(
    Bucket='bucket-name-A',
    Key=queryId+".csv"
)
response = s3.delete_object(
    Bucket='bucket-name-A',
    Key=queryId+".csv.metadata"
)
print('{file-name} csv generated')
© www.soinside.com 2019 - 2024. All rights reserved.