在 python 函数上使用 pickle 时出现 AttributeError

Question

加载腌制函数时出现以下错误

{
  "errorMessage": "Can't get attribute 'customer_transform' on <module '__main__' from '/var/runtime/bootstrap.py'>",
  "errorType": "AttributeError",
  "stackTrace": [
    "  File \"/var/task/on_demand_dataset_load_and_transform/handler.py\", line 14, in lambda_handler\n    return transform_data(load_response, udf_s3_key)\n",
    "  File \"/var/task/on_demand_dataset_load_and_transform/transform_data.py\", line 26, in transform_data\n    udf = pickle.loads(udf_bytes)\n"
  ]
}

我正在使用jupyter笔记本编写一个函数，将其pickle并将其存储在S3中。

def customer_transform(input):
     # implementation

import pickle

def serialize_func(func):
    return pickle.dumps(func)

serialized_transform = serialize_func(customer_transform)

s3_client.put_object(Body=serialized_transform, Bucket="Bucket_name", Key="key")
    

当我尝试从 S3 读取数据并取消它时，出现上述错误。下面是代码

udf_bytes = s3_client.get_object(Bucket="Bucket_name", Key="key")["Body"].read()

downloaded_func = pickle.loads(udf_bytes)

Answer 1

我解决了这个导入问题

dill

import dill as pickle