在 sagemaker 管道中安装其他软件包

Question

我想安装将在处理步骤中使用的附加软件包。

sklearn_processor = FrameworkProcessor(
    estimator_cls=SKLearn,
    framework_version='0.23-1',
    instance_type="ml.t3.medium",
    instance_count=1,
    base_job_name="sklearn-abalone-process",
    sagemaker_session=sagemaker_session,
    role=role
)

outputs = [
    ProcessingOutput(output_name="train", source="/opt/ml/processing/train"),
    ProcessingOutput(output_name="validation", source="/opt/ml/processing/validation"),
    ProcessingOutput(output_name="test", source="/opt/ml/processing/test")
]

step_process = ProcessingStep(
    name="Preprocess_Data",
    processor = sklearn_processor.run(outputs=outputs,
        code="pre-process.py", dependencies=["/home/sagemaker-user/dependencies/requirements.txt"])
)

运行ProcessingStep后，我收到ValueError：需要给出step_args或processor，但不能两者都给出。

Sagemaker版本是2.197.0

我尝试查看 AWS 文档，但没有成功

Answer 1

试试这个：

processor_args = sklearn_processor.run(outputs=outputs,
        code="pre-process.py", dependencies=["/home/sagemaker-user/dependencies/requirements.txt"])

step_process = ProcessingStep(
    name="Preprocess_Data",
    step_args = processor_args
)