我正在使用 Sagemaker Lifecycle 配置运行笔记本,但遇到超时问题。我浏览了在线博客,了解如何运行需要 >5 分钟的脚本/代码。
发现使用
nohup
和 &
会将进程置于后台完成。但是我遇到了超时问题并且无法找出原因。
下面是我收到的脚本和错误
set -e
ENVIRONMENT=python3
NOTEBOOK_FILE="/home/ec2-user/SageMaker/mynotebook.ipynb"
AUTO_STOP_FILE="/home/ec2-user/SageMaker/auto-stop.py"
IDLE_TIME=300 # 5minute
echo "<>Activating conda env"
echo "<>PWD: $PWD"
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT"
echo "<>Installing packages"
pip install cloudscraper==1.2.58 bs4
pip install pandas
pip install boto3 s3fs
echo "<>Starting notebook"
nohup jupyter nbconvert "$NOTEBOOK_FILE" --ExecutePreprocessor.kernel_name=python3 --to notebook --inplace --ExecutePreprocessor.timeout=-1 --execute &
echo "<>Decativating conda env"
source deactivate
# PARAMETERS
echo "<>Fetching the autostop script"
wget https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/master/scripts/auto-stop-idle/autostop.py
echo "<>Starting the SageMaker autostop script in cron"
(crontab -l 2>/dev/null; echo "*/1 * * * * /usr/bin/python $PWD/autostop.py --time $IDLE_TIME --ignore-connections") | crontab -
"
错误-
失败原因Notebook实例生命周期配置 'arn:aws:sagemaker:ap-south-1:588770669142:notebook-instance-lifecycle-config/web-scraping-lifecycle' 对于笔记本实例 'arn:aws:sagemaker:ap-south-1:588770669142:notebook-instance/web-scraping-routines-lambda' 花费了超过5分钟的时间。请检查您的 CloudWatch 日志以了解更多信息 您的笔记本实例是否可以访问互联网的详细信息。
尝试一下,这对我来说很有效
jupyter nbconvert --to html "$NOTEBOOK_FILE" --ExecutePreprocessor.kernel_name=python3 ExecutePreprocessor.timeout=-1 --execute
对于那些收到错误的人来说,可能是这样的情况: