我正在使用 boostrap 文件为运行 Pyspark 的 EMR 项目安装 python 3.10。 Python 3.10 需要 openSSL 1.1 或更高版本才能使用，但 EMR 有 OpenSSL 1.0.2k-fips（运行命令
openssl version我尝试使用 yum 安装我需要的版本：
sudo yum install -y openssl11-devel <+_some_other_packages>但是 EMR 有一些存储库优先级保护，阻止我安装 1.1 版本：
13 packages excluded due to repository priority protections
Package gcc-7.3.1-15.amzn2.x86_64 already installed and latest version
Package 2:tar-1.26-35.amzn2.0.2.x86_64 already installed and latest version
Package gzip-1.5-10.amzn2.0.1.x86_64 already installed and latest version
Package wget-1.14-18.amzn2.1.x86_64 already installed and latest version
Package 1:make-3.82-24.amzn2.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package bzip2-devel.x86_64 0:1.0.6-13.amzn2.0.3 will be installed
---> Package libffi-devel.x86_64 0:3.0.13-18.amzn2.0.2 will be installed
---> Package openssl11-devel.x86_64 1:1.1.1g-12.amzn2.0.15 will be installed
--> Processing Conflict: 1:openssl11-devel-1.1.1g-12.amzn2.0.15.x86_64 conflicts openssl-devel
--> Finished Dependency Resolution
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
然后，我尝试使用上述建议，使用标志 --skip-broken 并禁用优先级检查：
sudo yum install -y --disableplugin=priorities --skip-broken openssl11-devel <+ some other packages>日志显示由于依赖问题，它跳过了 1.1 版本：
Resolving Dependencies
--> Running transaction check
---> Package bzip2-devel.x86_64 0:1.0.6-13.amzn2.0.3 will be installed
---> Package libffi-devel.x86_64 0:3.0.13-18.amzn2.0.2 will be installed
---> Package openssl11-devel.x86_64 1:1.1.1g-12.amzn2.0.15 will be installed
--> Processing Conflict: 1:openssl11-devel-1.1.1g-12.amzn2.0.15.x86_64 conflicts openssl-devel
--> Running transaction check
---> Package openssl11-devel.x86_64 1:1.1.1g-12.amzn2.0.15 will be installed
--> Running transaction check
---> Package bzip2-devel.x86_64 0:1.0.6-13.amzn2.0.3 will be installed
---> Package libffi-devel.x86_64 0:3.0.13-18.amzn2.0.2 will be installed
--> Finished Dependency Resolution
Packages skipped because of dependency problems:
1:openssl11-devel-1.1.1g-12.amzn2.0.15.x86_64 from amzn2-core
最后，我尝试在安装新版本1.1之前卸载当前版本1.0：
sudo yum remove openssl日志显示在依赖项检查后它已被删除：
================================================================================
Package Arch Version Repository Size
================================================================================
Removing:
openssl x86_64 1:1.0.2k-24.amzn2.0.7 installed 830 k
Removing for dependencies:
authconfig x86_64 6.2.8-30.amzn2.0.2 installed 2.2 M
cloud-init noarch 19.3-46.amzn2.0.1 installed 3.2 M
ec2-instance-connect noarch 1.1-19.amzn2 installed 23 k
perl-IO-Socket-SSL noarch 1.94-7.amzn2.0.1 @amzn2-core 216 k
perl-Net-HTTP noarch 6.06-2.amzn2 @amzn2-core 51 k
perl-XML-LibXML x86_64 1:2.0018-5.amzn2.0.2 @amzn2-core 940 k
perl-XML-SAX noarch 0.99-9.amzn2 @amzn2-core 125 k
perl-libwww-perl noarch 6.05-2.amzn2 @amzn2-core 487 k
platform-startup-install noarch 1.17.0-1 @emr-platform-repo 2.0 k
python-jwcrypto noarch 0.4.2-1.amzn2 installed 249 k
python2-cryptography x86_64 1.7.2-2.amzn2 installed 2.6 M
python2-oauthlib noarch 2.0.1-8.amzn2.0.1 installed 780 k
rng-tools x86_64 6.8-3.amzn2.0.5 installed 102 k
system-lsb x86_64 4.1-27.amzn2.3.6 @amzn2-core 0.0
system-lsb-languages x86_64 4.1-27.amzn2.3.6 @amzn2-core 814
Transaction Summary
================================================================================
Remove 1 Package (+15 Dependent packages)
但是日志在这里停止，引导程序在此时冻结一小时，直到失败。我看不到它是否尝试安装 1.1 版本（此步骤之后出现）。
所以我的问题是：如何在 EMR 上安装/升级当前版本的 OpenSSL 到 1.1 以运行 pyhton 3.10？
我的引导文件如下：
#!/usr/bin/env bash
PYTHON_VERSION=3.10.6
echo
echo "-----------------------------------------------"
echo "Removing SSL"
echo "-----------------------------------------------"
echo
echo "Current SSL version:"
openssl version
sudo yum update -y
sudo yum remove openssl
echo
echo "-----------------------------------------------"
echo "Installing OpenSSL"
echo "-----------------------------------------------"
echo
sudo yum update -y
sudo yum install -y --disableplugin=priorities --skip-broken gcc openssl11-devel bzip2-devel libffi-devel tar gzip wget make
echo "New SSL version:"
openssl version
echo
echo "-----------------------------------------------"
echo "Installing Python ${PYTHON_VERSION}"
echo "-----------------------------------------------"
echo
wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz
tar xzf Python-${PYTHON_VERSION}.tgz
cd Python-${PYTHON_VERSION}
./configure --enable-optimizations
make install
make altinstall prefix=~/local # specify local installation directory
ln -s ~/local/bin/python3.10 ~/local/bin/python
cd ..
我在基于 EMR emr-on-eks/spark/emr-6.14.0 映像的 dockerfile 中安装 python 3.10 时解决了类似的问题。
步骤如下：
yum install -y git wget gcc make patch zlib-devel bzip2 bzip2-devel \
readline-devel sqlite sqlite-devel openssl11 openssl11-devel tk-devel libffi-devel xz-devel
这里最重要的是：openssl11 和 openssl11-devel。并且不要安装 openssl-devel。
PYTHON_VERSION=3.10.13
curl -O https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz
tar -xvzf Python-${PYTHON_VERSION}.tgz
cd Python-${PYTHON_VERSION}
./configure --enable-optimizations
make altinstall
ln -s /usr/local/bin/python3.10 /usr/local/bin/python3
通过使用 make altinstal，您可以将操作系统附带的 python 安装保留在 /usr/bin/python 中。使用 altinstall python 被单独安装到 usr/local/bin/ 中。 pip 可以参考新的 python 版本 python3.10 和 pip3.10 （当然将 3.10 替换为您想要安装的适当版本的 python）
或者，您可以使用命令
make installmake altinstall