OpenAIgymSuperMarioBros 对象没有属性“render_mode”

Question

我正在尝试使用 stable-baselines3 PPO 模型来训练代理玩健身房超级马里奥兄弟，但当它运行时，这是基本模型训练代码：

from nes_py.wrappers import JoypadSpace
import gym_super_mario_bros
from gym_super_mario_bros.actions import SIMPLE_MOVEMENT
import time
from matplotlib import pyplot as plt
from stable_baselines3 import PPO
import gym


env = gym_super_mario_bros.make('SuperMarioBros-v0')
env = JoypadSpace(env, SIMPLE_MOVEMENT)


tensorboard_log = r'./tensorboard_log/'

model = PPO("CnnPolicy", env, verbose=1,tensorboard_log = tensorboard_log)
model.learn(total_timesteps=25000)
model.save("mario_model")

我收到了这个错误：

Using cpu device
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-73-f912901bbdee> in <cell line: 6>()
      4 
      5 #model = PPO("CnnPolicy", env, verbose=1,tensorboard_log = tensorboard_log)
----> 6 model = PPO('MlpPolicy', env, verbose=1)
      7 model.learn(total_timesteps=25000)
      8 model.save("mario_model")

5 frames
/usr/local/lib/python3.10/dist-packages/shimmy/openai_gym_compatibility.py in __init__(self, env_id, make_kwargs, env)
     88 
     89         self.metadata = getattr(self.gym_env, "metadata", {"render_modes": []})
---> 90         self.render_mode = self.gym_env.render_mode
     91         self.reward_range = getattr(self.gym_env, "reward_range", None)
     92         self.spec = getattr(self.gym_env, "spec", None)

AttributeError: 'SuperMarioBrosEnv' object has no attribute 'render_mode'

我正在使用colab， stable-baselines3 版本是 2.0.0 健身房超级马里奥兄弟 7.4.0 健身房0.23.1 我尝试安装旧版本的 stable-baselines3，但得到了这个：

Collecting stable-baselines3==1.5.0
  Using cached stable_baselines3-1.5.0-py3-none-any.whl (177 kB)
Collecting gym==0.21 (from stable-baselines3==1.5.0)
  Using cached gym-0.21.0.tar.gz (1.5 MB)
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

我是强化学习的初学者，希望你能帮助我。

Answer 1

同步nes-py(8.2.1)和gym(0.26.0+)存在错误

建议使用

gym==0.21.0
stable_baselines==1.16.0

安装上述要求的问题是它不适用于 setuptools 0.67+ 和wheel>0.40.0

所以建议使用

!pip install setuptools==65.5.0 "wheel<0.40.0"

完整的解决方案是运行以下命令：

!pip install nes-py
!pip install gym-super-mario-bros==7.3.0
!pip install setuptools==65.5.0 "wheel<0.40.0"
!pip install gym==0.21.0
!pip install stable-baselines3[extra]==1.6.0