DummyVecEnv 与 easyrl : AttributeError: 'tuple' 对象没有属性 'shape'

Question

我正在为德州扑克（扑克）创建一个 PPO 代理，以下是我启动代理的方法：

    def initiate_agent(self, env):
        self.envwrapped = DummyVecEnv([lambda: env])
        ob_size = env.observation_space[0]
        
        actor_body = MLP(input_size=ob_size,
                        hidden_sizes=[256] * 3,
                        output_size=256,
                        hidden_act=nn.Tanh,
                        output_act=nn.Tanh).to(self.device)

        critic_body = MLP(input_size=ob_size,
                        hidden_sizes=[256] * 3,
                        output_size=256,
                        hidden_act=nn.Tanh,
                        output_act=nn.Tanh).to(self.device)


        if isinstance(env.action_space, gym.spaces.Discrete):
            act_size = env.action_space.n
            self.actor = CategoricalPolicy(actor_body,in_features=256,action_dim=act_size)

        elif isinstance(env.action_space, gym.spaces.Box):
            act_size = env.action_space.shape[0]
            self.actor = DiagGaussianPolicy(actor_body,in_features=256,
                                            action_dim=act_size, 
                                            tanh_on_dist=cfg.alg.tanh_on_dist,
                                            std_cond_in=cfg.alg.std_cond_in)

        else:
            raise TypeError(f'Unknown action space type: {env.action_space}')

        
        self.critic = ValueNet(critic_body, in_features=256)
        self.agent = PPOAgent(actor=self.actor, critic=self.critic, env=self.envwrapped)

但是，我收到错误：

Traceback (most recent call last):
  File "main.py", line 394, in <module>
    command_line_parser()
  File "main.py", line 129, in command_line_parser
    runner.ppo_train(model_name, random = random)
  File "main.py", line 286, in ppo_train
    ppoAgent.initiate_agent(env)
  File "C:\Users\user\Desktop\AlphaHoldem-main\neuron_poker\agents\agent_ppo.py", line 63, in initiate_agent
    self.envwrapped = DummyVecEnv([lambda: env])
  File "C:\Users\user\anaconda3\envs\neuron_poker\lib\site-packages\easyrl\envs\dummy_vec_env.py", line 30, in __init__
    self.keys, shapes, dtypes = obs_space_info(obs_space)
  File "C:\Users\user\anaconda3\envs\neuron_poker\lib\site-packages\easyrl\utils\vec_env_util.py", line 50, in obs_space_info
    shapes[key] = box.shape
AttributeError: 'tuple' object has no attribute 'shape'

看来问题是

self.envwrapped = DummyVecEnv([lambda: env])

，但我在网上找不到任何解决方案。有什么解决办法吗？非常感谢任何反馈！！！

Answer 1

这是考虑元组观察空间的修改版本：

def initiate_agent(self, env):
    if isinstance(env.observation_space, gym.spaces.Box):
        self.envwrapped = DummyVecEnv([lambda: env])
    elif isinstance(env.observation_space, gym.spaces.Tuple):
        self.envwrapped = DummyVecEnv([lambda: env for _ in range(len(env.observation_space))])
    else:
        raise TypeError(f'Unknown observation space type: {env.observation_space}')

    ob_size = env.observation_space[0].shape[0]  # Update this based on your actual observation space

    # Rest of your code remains unchanged
    actor_body = MLP(input_size=ob_size,
                    hidden_sizes=[256] * 3,
                    output_size=256,
                    hidden_act=nn.Tanh,
                    output_act=nn.Tanh).to(self.device)

您需要检查

env.observation_space

的类型并进行相应处理。如果它是一个元组，您可能需要修改

initiate_agent

方法来处理这种情况。