Gymnasium Release Notes#

v0.28.1#

Released on 2023-03-25 - GitHub - PyPI

v0.28.1 Release notes

Small emergency release to fix several issues

  • Fixed gymnasium.vector as the gymnasium/__init__.py as it isn't imported #403
  • Update third party envs to separate environments that support gymnasium and gym and have a consistent style #404
  • Update the documentation for v0.28 as frontpage gif had the wrong link, experimental documentation was missing and add gym release notes #405

Full Changelog: v0.28.0...v0.28.1

v0.28.0#

Released on 2023-03-24 - GitHub - PyPI

v0.28.0 Release notes

This release introduces improved support for the reproducibility of Gymnasium environments, particularly for offline reinforcement learning. gym.make can now create the entire environment stack, including wrappers, such that training libraries or offline datasets can specify all of the arguments and wrappers used for an environment. For a majority of standard usage (gym.make(”EnvironmentName-v0”)), this will be backwards compatible except for certain fairly uncommon cases (i.e. env.spec and env.unwrapped.spec return different specs) this is a breaking change. See the reproducibility details section for more info.
In v0.27, we added the experimental folder to allow us to develop several new features (wrappers and hardware accelerated environments). We’ve introduced a new experimental VectorEnv class. This class does not inherit from the standard Env class, and will allow for dramatically more efficient parallelization features. We plan to improve the implementation and add vector based wrappers in several minor releases over the next few months.
Additionally, we have optimized module loading so that PyTorch or Jax are only loaded when users import wrappers that require them, not on import gymnasium.

Reproducibility details

In previous versions, Gymnasium supported gym.make(spec) where the spec is an EnvSpec from gym.spec(str) or env.spec and worked identically to the string based gym.make(“”). In both cases, there was no way to specify additional wrappers that should be applied to an environment. With this release, we added additional_wrappers to EnvSpec for specifying wrappers applied to the base environment (TimeLimit, PassiveEnvChecker, Autoreset and ApiCompatibility are not included as they are specify in other fields).
This additional field will allow users to accurately save or reproduce an environment used in training for a policy or to generate an offline RL dataset. We provide a json converter function (EnvSpec.to_json) for saving the EnvSpec to a “safe” file type however there are several cases (NumPy data, functions) which cannot be saved to json. In these cases, we recommend pickle but be warned that this can allow remote users to include malicious data in the spec.

import gymnasium as gym

env = gym.make("CartPole-v0")
env = gym.wrappers.TimeAwareObservation(env)
print(env)  
# <TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
env_spec = env.spec
env_spec.pprint()
# id=CartPole-v0
# reward_threshold=195.0
# max_episode_steps=200
# additional_wrappers=[
# 	name=TimeAwareObservation, kwargs={}
# ]

import json
import pickle

json_env_spec = json.loads(env_spec.to_json())
pickled_env_spec = pickle.loads(pickle.dumps(env_spec))
recreated_env = gym.make(json_env_spec)
print(recreated_env)  
# <TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
# Be aware that the `TimeAwareObservation` was included by `make`

To support this type of recreation, wrappers must inherit from gym.utils.RecordConstructorUtils to allow gym.make to know what arguments to create the wrapper with. Gymnasium has implemented this for all built-in wrappers but for external projects, should be added to each wrapper. To do this, call gym.utils.RecordConstructorUtils.__init__(self, …) in the first line of the wrapper’s constructor with identical l keyword arguments as passed to the wrapper’s constructor, except for env. As an example see the Atari Preprocessing wrapper
For a more detailed discussion, see the original PRs - #292 and #355

Other Major Changes

  • In Gymnasium v0.26, the GymV22Compatibility environment was added to support Gym-based environments in Gymnasium. However, the name was incorrect as the env supported Gym’s v0.21 API, not v0.22, therefore, we have updated it to GymV21Compatibility to accurately reflect the API supported. #282
  • The Sequence space allows for a dynamic number of elements in an observation or action space sample. To make this more efficient, we added a stack argument which can support which can support a more efficient representation of an element than a tuple, which was what was previously supported. #284
  • Box.sample previously would clip incorrectly for up-bounded spaces such that 0 could never be sampled if the dtype was discrete or boolean. This is fixed such that 0 can be sampled in these cases. #249
  • If jax or pytorch was installed then on import gymnasium both of these modules would also be loaded causing significant slow downs in load time. This is now fixed such that jax and torch are only loaded when particular wrappers is loaded by the user. #323
  • In v0.26, we added parameters for Wrapper to allow different observation and action types to be specified for the wrapper and its sub-environment. However, this raised type issues with pyright and mypy, this is now fixed through Wrapper having four generic arguments, [ObsType, ActType, WrappedEnvObsType, WrappedEnvActType]. #337
  • In v0.25 and 0.v26 several new space types were introduced, Text, Graph and Sequence however the vector utility functions were not updated to support these spaces. Support for these spaces has been added to the experimental vector space utility functions: batch_space, concatenate, iterate and create_empty_array. #223
  • Due to a lack of testing the experimental stateful observation wrappers (FrameStackObservation, DelayObservation and TimeAwareObservation) did not work as expected. These wrappers are now fixed and testing has been added. #224

Minor changes

Documentation changes

What's Changed

Thank you to our new contributors in this release: @Matyasch, @DrRyanHuang, @nerdyespresso, @khoda81, @howardh, @mihaic, and @keyb0ardninja.

Full Changelog: v0.27.1...v0.28.0

v0.27.1#

Released on 2023-01-20 - GitHub - PyPI

Release Notes

Bugs fixed

New features/improvements

  • Improve LunarLander-v2 step performance by >1.5x by @PaulMest in #235
  • Added vector env support to StepAPICompatibility wrapper by @nidhishs in #238
  • Allow sequence to accept stacked np arrays if the feature space is Box by @jjshoots in #241
  • Improve the warning when an error is raised from a plugin by @pseudo-rnd-thoughts in #225
  • Add changelog (release notes) to the website by @mgoulao in #257
  • Implement RecordVideoV0 by @younik in #246
  • Add explicit error messages when unflatten discrete and multidiscrete fail by @PierreMardon in #267

Documentation updates

Thanks to the new contributors to Gymnasium, if you want to get involved, join our discord server. Linked in the readme.

Full Changelog: v0.27.0...v0.27.1

v0.27.0#

Released on 2022-12-12 - GitHub - PyPI

Release Notes

Gymnasium 0.27.0 is our first major release of Gymnasium. It has several significant new features, and numerous small bug fixes and code quality improvements as we work through our backlog. There should be no breaking changes beyond dropping Python 3.6 support and remove the mujoco Viewer class in favor of a MujocoRendering class. You should be able to upgrade your code that's using Gymnasium 0.26.x to 0.27.0 with little-to-no-effort.

Like always, our development roadmap is publicly available here so you can follow our future plans. The only large breaking changes that are still planned are switching selected environments to use hardware accelerated physics engines and our long standing plans for overhauling the vector API and built-in wrappers.

This release notably includes an entirely new part of the library: gymnasium.experimental. We are adding new features, wrappers and functional environment API discussed below for users to test and try out to find bugs and provide feedback.

New Wrappers

These new wrappers, accessible in gymnasium.experimental.wrappers, see the full list in https://gymnasium.farama.org/main/api/experimental/ are aimed to replace the wrappers in gymnasium v0.30.0 and contain several improvements

  • (Work in progress) Support arbitrarily complex observation / action spaces. As RL has advanced, action and observation spaces are becoming more complex and the current wrappers were not implemented with this mind.
  • Support for Jax-based environments. With hardware accelerated environments, i.e. Brax, written in Jax and similar PyTorch based programs, NumPy is not the only game in town anymore for writing environments. Therefore, these upgrades will use Jumpy, a project developed by Farama Foundation to provide automatic compatibility for NumPy, Jax and in the future PyTorch data for a large subset of the NumPy functions.
  • More wrappers. Projects like Supersuit aimed to bring more wrappers for RL, however, many users were not aware of the wrappers, so we plan to move the wrappers into Gymnasium. If we are missing common wrappers from the list provided above, please create an issue and we would be interested in adding it.
  • Versioning. Like environments, the implementation details of wrappers can cause changes in agent performance. Therefore, we propose adding version numbers to all wrappers, i.e., LambaActionV0. We don't expect these version numbers to change regularly and will act similarly to environment version numbers. This should ensure that all users know when significant changes could affect your agent's performance for environments and wrappers. Additionally, we hope that this will improve reproducibility of RL in the future, which is critical for academia.
  • In v28, we aim to rewrite the VectorEnv to not inherit from Env, as a result new vectorized versions of the wrappers will be provided.

Core developers: @gianlucadecola, @RedTachyon, @pseudo-rnd-thoughts

Functional API

The Env class provides a very generic structure for environments to be written in allowing high flexibility in the program structure. However, this limits the ability to efficiently vectorize environments, compartmentalize the environment code, etc. Therefore, the gymnasium.experimental.FuncEnv provides a much more strict structure for environment implementation with stateless functions, for every stage of the environment implementation. This class does not inherit from Env and requires a translation / compatibility class for doing this. We already provide a FuncJaxEnv for converting jax-based FuncEnv to Env. We hope this will help improve the readability of environment implementations along with potential speed-ups for users that vectorize their code.

This API is very experimental so open to changes in the future. We are interested in feedback from users who try to use the API which we believe will be in particular interest to users exploring RL planning, model-based RL and modifying environment functions like the rewards.

Core developers: @RedTachyon, @pseudo-rnd-thoughts, @balisujohn

Other Major changes

  • Refactor Mujoco Rendering mechanisms to use a separate thread for OpenGL. Remove Viewer in favor of MujocoRenderer which offscreen, human and other render mode can use by @rodrigodelazcano in #112
  • Add deprecation warning to gym.make(..., apply_env_compatibility=True) in favour of gym.make("GymV22Environment", env_id="...") by @pseudo-rnd-thoughts in #125
  • Add gymnasium.pprint_registry() for pretty printing the gymnasium registry by @kad99kev in #124
  • Changes Discrete.dtype to np.int64 such that samples are np.int64 not python ints. by @pseudo-rnd-thoughts in #141
  • Add migration guide for OpenAI Gym v21 to v26 by @pseudo-rnd-thoughts in #72
  • Add complete type hinting of core.py for Env, Wrapper and more by @pseudo-rnd-thoughts in #39
  • Add complete type hinting for all spaces in gymnasium.spaces by @pseudo-rnd-thoughts in #37
  • Make window in play() resizable by @Markus28 in #190
  • Add REINFORCE implementation tutorial by @siddarth-c in #155

Bug fixes and documentation changes

  • Remove auto close in VideoRecorder wrapper by @younik in #42
  • Change seeding.np_random error message to report seed type by @theo-brown in #74
  • Include shape in MujocoEnv error message by @ikamensh in #83
  • Add pretty Feature/GitHub issue form by @tobirohrer in #89
  • Added testing for the render return data in check_env and PassiveEnvChecker by @Markus28 in #117
  • Fix docstring and update action space description for classic control environments by @Thytu in #123
  • Fix __all__ in root __init__.py to specify the correct folders by @pseudo-rnd-thoughts in #130
  • Fix play() assertion error by @Markus28 in #132
  • Update documentation for Frozen Lake is_slippy by @MarionJS in #136
  • Fixed warnings when render_mode is None by @younik in #143
  • Added is_np_flattenable property to documentation by @Markus28 in #172
  • Updated Wrapper documentation by @Markus28 in #173
  • Updated formatting of spaces documentation by @Markus28 in #174
  • For FrozenLake, add seeding in random map generation by @kir0ul in #139
  • Add notes for issues when unflattening samples from flattened spaces by @rusu24edward in #164
  • Include pusher environment page to website by @axb2035 in #171
  • Add check in AsyncVectorEnv for success before splitting result in step_wait by @aaronwalsman in #178
  • Add documentation for MuJoCo.Ant-v4.use_contact_forces by @Kallinteris-Andreas in #183
  • Fix typos in README.md by @cool-RR in #184
  • Add documentation for MuJoCo.Ant v4 changelog by @Kallinteris-Andreas in #186
  • Fix MuJoCo.Ant action order in documentation by @Kallinteris-Andreas in #208
  • Add raise-from exception for the whole codebase by @cool-RR in #205

Behind-the-scenes changes

v0.26.3#

Released on 2022-10-24 - GitHub - PyPI

Release Notes

Note: ale-py (atari) has not updated to Gymnasium yet. Therefore pip install gymnasium[atari] will fail, this will be fixed in v0.27. In the meantime, use pip install shimmy[atari] for the fix.

Bug Fixes

  • Added Gym-Gymnasium compatibility converter to allow users to use Gym environments in Gymnasium by @RedTachyon in #61
  • Modify metadata in the HumanRendering and RenderCollection wrappers to have the correct metadata by @RedTachyon in #35
  • Simplified EpisodeStatisticsRecorder wrapper by @DavidSlayback in #31
  • Fix integer overflow in MultiDiscrete.flatten() by @olipinski in #55
  • Re-add the ability to specify the XML file for Mujoco environments by @Kallinteris-Andreas in #70

Documentation change

Full Changelog: v0.26.2...v0.26.3

Thank you for the new contributors

v0.26.2 #

Released on 2022-10-05 - GitHub - PyPI

This Release is an upstreamed version of Gym v26.2

Bugs Fixes

  • As reset now returns (obs, info) then in the vector environments, this caused the final step's info to be overwritten. Now, the final observation and info are contained within the info as "final_observation" and "final_info" @pseudo-rnd-thoughts
  • Adds warnings when trying to render without specifying the render_mode @younik
  • Updates Atari Preprocessing such that the wrapper can be pickled @vermouth1992
  • Github CI was hardened to such that the CI just has read permissions @sashashura
  • Clarify and fix typo in GraphInstance @ekalosak

v0.26.1#

Released on 2022-09-16 - GitHub - PyPI

This Release is an upstreamed version of Gym v26.1

In addition, the gym docs repo has been merged in with the new website https://gymnasium.farama.org/

v0.26.0: Initial Release#

Released on 2022-09-13 - GitHub - PyPI

This is the first release of Gymnasium, a maintained fork of OpenAI Gym

This release is identical to the Gym v0.26.0 except for the project name (Gymnasium) and Code of Conduct

Read #12 for the roadmap of changes