External Environments

First-Party Environments

The Farama Foundation maintains a number of other projects, which use the Gymnasium API, environments include: gridworlds (Minigrid), robotics (Gymnasium-Robotics), 3D navigation (Miniworld), web interaction (MiniWoB++), arcade games (Arcade Learning Environment), Doom (ViZDoom), Meta-objective robotics (Metaworld), autonomous driving (HighwayEnv), Retro Games (stable-retro), and many more.

The Farama Foundation also maintains alternate APIs for RL, including: multi-agent RL (PettingZoo), offline-RL (Minari), multi-objective RL (MO-Gymnasium), goal-RL (Gymnasium-Robotics).

Third-party environments with Gymnasium

This page contains environments which are not maintained by Farama Foundation and, as such, cannot be guaranteed to function as intended.

If you’d like to contribute an environment, please reach out on Discord, then submit a PR by editing this file, additional instructions can be found inside that file

Autonomous Driving environments

Autonomous vehicles and traffic management, on the road and in the air.

• BlueSky-Gym: Reinforcement Learning Environments for Air Traffic Applications

  

  A collection of Gymnasium environments for air traffic management tasks, allowing for both civil and urban aviation applications. Built on the open-source air traffic simulator BlueSky.

• racecar_gym: Miniature racecar env using PyBullet

  A gym environment for a miniature racecar using the PyBullet physics engine.

• sumo-rl: Reinforcement Learning using SUMO traffic simulator

  

  Gymnasium wrapper for various environments in the SUMO traffic simulator. Supports both single and multiagent settings (using pettingzoo).

Biological / Medical environments

Interacting with Biological Systems.

• ICU-Sepsis: A Benchmark MDP Built from Real Medical Data

  

  ICU-Sepsis is a tabular reinforcement learning environment that simulates the treatment of sepsis in an intensive care unit (ICU). Introduced in the paper ICU-Sepsis: A Benchmark MDP Built from Real Medical Data, the environment is lightweight and easy to use, yet challenging for most reinforcement learning algorithms.

Cybersecurity environments

Interacting with systems and cybersecurity tools.

• Security-Gym: Continual learning on live Linux server logs and kernel events

  

  A gymnasium environment that sends continuous Linux server log data and kernel events to simulate an environment where an agent runs continually on the Linux server. The action space supports actions an agent can take to react to cybersecurity attacks on the server.

Economic / Financial environments

Everything Economics related.

• gym-anytrading: Financial trading environments for FOREX and STOCKS

  AnyTrading is a collection of Gym environments for reinforcement learning-based trading algorithms with a great focus on simplicity, flexibility, and comprehensiveness.

• gym-invmgmt: Inventory management and supply chain optimization environments

  

  gym-invmgmt provides Gymnasium-compatible multi-echelon inventory-management environments for supply-chain optimization research. It includes configurable network topologies, production capacities, lead times, backlog penalties, non-stationary demand generation, YAML-defined custom networks, and tools for evaluating optimization, heuristic, and reinforcement-learning controllers.

• gym-mtsim: Financial trading for MetaTrader 5 platform

  MtSim is a simulator for the MetaTrader 5 trading platform for reinforcement learning-based trading algorithms.

• gym-trading-env: Trading Environment

  

  Gym Trading Env simulates stock (or crypto) market from historical data. It was designed to be fast and easily customizable.

• tradefloor: Simulated stock markets with a limit order book per company

  

  tradefloor generates simulated stock markets in which each company has a limit order book and an economy runs underneath. Its TradingEnv takes a target portfolio weight per stock as the action, and the policy’s orders fill against the book and move prices, so the reward, the step’s change in net worth, includes the cost of its own trading. Each seed is a new market, and any episode can be replayed exactly from its seed. Install with pip install "tradefloor[rl]".

Electrical / Energy environments

Manage the flow of Electrons.

• EV2Gym: A Realistic EV-V2G-Gym Simulator for EV Smart Charging

  

  EV2Gym is a fully customizable and easily configurable environment for Electric Vehicle (EV) smart charging simulations on a small and large scale. Also, includes non-RL baseline implementations such as mathematical programming, model predictive control, and heuristics.

• gym-electric-motor: Gym environments for electric motor simulations

  An environment for simulating a wide variety of electric drives taking into account different types of electric motors and converters.

Environmental / Climate environments

Simulation of natural phenomena, disaster management, and ecological systems.

• FirecastRL: Wildfire Suppression Environment

  

  A Gymnasium-compatible wildfire simulation environment that models fire spread using physics-based dynamics and incorporates helicopter firefighting strategies. It’s built on real terrain data, including land cover and elevation, and supports customizable observation and reward wrappers.

• Ignis: 3D Structural & Wildfire Fire Suppression

  

  A Gymnasium environment for fire suppression that extends cellular-automaton wildfire RL to 3D indoor / structural fire. A single JSON scene config (geometry, obstacles, materials, ignition, budgets) defines the world; Ignis-Indoor-v0 operates zoned ceiling sprinklers to contain a fire spreading through a 3D voxel building, while Ignis-Wildfire-v0 is a 2D wildfire air-tanker variant. It is pure-NumPy and CPU-only, ships CEM and PPO baselines, and includes evacuation and safe-building-design tooling.

Game environments

Board Games, Video Games and all other interactive entertainment mediums.

• Advanced Minesweeper Environment

  

  This is a custom minesweeper gymnasium environment that allows for an optional custom mask for added complexity. Fully customizable with board size, mine density and custom masks.

• BoltCrypt: Procedural Dungeon RL Environment

  

  BoltCrypt is a lightweight environment featuring procedurally generated dungeons. It challenges Reinforcement Learning agents to navigate complex layouts, solve sokoban-style boulder puzzles, and manage inventory items like keys to reach the exit. BoltCrypt also contains a Natural Language wrapper to enable testing LLM’s long term planning.

• Connect-4-gym : An environment for practicing self playing

  Connect-4-Gym is an environment designed for creating AIs that learn by playing against themselves and assigning them an Elo rating. This environment can be used to train and evaluate reinforcement learning agents on the classic board game Connect Four.

• Craftium: an extensible framework for creating RL environments

  

  Craftium wraps the Minetest game engine into the Gymnasium API, providing a modern and easy-to-use platform for designing Minecraft-like RL environments.

• flappy-bird-env

  

  Flappy Bird as a Farama Gymnasium environment.

• flappy-bird-gymnasium: A Flappy Bird environment for Gymnasium

  

  A simple environment for single-agent reinforcement learning algorithms on a clone of Flappy Bird, the hugely popular arcade-style mobile game. Both state and pixel observation environments are available.

• Generals.io bots: Develop your agent for generals.io!

  

  Generals.io is a fast-paced strategy game on a 2D grid. We make bot development accessible via the Gymnasium/PettingZoo API.

• Hill Climb Racing Env: A Hill Climb Racing RL environment

  

  An environment inspired by the mobile game Hill Climb Racing, featuring Box2D physics, Pygame rendering, procedurally generated Perlin noise terrain, discrete and continuous action spaces, five reward functions, and 13 pre-trained PPO baseline models.

• Kraken Crossing: A dark, stochastic nautical environment

  

  Kraken Crossing is a dark, stochastic 8x8 nautical navigation game designed for tabular reinforcement learning. An agent steers a pirate ship in eight directions and fires port or starboard cannons while learning to avoid hidden whirlpools, defeat a Kraken, and reach treasure. The package includes animated agent and omniscient rendering, a reproducible Q-learning baseline, examples, tests, and a companion Jupyter notebook.

• Pickomino (Heckmeck): Dice and Tile game

  

  Push your luck dice game with a twist. Your agent plays against 1 to 6 bots.

• pystk2-gymnasium: SuperTuxKart races gymnasium wrapper

  

  Uses a python wrapper around SuperTuxKart that allows to access the world state and to control a race.

• QWOP: An environment for Bennet Foddy’s game QWOP

  

  QWOP is a game about running extremely fast down a 100 meter track. With this Gymnasium environment you can train your own agents and try to beat the current world record (5.0 in-game seconds for humans and 4.7 for AI).

• Reinforce Tactics: 2D Turn-Based Strategy Game

  

  Reinforce Tactics is a turn-based strategy game designed for reinforcement learning research, combining tactical grid-based combat with an economy system while supporting multiple AI opponent types including rule-based bots, LLM-powered agents, and trainable RL models.

• Spatial-Gym: Step-by-step spatial reasoning on SPaRC grid puzzles

  

  Spatial-Gym turns the 1,000 SPaRC grid puzzles into a Gymnasium environment for step-by-step spatial reasoning. The agent draws a line from the start node to the end node one cell at a time (Discrete(4)), and the line must obey every rule on the board. An episode ends with a reward of +1 if the path matches a stored solution and -1 otherwise, with +0.01 or -0.01 per step for staying on or leaving a solution path. Observations are either grid arrays or the SPaRC text grid, so the same puzzles work for RL and LLM agents, and an optional mode lets the agent backtrack. Install with pip install Spatial-Gym.

• Tetris Gymnasium: A fully configurable Gymnasium compatible Tetris environment

  

  Tetris Gymnasium is a clean implementation of Tetris as a Gymnasium environment. It can be extensively customized (e.g. board dimensions, gravity, …), is well documented and includes many examples on how to use it e.g. by providing training scripts.

• tmrl: TrackMania 2020 through RL

  

  tmrl is a distributed framework for training Deep Reinforcement Learning AIs in real-time applications. It is demonstrated on the TrackMania 2020 video game.

Mathematics / Computational

Reduce computational compute, prove math theorems, and more.

• gym-saturation: Environments used to prove theorems

  

  An environment for guiding automated theorem provers based on saturation algorithms (e.g. Vampire).

• spark-sched-sim: Environments for scheduling DAG jobs in Apache Spark

  

  spark-sched-sim simulates Spark clusters for RL-based job scheduling algorithms. Spark jobs are encoded as directed acyclic graphs (DAGs), providing opportunities to experiment with graph neural networks (GNN’s) in the RL context.

Robotics environments

Autonomous Robots.

• aas-gym: PX4 and ArduPilot multi-drone SITL simulation with YOLO and LiDAR

  

  Multi-drone PX4 and ArduPilot ROS2 control of quadrotors and VTOL planes in Gazebo Sim, with YOLO, LiDAR, Dockerized Gym simulation, and JetPack deployment.

• Balloon Popping Challenge: A 6-DoF Rocket GNC Simulation Gymnasium Environment

  

  Balloon Popping Challenge is a 6-DoF rocket guidance, navigation, and control (GNC) simulation environment. The environment is designed to simulate an active controlled rocket to pop balloons scattered in the sky. The simulator incorporates realistic physics, including atmospheric conditions and rocket dynamics, to provide a challenging platform for developing and testing GNC algorithms. This project is based on ActiveRocketPy, a fork of open-source software RocketPy.

• BSK-RL: Environments for Spacecraft Planning and Scheduling

  

  BSK-RL is a Python package for constructing Gymnasium environments for spacecraft tasking problems. It is built on top of Basilisk, a modular and fast spacecraft simulation framework, making the simulation environments high-fidelity and computationally efficient. BSK-RL also includes a collection of utilities and examples for working with these environments

• deformable_gym: RL environments for grasping deformable objects

  

  A collection of gymnasium environments for learning to grasp 3D deformable objects built with PyBullet and Mujoco.

• FlyCraft: A Fixed-wing UAV Environment

  

  FlyCraft is a Gymnasium environment for fixed-wing UAV tasks. By default, FlyCraft provides two tasks: attitude control and velocity vector control. These tasks are characterized by their multi-goal and long-horizon nature, posing significant challenges for RL exploration. Additionally, the rewards can be configured as either Markovian or non-Markovian, making FlyCraft suitable for research on non-Markovian problems.

• gym-jiminy: Training Robots in Jiminy

  

  gym-jiminy presents an extension of the initial Gym for robotics using Jiminy, an extremely fast and light-weight simulator for poly-articulated systems using Pinocchio for physics evaluation and Meshcat for web-based 3D rendering.

• gym-pybullet-drones: Environments for quadcopter control

  

  A simple environment using PyBullet to simulate the dynamics of a Bitcraze Crazyflie 2.x nanoquadrotor.

• gymnasium-docker-ros2: Gymnasium wrapping of Dockerized Gazebo/ROS2 simulation

  

  GDR2 shows how to wrap a multi-container Docker/Gazebo/ROS2 app into gymnasium and train stable-baselines3’s PPO, using pyzmq for communication and gz service to synchronously step the /clock

• Isaac Lab: GPU-accelerated robot learning on NVIDIA Isaac Sim

  

  A unified framework for robot learning built on NVIDIA Isaac Sim, the successor to the archived OmniIsaacGymEnvs and IsaacGymEnvs. It provides GPU-parallel Gymnasium environments for manipulation, locomotion and navigation tasks.

• Itomori: UAV Risk-aware Flight Environment

  

  Itomori is an environment for risk-aware UAV flight, it provides tools to solve Chance-Constrained Markov Decision Processes (CCMDP). The env allows to simulate, visualize, and evaluate UAV navigation in complex and risky environments, incorporating variables like GPS uncertainty, collision risk, and adaptive flight planning. Itomori is intended to support UAV path-planning research by offering adjustable parameters, detailed visualizations, and insights into agent behavior in uncertain environments.

• MarsExplorer-Gymnasium: Gymnasium-compatible Mars rover exploration environment

  

  MarsExplorer-Gymnasium is a fully Gymnasium-compatible environment for autonomous Mars rover exploration. It enables research on terrain coverage, navigation, and exploration efficiency in unknown, procedurally generated landscapes, supporting modern deep reinforcement learning algorithms through the standard Gymnasium API.

• Mesozoic Labs: Robotic Dinosaur Locomotion

  

  Mesozoic Labs is a reinforcement learning research project that trains robotic dinosaurs (Velociraptor, T-Rex, Brachiosaurus) to walk, run, and hunt using MuJoCo physics simulation, curriculum learning, and algorithms like PPO and SAC.

• panda-gym: Robotics environments using the PyBullet physics engine

  

  PyBullet based simulations of a robotic arm moving objects.

• PyFlyt: UAV Flight Simulator Environments for Reinforcement Learning Research

  A library for testing reinforcement learning algorithms on various UAVs. It is built on the Bullet physics engine, offers flexible rendering options, time-discrete steppable physics, Python bindings, and support for custom drones of any configuration, be it biplanes, quadcopters, rockets, and anything you can think of.

• relign: Active alignments of optical systems

  

  An environment for active alignments of lens systems with reinforcement learning using high-dimensional optical sensor output as observation.

• safe-control-gym: Evaluate safety of RL algorithms

  Evaluate safety, robustness and generalization via PyBullet based CartPole and Quadrotor environments—with CasADi (symbolic) a priori dynamics and constraints.

• Safety-Gymnasium: Ensuring safety in real-world RL scenarios

  

  Highly scalable and customizable Safe Reinforcement Learning library.

• WireHarness-MultiAgent-RL: Centralized multi-robot wire handling in MuJoCo

  

  MuJoCo-based Gymnasium environment for centralized multi-robot wire handling. A single policy controls five planar movers via a joint action space to navigate cable harnesses to target configurations on an assembly board without causing collisions.

• WireHarness-MultiGripper-RL: Multi-branch wire-harness picking with a multi-gripper in MuJoCo

  

  MuJoCo-based Gymnasium environment with deformable cable physics for robotic wire-harness picking. A multi-gripper end-effector navigates a dense multi-branch harness to reach and grasp connector housings while minimizing induced wire stress and collisions.

Telecommunication Systems environments

Interact and/or manage wireless and/or wired telecommunication systems.

• linkgym: 5G NR link adaptation on NVIDIA Sionna

  

  A Gymnasium environment for 5G NR link adaptation (MCS selection) built on NVIDIA Sionna SYS. The channel is 3GPP TDL fading or ray-traced with Sionna RT, including a published dataset of Munich streets and a generator for your own scenes. It comes with classical baselines (ILLA, OLLA, oracle) and an evaluation protocol with common random numbers and confidence intervals.

• mobile-env: Environments for coordination of wireless mobile networks

  

  An open, minimalist Gymnasium environment for autonomous coordination in wireless mobile networks.

Other

• Buffalo-Gym: Multi-Armed Bandit Gymnasium

  

  Buffalo-Gym is a Multi-Armed Bandit (MAB) gymnasium built primarily to assist in debugging RL implementations. MABs are often easy to reason about what the agent is learning and whether it is correct. Buffalo-gym encompasses Bandits, Contextual bandits, and contextual bandits with aliasing.

• CARL: context adaptive RL

  

  Contextual extensions of popular reinforcement learning environments that enable training and test distributions for generalization, e.g. CartPole with variable pole lengths or Brax robots with different ground frictions.

• DACBench: a benchmark for Dynamic Algorithm Configuration

  

  A benchmark library for Dynamic Algorithm Configuration. Its focus is on reproducibility and comparability of different DAC methods as well as easy analysis of the optimization process.

• EnvPool: C++-based high-performance parallel environment execution

  

  A highly parallel, C++-based vectorized execution engine for Atari, MuJoCo, classic control and other environments, with a Gymnasium-compatible API (envpool.make(..., env_type="gymnasium")).

• gym-cellular-automata: Cellular Automata environments

  

  Environments where the agent interacts with Cellular Automata by changing its cell states.

• Gym-Gridworlds: Collection of Customizable Minimalistic Gridworlds

  

  The default class implements a “go-to goal”, but it can be easily customized for different tasks, with a variety of grids, rewards, dynamics, and tasks. It supports different observation types (discrete, coordinates, binary, pixels, partial). Useful for quickly testing and prototyping RL algorithms, both tabular and with function approximation.

• Gym4ReaL

  

  Gym4ReaL is a comprehensive suite of realistic environments designed to support the development and evaluation of RL algorithms that can operate in real-world scenarios. The suite includes a diverse set of tasks exposing RL algorithms to a variety of practical challenges, such as water management, energy management in microgrids, financial trading, and more.

• gymnax: Hardware Accelerated RL Environments

  RL Environments in JAX which allows for highly vectorised environments with support for a number of environments, Gym, MinAtari, bsuite and more.

• LineFlow: Active control of production lines

  

  An environment to learn active control of production lines with reinforcement learning.

• matrix-mdp: Easily create discrete MDPs

  

  An environment to easily implement discrete MDPs as gym environments. Turn a set of matrices (P_0(s), P(s'| s, a) and R(s', s, a)) into a gym environment that represents the discrete MDP ruled by these dynamics.

• SimpleGrid: a simple grid environment for Gymnasium

  

  SimpleGrid is a super simple and minimal grid environment for Gymnasium. It is easy to use and customise and it is intended to offer an environment for rapidly testing and prototyping different RL algorithms.

Third-Party Environments using Gym

There are a large number of third-party environments using various versions of Gym. Many of these can be adapted to work with gymnasium (see Compatibility with Gym), but are not guaranteed to be fully functional.

Video Game environments

• gym-derk: GPU accelerated MOBA environment

  A 3v3 MOBA environment where you train creatures to fight each other.

• SlimeVolleyGym: A simple environment for Slime Volleyball game

  A simple environment for benchmarking single and multi-agent reinforcement learning algorithms on a clone of Slime Volleyball game.

• Unity ML Agents: Environments for Unity game engine

  Gym (and PettingZoo) wrappers for arbitrary and premade environments with the Unity game engine.

• PGE: Parallel Game Engine

  Uses The Open 3D Engine for AI simulations and can interoperate with the Gym. Uses PyBullet physics.

Robotics environments

• MarsExplorer: Environments for controlling robot on Mars

  Mars Explorer is a Gym compatible environment designed and developed as an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of an unknown terrain.

• robo-gym: Real-world and simulation robotics

  Robo-gym provides a collection of reinforcement learning environments involving robotic tasks applicable in both simulation and real-world robotics.

• Offworld-gym: Control real robots remotely for free

  Gym environments that let you control real robots in a laboratory via the internet.

• gym-softrobot: Soft-robotics environments

  A large-scale benchmark for co-optimizing the design and control of soft robots.

• iGibson: Photorealistic and interactive robotics environments

  A simulation environment with high-quality realistic scenes, with interactive physics using PyBullet.

• DexterousHands: Dual dexterous hand manipulation tasks

  This is a library that provides dual dexterous hand manipulation tasks through Isaac Gym.

Autonomous Driving environments

• gym-duckietown: Lane-following for self-driving cars

  A lane-following simulator built for the Duckietown project (small-scale self-driving car course).

• CommonRoad-RL: Motion planning for traffic scenarios

  A Gym for solving motion planning problems for various traffic scenarios compatible with CommonRoad benchmarks, which provides configurable rewards, action spaces, and observation spaces.

• racing_dreamer: Latent imagination in autonomous racing

  Train a model-based RL agent in simulation and, without finetuning, transfer it to small-scale race cars.

• l2r: Multimodal control environment where agents learn how to race

  An open-source reinforcement learning environment for autonomous racing.

Other environments

• CompilerGym: Optimise compiler tasks

  Reinforcement learning environments for compiler optimization tasks, such as LLVM phase ordering, GCC flag tuning, and CUDA loop nest code generation.

• gym-sokoban: 2D Transportation Puzzles

  The environment consists of transportation puzzles in which the player’s goal is to push all boxes to the warehouse’s storage locations.

• NLPGym: A toolkit to develop RL agents to solve NLP tasks

  NLPGym provides interactive environments for standard NLP tasks such as sequence tagging, question answering, and sequence classification.

• ShinRL: Environments for evaluating RL algorithms

  ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectives (Deep RL Workshop 2021)

• openmodelica-microgrid-gym: Environments for controlling power electronic converters in microgrids

  The OpenModelica Microgrid Gym (OMG) package is a software toolbox for the simulation and control optimization of microgrids based on energy conversion by power electronic converters.

• GymFC: A flight control tuning and training framework

  GymFC is a modular framework for synthesizing neuro-flight controllers. Has been used to generate policies for the world’s first open-source neural network flight control firmware Neuroflight.