skrl-v1.3.0

[1.3.0] - 2024-09-11

Added

• Distributed multi-GPU and multi-node learning (JAX implementation)
• Utilities to start multiple processes from a single program invocation for distributed learning using JAX
• Model instantiators return_source parameter to get the source class definition used to instantiate the models
• Runner utility to run training/evaluation workflows in a few lines of code
• Wrapper for Isaac Lab multi-agent environments
• Wrapper for Google Brax environments

Changed

• Move the KL reduction from the PyTorch KLAdaptiveLR class to each agent that uses it in distributed runs
• Move the PyTorch distributed initialization from the agent base class to the ML framework configuration
• Upgrade model instantiator implementations to support CNN layers and complex network definitions,
  and implement them using dynamic execution of Python code
• Update Isaac Lab environment loader argument parser options to match Isaac Lab version
• Allow to store tensors/arrays with their original dimensions in memory and make it the default option

Changed (breaking changes)

• Decouple the observation and state spaces in single and multi-agent environment wrappers and add the state
  method to get the state of the environment
• Simplify multi-agent environment wrapper API by removing shared space properties and methods

Fixed

• Catch TensorBoard summary iterator exceptions in TensorboardFileIterator postprocessing utils
• Fix automatic wrapper detection issue (introduced in previous version) for Isaac Gym (previews),
  DeepMind and vectorized Gymnasium environments
• Fix vectorized/parallel environments reset method return values when called more than once
• Fix IPPO and MAPPO act method return values when JAX-NumPy backend is enabled

skrl-v1.2.0

[1.2.0] - 2024-06-23

Added

• Define the environment_info trainer config to log environment info (PyTorch implementation)
• Add support to automatically compute the write and checkpoint intervals and make it the default option
• Single forward-pass in shared models
• Distributed multi-GPU and multi-node learning (PyTorch implementation)

Changed

• Update Orbit-related source code and docs to Isaac Lab

Fixed

• Move the batch sampling inside gradient step loop for DDPG and TD3
• Perform JAX computation on the selected device

skrl-v1.1.0

[1.1.0] - 2024-02-12

Added

• MultiCategorical mixin to operate MultiDiscrete action spaces

Changed (breaking changes)

• Rename the ManualTrainer to StepTrainer
• Output training/evaluation progress messages to system's stdout
• Get single observation/action spaces for vectorized environments
• Update Isaac Orbit environment wrapper

skrl-v1.0.0

[1.0.0] - 2023-08-16

Transition from pre-release versions (1.0.0-rc.1 and1.0.0-rc.2) to a stable version.

This release also announces the publication of the skrl paper in the Journal of Machine Learning Research (JMLR):
https://www.jmlr.org/papers/v24/23-0112.html

Summary of the most relevant features:

• JAX support
• New documentation theme and structure
• Multi-agent Reinforcement Learning (MARL)

skrl-v1.0.0-rc.2

[1.0.0-rc.2] - 2023-08-11

Added

• Get truncation from time_outs info in Isaac Gym, Isaac Orbit and Omniverse Isaac Gym environments
• Time-limit (truncation) boostrapping in on-policy actor-critic agents
• Model instantiators initial_log_std parameter to set the log standard deviation's initial value

Changed

• Structure environment loaders and wrappers file hierarchy coherently [breaking change]
• Drop support for versions prior to PyTorch 1.9 (1.8.0 and 1.8.1)

skrl-v1.0.0-rc.1

[1.0.0-rc.1] - 2023-07-25

Added

• JAX support (with Flax and Optax)
• RPO agent
• IPPO and MAPPO multi-agent
• Multi-agent base class
• Bi-DexHands environment loader
• Wrapper for PettingZoo and Bi-DexHands environments
• Parameters num_envs, headless and cli_args for configuring Isaac Gym, Isaac Orbit
  and Omniverse Isaac Gym environments when they are loaded

Changed

• Migrate to pyproject.toml Python package development
• Define ML framework dependencies as optional dependencies in the library installer
• Move agent implementations with recurrent models to a separate file
• Allow closing the environment at the end of execution instead of after training/evaluation
• Documentation theme from sphinx_rtd_theme to furo
• Update documentation structure and examples

Fixed

• Compatibility for Isaac Sim or OmniIsaacGymEnvs (2022.2.0 or earlier)
• Disable PyTorch gradient computation during the environment stepping
• Get categorical models' entropy
• Typo in KLAdaptiveLR learning rate scheduler
  (keep the old name for compatibility with the examples of previous versions.
  The old name will be removed in future releases)

skrl-v0.10.2

[0.10.2] - 2023-03-23

Changed

• Update loader and utils for OmniIsaacGymEnvs 2022.2.1.0
• Update Omniverse Isaac Gym real-world examples

skrl-v0.10.1

[0.10.1] - 2023-01-26

Fixed

• Tensorboard writer instantiation when write_interval is zero

skrl-v0.10.0

[0.10.0] - 2023-01-22

Added

• Isaac Orbit environment loader
• Wrap an Isaac Orbit environment
• Gaussian-Deterministic shared model instantiator

skrl-v0.9.1

[0.9.1] - 2023-01-17

Added

• Utility for downloading models from Hugging Face Hub

Fixed

• Initialization of agent components if they have not been defined
• Manual trainer train/eval method default arguments