Skip to content

Latest commit

 

History

History
131 lines (89 loc) · 4.81 KB

README.md

File metadata and controls

131 lines (89 loc) · 4.81 KB
Raw

About

This repository is a fork of https://github.com/simondlevy/neat-gym containing code allowing you to train, test, and visualize Gymnasium environments (games) using the NEAT algorithm and its variants.

The two goals of this project are

  1. Make this work as simple as possible, via config files.

  2. Make the code run fast, by simultaneously evaluating the fitnesses of the population on multiprocessor machines.

Quickstart

To get started you should install neat-python and PUREPLES from source. Then do the following:

% python3 neat-evolve.py config/cartpole

This will run neat-python on the CartPole-v1 environment using the parallel fitness evaluator, so you can take advantage of all the cores on your computer.

Once evolution finishes, you can try out your evolved network by doing:

% python3 neat-test.py models/CartPole-v1<fitness>.dat

where <fitness> is the fitness of your evolved network. The visuals folder will contain a PDF showing the corresponding model, and the runs folder will contain a CSV file with the history of the fitnesses (mean, standard deviation, max). To visualize this history you can run the neat-plot.py script on this CSV file.

Running Gymnasium-Atari Games

To train spaceinvaders-RGB (https://gymnasium.farama.org/environments/atari/space_invaders/), run

% python3 neat-evolve.py models/spaceinvadersRGB

To train spaceinvaders-RAM, run

% python3 neat-evolve.py models/spaceinvadersRAM

Info on how to set up the config files for atari games can be found in the README located in the config folder

HyperNEAT and ES-HyperNEAT

IMPORTANT

Running gymnasium games is currently untested with HyperNEAT and ES-HyperNEAT, and may not work.

NEAT-Gym supports HyperNEAT via the --hyper option and and ES-HyperNEAT via the
--eshyper option.

There are two ways to specify the substrate:

  1. In the [Substrate] section of the config file (default)

  2. Via a get_substrate() method in your environment. This method should return a tuple containing the input, hidden, and output coordinates and the name of the activation function (e.g., sigmoid). For ES-HyperNEAT, the hidden coordinates should be None.

Novelty Search

IMPORTANT

Running gymnasium games is currently untested with Novelty Search, and may not work.

NEAT-Gym supports Novelty Search via the --novelty option. To use this option, the info dictionary returned by your environment's step() method should have an entry for behavior, whose value is the behavior of the agent at the end of the episode (for example, its final position in the maze), or None before the end of the episode. For an example, try out the maze environment.

Using NEAT-Gym in other projects

The neat_gym module exports two functions:

  • read_file loads a .dat file from command-line arguments, returning the evolved network and the name of the environment used to evolve it (as well as flags for recording a move and suppressing the display).

  • eval_net takes the output of read-file and runs an episode of the environment using the network.

To make these functions available other projects (e.g., for 3D animation of your environment during testing), do:

% sudo python3 setup.py install

You will also need to create a NEAT configuration file for your environment. As usual, the easiest way to do this is to take something that works (like the config file for CartPole-v1) and modify it to do what you want.

Related projects