Minecraft RL

This project is a python framework for running reinforcement learning experiments in Minecraft.

Usage/Examples

An experiment is made out of three abstract components:

• The agent observes and acts upon the environment.
• The policy uses observations to decide what actions to take, and updates itself.
• The task creates the scenario within the environment, and evaluates the agent's reward each step.

Here is a (simplified) example of what my_experiment.py could look like:

import MyTask
import MyAgent
import MyModel

agent = MyAgent(viewer_port = 25565,name = "bot")
model = MyModel()

for r in range(5):
    MyTask.reset_task(agent)
    for step in range(100):
        observation = agent.get_image()
        action = model.get_result(observation)
        agent.apply_action(action)
        reward = MyTask.get_reward(agent)
        model.train(observation,action,reward)

Then launch your server, and run

python my_experiment.py

By creating multiple agents, tasks, and models, you can easily mix-and-match the components to try different experiments with minimal effort.

Demo

This YouTube video demonstrates a few minutes of the training process for the maze_task. An agent spends a few seconds exploring the maze, then the task is reset and the agent tries again.

Dependencies

Python Packages

pytorch - For all neural networks.

javascript - To act as a bridge to the mineflayer and prismarine-viewer javascript libraries, which do not have native python bindings.

selenium - To create firefox instances for prismarine-viewer to stream game footage to.

opencv - For processing captured screenshots.

pillow - For processing captured screenshots.

numpy - For general purpose data manipulation.

pandas - For general purpose data manipulation.

Javascript Packages

mineflayer - To create autonomous Minecraft players as agents controlled by a neural network.

prismarine-viewer - To capture the view of players, used for visual input to nerual networks.

Minecraft

A dedicated server or local host running Minecraft 1.12, which gives admin access to all players who join. At the time of writing, Mineflayer supports Minecraft versions 1.8 to 1.20, however all code in this repository was tested with version 1.12 so may be unstable in other versions.

Configuration

Edit the configuration.py file to change environment parameters. Available parameters:

• SERVER_IP - The ip address of the server your bots should connect to. If running a local server on the same machine as your bots, set this to 'localhost'.
• SERVER_PORT - The port of your server. If hosting locally, the port will change each time you open the server, and will be given via an automated message in the game's chat.
• SCREENSHOT_X_RES, SCREENSHOT_Y_RES - The dimensions (in pixels) that the minecraft screenshots will be resized to before being inputted to neural networks.
• MAZE_SIZE - The dimensions (in blocks) that square mazes will be generated for the maze_task.
• MAZE_SEED - The seed used to generate mazes for the maze_task. Set to None to get a new maze each time the experiment is run.