Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add PPO + Transformer-XL #459

Merged
merged 37 commits into from
Sep 18, 2024
Merged

Add PPO + Transformer-XL #459

merged 37 commits into from
Sep 18, 2024

Commits on Apr 15, 2024

  1. initial commit of ppo trxl

    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    990238e View commit details
    Browse the repository at this point in the history

  2. removed video capture

    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    513d2f8 View commit details
    Browse the repository at this point in the history

  3. Switched from grayscale to RGB

    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    cd596c8 View commit details
    Browse the repository at this point in the history

  4. RGB obs reconstruction

    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    cab8e07 View commit details
    Browse the repository at this point in the history

  5. print reconstruction loss instead of total loss

    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    2a012eb View commit details
    Browse the repository at this point in the history

  6. Ensure that transformer memory length is not larger than max episode … 

    …steps
    @MarcoMeter
    MarcoMeter committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    2efe26a View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. fixed enjoy.py in the case of Searing Spotlights

    @MarcoMeter
    MarcoMeter committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    8737485 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. added video capture support again after updating memory gym

    @MarcoMeter
    MarcoMeter committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    5ef2e07 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. default hyperparameters

    @MarcoMeter
    MarcoMeter committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    4f88baa View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. remove unnecessary padding from TrXL memory, if applicable

    @MarcoMeter
    MarcoMeter committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    9974f20 View commit details
    Browse the repository at this point in the history

  2. print SPS

    @MarcoMeter
    MarcoMeter committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    7761993 View commit details
    Browse the repository at this point in the history

  3. updated pyproject.toml because of memory-gym 1.0.2

    @MarcoMeter
    MarcoMeter committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    5374027 View commit details
    Browse the repository at this point in the history

  4. fixed comment

    @MarcoMeter
    MarcoMeter committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    fe85e75 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. refactored code + added a comment

    @Horrible22232
    Horrible22232 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    eb36a32 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. added annealing entropy coefficient. learning rate anneals also from … 

    …initial to final value.
    @MarcoMeter
    MarcoMeter committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    2fe62d4 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. aligned monitoring of losses and further metrics

    @MarcoMeter
    MarcoMeter committed May 27, 2024
    Configuration menu
    Copy the full SHA
    86e59ef View commit details
    Browse the repository at this point in the history

  2. slight adjustments due to pre-commit, pre-commit still fails due to u… 

    …nused imports, however these imports are necessary for the used environments to be registered
    @MarcoMeter
    MarcoMeter committed May 27, 2024
    Configuration menu
    Copy the full SHA
    62fea9d View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. heads share parameters again for reproduction purposes, set default h… 

    …yperparameters for succesfull MMG training
    @MarcoMeter
    MarcoMeter committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    3297da4 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. fixed entropy for multi-discrete action spaces

    @MarcoMeter
    MarcoMeter committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    8ff1edf View commit details
    Browse the repository at this point in the history

  2. advantage normalization is off per default, added hidden layer after … 

    …TrXL
    @MarcoMeter
    MarcoMeter committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b9f8131 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. set max_episode_steps for endless environments

    @MarcoMeter
    MarcoMeter committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    931bc81 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. updated poetry.lock, define max episode steps for endless environment… 

    …s, started docs, added benchmark sh
    @MarcoMeter
    MarcoMeter committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    cc24bfa View commit details
    Browse the repository at this point in the history

  2. added Transformer-XL (PPO-TrXL) to the navigation bar, improved docs

    @MarcoMeter
    MarcoMeter committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    476757a View commit details
    Browse the repository at this point in the history

  3. Added report to docs

    @MarcoMeter
    MarcoMeter committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    9a50c0a View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. refactored "blocks" to "layers"

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    c4fde03 View commit details
    Browse the repository at this point in the history

  2. Finalized enjoy.py,which can load pre-trained models from huggingface… 

    … hub, updated docs
    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    50845ab View commit details
    Browse the repository at this point in the history

  3. pre-commit enjoy.py

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    e9c2f54 View commit details
    Browse the repository at this point in the history

  4. add #noqa to fix pre-commit

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    2b13995 View commit details
    Browse the repository at this point in the history

  5. fix pre-commit #noqa

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    10ecb84 View commit details
    Browse the repository at this point in the history

  6. pre-commit fixed enjoy import order

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    71be7ee View commit details
    Browse the repository at this point in the history

  7. last pass of pre-commit

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    e72f25c View commit details
    Browse the repository at this point in the history

  8. fixed spelling

    @MarcoMeter
    MarcoMeter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    4a46f71 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. remove macos-latest from .github/workflows/test.yaml

    @MarcoMeter
    MarcoMeter committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    40cb39f View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Added ppo_trxl to README.md, fixed enjoy and ppo_trxl for MiniGrid, a… 

    …dded proper rendering to ProofofMemory-v0, updated docs for training and enjoying MiniGrid and ProofofMemory-v0
    @MarcoMeter
    MarcoMeter committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    2545864 View commit details
    Browse the repository at this point in the history

  2. pre-commit fixes

    @MarcoMeter
    MarcoMeter committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    c25ba62 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Add requirements.txt, update poetry, torch defaults to CUDA, updated … 

    …docs
    @MarcoMeter
    MarcoMeter committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    d3d6485 View commit details
    Browse the repository at this point in the history

  2. updated doc links, added memory gym requirements to README.md

    @MarcoMeter
    MarcoMeter committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    a7e708a View commit details
    Browse the repository at this point in the history