Skip to content

add miniPile dataset for pretraining, 1M entries (solves the 'out of data' at 40 iters issue) #21

add miniPile dataset for pretraining, 1M entries (solves the 'out of data' at 40 iters issue)

add miniPile dataset for pretraining, 1M entries (solves the 'out of data' at 40 iters issue) #21

Workflow file for this run

name: CPU Unit Test
on:
push:
branches: [ main ]
pull_request:
concurrency:
group: unit-test${{ github.workflow }}-${{ github.ref == 'refs/heads/main' && github.run_number || github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash -l -eo pipefail {0}
jobs:
cpu_unit_tests:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10']
steps:
- name: Check out repo
uses: actions/checkout@v3
- name: Setup conda env
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
miniconda-version: "latest"
activate-environment: test
python-version: ${{ matrix.python-version }}
- name: Update pip
run: python -m pip install --upgrade pip
- name: Install dependencies
run: |
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121
python -m pip install -r requirements.txt
python -m pip install -r dev-requirements.txt
python -m pip install -e .
- name: Run unit tests with coverage
run: pytest test --cov=. --cov-report=xml --durations=20 -vv
- name: Upload Coverage to Codecov
uses: codecov/codecov-action@v3