Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark script (gsplat evaluation metrics) #324

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

maturk
Copy link
Collaborator

@maturk maturk commented Aug 8, 2024

Note, this PR does not have to be merged. It can stay as a PR or if people find it useful, it can be merged.

As mentioned in #323, here is a benchmark script to run various configurations of the gsplat library. This script was used to reproduce the metrics shown in 1cc3d22.

This script is inspired by @niujinshuchong work.

This script is usable for multi-gpu batch runs.

It makes a queue of jobs of the form [(config_0, scene_0), (config_0, scene_1), ... (config_0, scene_n), (config_1, scene_0), ...] and available GPUs take a config from the queue and processes it.

Configs can be defined easily. E.g.

baseline_config = BenchmarkConfig()
mcmc_config = BenchmarkConfig(
    trainer="simple_trainer_mcmc.py",
    result_dir="results/mcmc",
)
# Configs to run in batch
configs_to_run = [baseline_config, mcmc_config]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant