Refactor Grouper objects #11646

Job	Run time
Event File	2s
detect ci trigger	4s
ubuntu-latest py3.9	8m 32s
ubuntu-latest py3.11	9m 26s
ubuntu-latest py3.12	8m 43s
macos-latest py3.9	15m 14s
macos-latest py3.11	15m 35s
macos-latest py3.12	32m 41s
windows-latest py3.9	17m 55s
windows-latest py3.11	19m 59s
windows-latest py3.12	19m 42s
ubuntu-latest py3.9 bare-minimum	1m 34s
ubuntu-latest py3.9 min-all-deps	6m 50s
ubuntu-latest py3.10 all-but-dask	6m 41s
ubuntu-latest py3.10 flaky	9m 5s
	2h 52m 3s

Provide feedback