Setup CI for adapter with service dependencies

[timkpaine]

No description provided.

[timkpaine]

ok this is finally working ok, does not appreciably increase CI/CD time and worth it until we can separate adapters.

[ptomecek]

There are two warnings from pytest that come up:

csp/tests/adapters/test_kafka.py:297
/home/runner/work/csp/csp/csp/tests/adapters/test_kafka.py:297: PytestRemovedIn9Warning: Marks applied to fixtures have no effect
See docs: https://docs.pytest.org/en/stable/deprecations.html#applying-a-mark-to-a-fixture-function
@pytest.mark.skipif(not os.environ.get("CSP_TEST_KAFKA"), reason="Skipping kafka adapter tests")

and

csp/tests/adapters/test_kafka.py:373
/home/runner/work/csp/csp/csp/tests/adapters/test_kafka.py:373: PytestRemovedIn9Warning: Marks applied to fixtures have no effect
See docs: https://docs.pytest.org/en/stable/deprecations.html#applying-a-mark-to-a-fixture-function
@pytest.mark.skipif(not os.environ.get("CSP_TEST_KAFKA"), reason="Skipping kafka adapter tests")

I'm not sure that it is intended for test_raw_pubsub and test_invalid_topic to be tagged with @pytest.fixture. These two tests don't appear in the list of tests that were run via the action either.

[timkpaine]

There is a race condition in one of the tests it seems, wanted to note here in case it comes up. I cannot fully hammer it down, and unclear if its an actual issue with the historical->realtime switch or if the test itself has buggy assumptions.

[robambalu]

There is a race condition in one of the tests it seems, wanted to note here in case it comes up. I cannot fully hammer it down, and unclear if its an actual issue with the historical->realtime switch or if the test itself has buggy assumptions.

do you have an example link

[timkpaine]

https://github.com/Point72/csp/actions/runs/10149403062/job/28067369034

[robambalu]

https://github.com/Point72/csp/actions/runs/10149403062/job/28067369034

Based on the test and the error message it looks like whats happening is that the sub is missing the first pub. Its possible the daemon is late to register the subscriber before the first publish. One way to try and fix it is to delay the first pub a bit. Another possibility is to compare the last N msgs rather than the first N