tests: Add feature presence test suite

[jakub-vavra-cz]

We need to check that features are present where expected.

[ikerexxe]

Please note my comment at SSSD/sssd-test-framework#133 (comment)

[jakub-vavra-cz]

Please note my comment at SSSD/sssd-test-framework#133 (comment)

Hi @ikerexxe ,
test should test one thing (one feature presence).
If I put everything in one test it would be:

1. Too big and unreadable.
2. fails on first failed check without showing what is fine/passed.
3. Un-maintainable if we do not keep the suite the same across all branches.

[shridhargadekar]

A suggestion: Could we manually verify the package info? These changes don't need testing in henceforth.

[ikerexxe]

Hi @ikerexxe , test should test one thing (one feature presence). If I put everything in one test it would be:

1. Too big and unreadable.

2. fails on first failed check without showing what is fine/passed.

3. Un-maintainable if we do not keep the suite the same across all branches.

You can use parametrization to avoid having a very long test. After all, if you look at the tests you have written they all do the same thing: they check the version of sssd and check that the functionality is (or is not) available in that specific version.
With parametrization the lines of code would be the same, only that the information referring to the functionality (sssd version and feature) would be defined in the parametrization itself. This way we reduce the lines of code to maintain and the checking of each feature is done separately (if a feature fails it does not affect the execution of the rest).

[jakub-vavra-cz]

Hi @ikerexxe , test should test one thing (one feature presence). If I put everything in one test it would be:

1. Too big and unreadable.

2. fails on first failed check without showing what is fine/passed.

3. Un-maintainable if we do not keep the suite the same across all branches.

You can use parametrization to avoid having a very long test. After all, if you look at the tests you have written they all do the same thing: they check the version of sssd and check that the functionality is (or is not) available in that specific version. With parametrization the lines of code would be the same, only that the information referring to the functionality (sssd version and feature) would be defined in the parametrization itself. This way we reduce the lines of code to maintain and the checking of each feature is done separately (if a feature fails it does not affect the execution of the rest).

Hi @ikerexxe I am pondering how to parametrize that apart from having something like dictionary with feature as key and Some kind of list with distribution+minversion+maxversion. Can You elaborate/show me some snippet?

[jakub-vavra-cz]

Looks test for "files-provider" does not work properly on centos 9 as it seems to be present on sssd 2.10 there.
For F39, F40 there seems to be issue with "non-privileged" that is not present on 2.10 there.

I guess these tests need tweaking if this is the intended result.

[ikerexxe]

Hi @ikerexxe I am pondering how to parametrize that apart from having something like dictionary with feature as key and Some kind of list with distribution+minversion+maxversion. Can You elaborate/show me some snippet?

@jakub-vavra-cz that's exactly what I had in mind. If you want to be more comprehensive and take into account the passkey functionality then you should also add the distribution and the version from which it is available.