Discuss the use of multiple requirements files and how to improve their use

[MMelQin]

Place holder for discussing how best to ensure env are set up correctly for using and developing apps as well App SDK itself.

[vikashg]

AFAIK, the current process of installation of monai-deploy-app-sdk is as follows

1. git clone the monai-deploy-app-sdk repo
2. switch to the appropriate branch
3. clone the monai core repo
4. copy everything in the monai-core/monai into the monai-deploy-app-sdk/monai folder
5. Make sure that the contents in the monai are in the same directory level.
6. Install using pip3 install -e .

This might work sometimes but will not work if there are some dependency conflict between monai-core packages which are copied manually and monai-deploy app-sdk and will lead to installation conflicts as shown below

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-gpu 2.4.1 requires absl-py~=0.10, which is not installed.
tensorflow-gpu 2.4.1 requires grpcio~=1.32.0, which is not installed.
tensorflow-gpu 2.4.1 requires protobuf>=3.9.2, which is not installed.
tensorflow-gpu 2.4.1 requires six~=1.15.0, which is not installed.
tensorflow-gpu 2.4.1 requires tensorboard~=2.4, which is not installed.
tensorflow-gpu 2.4.1 requires tensorflow-estimator<2.5.0,>=2.4.0, which is not installed.
statsmodels 0.12.0 requires pandas>=0.21, which is not installed.
statsmodels 0.12.0 requires scipy>=1.1, which is not installed.
scikit-learn 0.23.2 requires scipy>=0.19.1, which is not installed.
scikit-image 0.17.2 requires matplotlib!=3.0.0,>=2.0.0, which is not installed.
scikit-image 0.17.2 requires pillow!=7.1.0,!=7.1.1,>=4.3.0, which is not installed.
scikit-image 0.17.2 requires scipy>=1.0.1, which is not installed.
resampy 0.2.2 requires scipy>=0.13, which is not installed.
resampy 0.2.2 requires six>=1.3, which is not installed.
patsy 0.5.1 requires six, which is not installed.
noisereduce 1.1.0 requires matplotlib, which is not installed.
noisereduce 1.1.0 requires scipy, which is not installed.
noisereduce 1.1.0 requires tqdm, which is not installed.
librosa 0.8.1 requires packaging>=20.0, which is not installed.
librosa 0.8.1 requires scipy>=1.0.0, which is not installed.
keras-preprocessing 1.1.2 requires six>=1.9.0, which is not installed.
imageio 2.9.0 requires pillow, which is not installed.
h5py 2.10.0 requires six, which is not installed.
antspyx 0.2.4 requires matplotlib, which is not installed.
antspyx 0.2.4 requires nibabel, which is not installed.
antspyx 0.2.4 requires pandas, which is not installed.
antspyx 0.2.4 requires Pillow, which is not installed.
antspyx 0.2.4 requires scipy, which is not installed.
app 1.1.0 requires matplotlib, which is not installed.
app 1.1.0 requires opencv-python, which is not installed.
app 1.1.0 requires pillow, which is not installed.
app 1.1.0 requires scipy, which is not installed.
tensorflow-gpu 2.4.1 requires numpy~=1.19.2, but you have numpy 1.23.0 which is incompatible.

For this system of installation to work one suggestion is that with every pull request the author should specify from which version of monai-core (including the commitId) the user (developer) should copy the folders. This would solve the problem in short term atleast.

[MMelQin]

The above mentioned steps have been used as the workaround for developing App SDK modules with MONAI Core dependency, though with some caveats (in below the second point):

• Discussion was held on if App SDK should have its own package root path or under monai, anticipating the issues we are facing now where the monai core cannot be pip installed with App SDK in edit mode locally and depending on monai modules. I raised the concern but the other opinion is that we should do as Google where all Google modules are under under google, even though all MONAI project do NOT have a mono repo. We'll need to revisit this.
• For the dependency conflicts, I think it is because the requirements.txt from MONAI core is used. For my setup, I had deliberately avoided coping over or using any requirements.txt from MONAI core (version tagged) repo source code, for 1) App SDK is closely following the Core with the few minimal dependency packages, 2) the core modules that are imported in the App SDK are few, whose dependencies are all satisfied with the requirements in App SDK (the dev and example versions of the requirements).

Improving the experience on developing the App SDK with MONAI is definitely one of the key points we'd like to discuss.

[MMelQin]

Hope to provide more context and a few use case for dev env setup

• For most common dev env setup, using the released MONAI package, follow the steps in The Contribution Process, ensuing ./run setup in a virtual env.
• If added functionality in the pre-release MONAI package is needed, please pip install the monai-weekly in the virtual env before executing the ./run setup command.
• In the rare cases, where the latest commit in the monai core repo is needed and one cannot wait till the monai-weekly is published, one can ./run setup, uninstall monai, then treat monai core source code and the App SDK source as if in the same repo by pulling the monai code and copy the module folders into the same monai folder in the App SDK, or creating soft link from App SDK to the monai core folders.