This repository has been archived by the owner on Jan 25, 2018. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1
CellProfiler/CellProfiler-rpm
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
This document describes the setup for building Linux packages (RPMs) of CellProfiler and its dependencies and publishing them on the web. SSH is set up to allow logging into the build machines without paswords, but there have been some problems with this due to a CentOS 6 bug. You should make RSA public and private key files (id_rsa and id_rsa.pub) in the subdirectory, `ssh_keys`. Then, run `restorecon -R -v /root/.ssh` to make password-less login work on CentOS 6. ## Environment Make sure to use Python 2.6 or 2.7, then activate the virtual environment . venv/bin/activate ## Changing Centos 6 RPM dependencies in CellProfiler CellProfiler has a spec file stored in its GIT repository: jenkins/linux/cellprofiler-centos6.spec Each commit has its own set of dependencies because the spec file is part of the commit. Dependencies can be modified and added by editing the "Requires" and "BuildRequires" sections of the .spec file. ## Files and directories fabfile.py : Fabric tasks for building and testing. htdocs/linux : This is the directory that is exposed on the web. It contains a subdirectory : for each Linux distribution. For RPM-based distributions such as CentOS, that : directory is a yum repository created by createrepo (see below). ssh_keys : Contains SSH keys that are installed on the build and test machines by some : of the fabric tasks. SPECS : RPM spec files. ## How to rebuild only CellProfiler 0. Edit SPECS/cellprofiler.spec and update the version, release, and/or tarname. The tarname should be the git hash. Example: %define version 2.1.0.Release %define release 1 %define tarname 0c7fb94 1. Activate the virtualenv. . venv/bin/activate 2. Make a working directory. This directory will contain RPMs as you build them and test; you will finally copy them to the website. $ mkdir foo $ cd foo 3. Start a clean build machine by going to http://broad.io/imagingcloud, logging in, and add a vApp based on the template "Cent OS 6 x64" in the Build Catalog. 4. Wait for the build machine to start and get its IP address, 192.168.195.XXX. 5. Upgrade the operating system and install packages needed to build. $ fab -H 192.168.195.XXX deploy_build_machine 6. Build the cellprofiler RPM. $ fab -H 192.168.195.XXX build_cellprofiler_only 7. Start a clean test machine by going to http://broad.io/imagingcloud, logging in, and add a vApp based on the template "Cent OS 6 x64" in the Build Catalog. 8. Wait for the test machine to start and get its IP address, 192.168.195.YYY. 9. Upgrade the operating system and install packages needed to test. $ fab -H 192.168.195.YYY deploy_test_machine 10. Tell the test machine to get the dependencies from the public repository. $ fab -H 192.168.195.YYY use_public_repo 11. Copy the new RPM to the test machine. $ fab -H 192.168.195.YYY push 13. Install CellProfiler on the test machine. $ fab -H 192.168.195.YYY install_cp (Not 100% sure that this will get the newly built version and not the version that is in the public repository. Check.) 14. Log in and check that it works to start CellProfiler. $ ssh -Y johndoe@192.168.195.YYY cellprofiler 15. Add the new RPM files to the public repository. $ cp *.rpm ../htdocs/linux/centos6/ $ createrepo ../htdocs/linux/centos6/ 16. Delete the working directory. $ cd .. $ rm -rf foo ## How to build a particular version 1. Create an OAUTH token at github.com (needed to allow the script to make unlimited calls to the github API). Go to https://github.com/settings/applications and create a personal access token with all permissions unchecked. 2. Create and cd into a temporary directory 3. Run fab like this: $ fab -H 192.168.xxx.xxx build_cellprofiler_version:<git-hash>,<token> ## How to rebuild everything 0. Activate the virtualenv. . venv/bin/activate 1. Make a working directory. This directory will contain RPMs as you build them and test; you will finally copy them to the website. $ mkdir foo $ cd foo 2. Start a clean build machine by going to http://broad.io/imagingcloud, logging in, and add a vApp based on the template "Cent OS 6 x64" in the Build Catalog. 3. Wait for the build machine to start and get its IP address, 192.168.195.XXX. 4. Upgrade the operating system and install packages needed to build. $ fab -H 192.168.195.XXX deploy_build_machine 5. Copy the contents of the SOURCES directory to the build machine. $ fab -H 192.168.195.XXX push_sources 6. Build all the RPMs. $ fab -H 192.168.195.XXX maybe_build_all_rpms 7. If something won't build, modify the .spec file as needed and repeat step 6 until it works. If you add a file to SOURCES, run step 5 again. 8. Copy all the RPMs from the build machine. $ fab -H 192.168.195.XXX pull 9. Start a clean test machine by going to http://broad.io/imagingcloud, logging in, and add a vApp based on the template "Cent OS 6 x64" in the Build Catalog. 10. Wait for the test machine to start and get its IP address, 192.168.195.YYY. 11. Upgrade the operating system and install packages needed to test. $ fab -H 192.168.195.YYY deploy_test_machine 12. Copy the RPMs to the test machine. $ fab -H 192.168.195.YYY push 13. Install CellProfiler on the test machine. $ fab -H 192.168.195.YYY install_cp 14. Log in and start CellProfiler. $ ssh -Y johndoe@192.168.195.YYY cellprofiler 15. Add the new RPM files to the public repository. $ cp *.rpm ../htdocs/linux/centos6/ $ createrepo ../htdocs/linux/centos6/ 16. Delete the working directory. $ cd .. $ rm -rf foo ## How to build one thing 0. Activate the virtualenv. . venv/bin/activate 1. Make a working directory. This directory will contain RPMs as you build them and test; you will finally copy them to the website. $ mkdir foo $ cd foo 2. Start a clean build machine by going to http://broad.io/imagingcloud, logging in, and add a vApp based on the template "Cent OS 6 x64" in the Build Catalog. 3. Wait for the build machine to start and get its IP address, 192.168.195.XXX. 4. Upgrade the operating system and install packages needed to build. $ fab -H 192.168.195.XXX deploy_build_machine 5. Copy the contents of the SOURCES directory to the build machine. $ fab -H 192.168.195.XXX push_sources 6. Push the existing RPMs to the build machine $ fab -H 192.168.195.XXX push 7. Use public repositories $ fab -H 192.168.195.xxx use_public_repo 8. Make the single RPM, for instance cellprofiler-foo $ fab -H 192.168.195.XXX maybe_build_rpm:<package-name> 9. Copy all the RPMs from the build machine. $ fab -H 192.168.195.XXX pull ## Reindexing a yum repository After adding or removing RPM files from a yum repository, run createrepo in order to recreate the index files. For instance: createrepo htdocs/linux/centos6 ## Testing CellProfiler fab -H 192.168.195.XXX deploy_test_machine fab -H 192.168.195.XXX install_cp ## How the virtualenv was made The directory venv is a python virtualenv that contains software needed to start builds and create repositories. The virtualenv was made as follows: 1. Create virtualenv virtualenv --prompt='(linux_repositories)' venv 2. Activate virtualenv . venv/bin/activate 3. Install createrepo mkdir venv/src cd venv/src wget http://createrepo.baseurl.org/download/createrepo-0.3.6.tar.gz tar xzf createrepo-0.3.6.tar.gz cd createrepo-0.3.6 patch < ../../../createrepo-sysconfdir.patch make prefix=$(dirname $(dirname $(pwd))) install cd ../../.. # ManuallyrReplace /usr/share with $(dirname $(dirname "$0")) in # venv/bin/createrepo. 4. Install fabric pip install fabric
About
No description, website, or topics provided.
Resources
Code of conduct
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published