Skip to content

PRJenkinsSetupFirewall

Jump to bottom
Josh Hursey edited this page Jan 18, 2017 · 11 revisions

Setting up a Jenkins Pull Request Builder from behind a firewall

If you have testing equipment behind a firewall then it can be difficult to use the Open MPI Community Jenkins server since it will not be able to reach your machines from the open internet.

Requirements

  • You will need to setup a Jenkins master internally to your site. We will call this (Internal)Jenkins.
  • Must be able to test internally without any external entity having access and visibility on the test environment or (Internal)Jenkins.
  • (Internal)Jenkins must be able to connect to the GitHub API.
  • (Internal)Jenkins must be able to report build result to the open internet so other developers can view details of build failures.
  • (Internal)Jenkins must be able to connect to the BuildNode, where you will run the tests.

Setup

Setting up the BuildNode

Setting up the environment on the BuildNode

# Setup a log file
touch $PATH_TO_OMPI_TESTS/log/your-log.md
# Push this file to the Gist site
# Make sure to note the URL returned, as you will need the ID for later
# --public makes it publicly available, remove that option if you want it private
$PATH_TO_OMPI_TESTS/jenkins/bin/gist.pl --tokenfile $TOKENFILE -cmd create --public $PATH_TO_OMPI_TESTS/log/your-log.md

# Setup the configuration for the relay system. Start with the template.
cp $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc.sample $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc
$EDITOR $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc

# Setup a crontab entry for the following script
crontab -e
# This script removes gists older than N days (Defined in config.inc)
0 8 * * * $PATH_TO_OMPI_TESTS/jenkins/bin/clean-history.sh $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc

Setting up the (Internal)Jenkins Job

  • Define a default sha1 parameter and a default parameter that will be the default URL pushed to GitHub when the Jenkins job starts.
  -> "This build is parameterized"
     -> "String Parameter"
        -> Name: sha1
        -> Default Value: master
        -> Description: 
     -> "String Parameter"
        -> Name: GISTURL
        -> Default Value: https://gist.github.com
        -> Description: Initial URL for HitHub
  • (Suggested) Additional Behaviors
    • Wipe out repository & force clone
  • Configure the Pull Request Builder
  -> "GitHub Pull Request Builder"
     -> Define "Admin list" (add at least your bot account)
     -> "Advanced"
        -> Trigger phrase (replace 'ibm' with your org): .*bot:(ibm:)?retest.*
        -> Skip build phrase: .*((\[skip\W+ci\])|(\[ci\W+skip\])|(bot:notest)).*
        -> Crontab line (poll every 5 min): H/5 * * * *
        -> White list: open-mpi (your bot name)
        -> List of organizations. Their members will be whitelisted.: open-mpi
        -> (check) Build every pull request automatically without asking (Dangerous!).
        -> Whitelist Target Branches: master
     -> Poll SCM
        -> Schedule (should match earlier crontab line): H/5 * * * *
     -> "Trigger Setup"
        -> "Update commit status during build"
           # The "Commit Status Context" is what is displayed to GitHub next to the build status.
           # Usually seen as the "Details" link
           # So if you have multiple Jenkins jobs that run on a PR then distinguish them via
           # this string.
           Commit Status Context: YOUR-COMPANY-CI
           # This will be the URL posted next to the "Commit Status Context" on GitHub.
           # This environment variable is set in a .properties.txt file (in next step)
           Commit Status URL: ${GISTURL}
           Commit Status Build Triggered:
           Commit Status Build Started:
       -> "Build Status Messages" (optional if you want it to add a comment on failed builds)
          Build Result: Failure
          Message: "Test Failed! Some friendly string to put in the Comment field of the PR"
  • "Build" (in this order)
    • "Set build status to pending on GitHub commit"
    • "Execute shell"
#!/bin/bash -xe
# This script does a few things:
# - Post a placeholder file to Gist (we need the ID for later)
# - Save the GISTID to a .properties.txt file (so we can update it later)
# - Save the GISTURL to the .properties.txt file (so we can push it to GitHub later)
# - Create a .env.sh with some information that we will need in the "Execute shell" phase below
source $PATH_TO_OMPI_TESTS/jenkins/bin/pre-build.sh
  • "Inject environment variables"
    • Properties File Path: .properties.txt
  • "Execute shell"
#!/bin/bash -xe

# Pick up the environment variables from the earlier "Execute shell" phase
source $WORKSPACE/.env.sh

#-----------------------
# Redirect output to the file
exec > >(tee -i output.txt)
exec 2>&1

# Jenkins will fail this script at the point it has a non-zero exist status
# so we need a file to see if we made it through the CI script successfully.
touch $WORKSPACE/.in-progress.txt

# Run your CI script here...

# If we get here then everything is fine, so remove the file.
rm $WORKSPACE/.in-progress.txt
  • "Post-build Actions"
    • "Execute a set of scripts"
      • "Build Steps" -> "Execute shell"
#!/bin/bash -x

# This script does a few things
# - Checks for the existance of the $WORKSPACE/.in-progress.txt to determine success/failure
# - Updates the logfile $PATH_TO_OMPI_TESTS/log/your-log.md
# - Pushes the log file to Gist
# - Pushed the output file to Gist
#   - If the build was successful then the last N lines are sent (defined in config.inc)
#   - If the build was not successful then the entire output is sent
source $PATH_TO_OMPI_TESTS/jenkins/bin/post-build.sh
  • "Set build status on GitHub commit"
    • Leave "Content" blank, Select "FAILURE" in "Result on failure"

Assorted notes (work in progress - not formatted correctly)

...

Clone this wiki locally