Running Regression Tests Against PR Branches
To run the full JWST regression test suite against code contained in a PR branch, go to the RT Jenkins pipeline page that's dedicated to making special runs for developers: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/
Before running the tests, you need to configure the Jenkins pipeline to use the PR code branch. To do this, first click the "Configure" link on the left hand side of the page:
Second, on the new page that appears, scroll down to the Pipeline section and update the Repository URL and Branch Specifier fields to select a branch of jwst. The default settings for these, which will test against the master branch, are "https://github.com/spacetelescope/jwst.git" and "*/master". To test against a particular PR branch, set the Repository URL to the developer's repository, e.g. "https://github.com/mcara/jwst.git" and set Branch Specifier to the PR branch name, e.g. "*/fix-tweakreg-mltichip-test":
Scroll down and click the "SAVE" button to save your settings and return to the main page. Click "Build with Parameters" (just above "Configure"), which summons the following form:
The branch of jwst has already been selected by the previous steps, but if you need a dev branch of one or more dependencies, enter them as pip requirements in the OVERRIDE_REQUIREMENTS parameter. For example, to override stcal to a dev branch, add a line like this:
git+https://github.com/<username>/stcal.git@<branch>
Press BUILD to start the tests. A new job should appear on the pipeline's main page.
If you would like to re-run the job on the same branch, e.g. if you commit and push new changes that fix a test failure from a previous run, it is best practice to instantiate a completely new job (i.e., go through the steps above again). Do not use the "Replay" button. The reason is that someone else may have changed the configuration of JWST-Developers-Pull-Requests in the meantime.
The Jenkins run can be configured to run only a subset of the test suite. Usually, the preferred method to do this is to run the tests locally; see Maintaining Regression Tests. However, in some cases this functionality is useful, e.g., to diagnose why a test passes locally but fails on Jenkins. To do it, run the Configure steps as above, then on the Pipeline JWST-Developers-Pull-Requests page, modify the PYTEST_ARGS field. This field accepts any typical arguments to pytest, such as <test_whatever.py> or <-k name_of_test>.