Skip to content

Latest commit

 

History

History
172 lines (112 loc) · 5.7 KB

DEVELOPERS.md

File metadata and controls

172 lines (112 loc) · 5.7 KB

Developer documentation

Local development

Development credentials

Note: you will need the Bitwarden CLI tool installed in order to access passwords, but it is not a requirement.

  • There is an existing dotenv-sample template that you can use to base your own .env file on.
  • Use bw to login to the Bitwarden account.
  • When logged in to Bitwarden, run scripts/dev-env.sh <env_file_target> to retrieve and write the credentials to the target environment file specified.
    • .env is already in .gitignore to help prevent an accidental commit of credentials.

Native

Prerequisites

  • Python v3.9.x
  • virtualenv
  • Pip
  • Node.js v16.x (fnm is recommended)
  • npm v7.x

just commands

Each just command sets up a dev environment as part of running it. If you want to maintain your own virtualenv make sure you have activated it before running a just command and it will be used instead.

Steps

Set up an environment

just devenv

Run migrations:

python manage.py migrate

Optionally set up 1 or more administrators by setting ADMIN_USERS to a list of strings. For example: ADMIN_USERS=ghickman,ingelsp.

Note: this can only contain usernames which exist in the database. If necessary, you can create the required user(s) first with:

python manage.py createsuperuser

Then update their User records with:

python manage.py ensure_admins

Build the assets:

npm run build

Run the dev server:

just run

Access at localhost:8000

Docker Compose

Run docker-compose up.

Note: The dev server inside the container does not currently reload when changes are saved.

Frontend development (CSS/JS)

This project uses Vite, a modern build tool and development server, to build the frontend assets. Vite integrates into the Django project using the django-vite package.

Vite works by compiling JavaScript files, and outputs a manifest file, the JavaScript files, and any included assets such as stylesheets or images.

Vite adds all JavaScript files to the page using ES6 Module syntax. For legacy browsers, this project is utilising the Vite Legacy Plugin to provide a fallback using the module/nomodule pattern.

For styling this project uses Scss to compile the stylesheets, and then PostCSS for post-processing.

Running the local asset server

Vite has a built-in development server which will serve the assets and reload them on save.

To run the development server:

  1. Update the .env file to DJANGO_VITE_DEV_MODE=True
  2. Open a new terminal and run npm run dev

This will start the Vite dev server at localhost:3000 and inject the relevant scripts into the Django templates.

Compiling assets

To view the compiled assets:

  1. Update the .env file to DJANGO_VITE_DEV_MODE=False
  2. Run npm run build
  3. Run python manage.py collectstatic

Vite builds the assets and outputs them to the assets/dist folder.

Django Staticfiles app then collects the files and places them in the staticfiles/assets folder, with the manifest file located at staticfiles/manifest.json.

Deployment

It is currently configured to be deployed Heroku-style, and requires the environment variables defined in dotenv-sample.

The DataLab job server is deployed to our dokku2 instance, instructions are are in INSTALL.md.

Testing

Run the tests with:

just test

More details on testing can be found in TESTING.md.

Adding new backends

Steps to add a backend

  1. Add an empty migration with python manage.py makemigrations jobserver --empty --name <meaningful name>.
  2. Create your Backend(s) in a function via RunPython in the new migration file.
  3. Fix the tests (some will check the number of migrations).

Why add a backend?

Backends in this project represent a job runner instance somewhere. They are a Django model with a unique authentication token attached.

We generate them with a migration, but configure which ones are available to a deployment with the BACKENDS environment variable.

This has allowed us some benefits:

  • API requests can be tied directly to a Backend (eg get all JobRequests for TPP).
  • Adding a new Backend via environment variables allows for easy local testing.
  • Per-Backend API stats collection is trivial because requests are tied to a Backend via auth.

However it comes with sticky parts too:

  • You need to create them via migration and enable them via the environment.
  • The tests are inherently tied to the number of Backends because we want to test the default manager filters them by the names configured in the environment variable.