Health Data Platform

Setting-up a core clinical data repository to store data from different formats using openEHR specifications and the EHRbase API. EHRbase provides a standard-based backend for interoperable clinical applications, implementing the latest version of the openEHR Reference Model and the Archetype Definition Language (AQL).

This service is based on repositories:

• dh-hdp-templates: Custom-made openEHR templates
• dh-hdp-zib-templates: Custom-made openEHR templates matching the Dutch ZIBs
• dh-hdp-etl: ETL Python script
• dh-hdp-transform-rest: Java REST API for data class transformation into openEHR compositions
• dh-hdp-fhir-bridge: Java REST API for converting FHIR messages into openEHR composition and storage into EHRbase
• dh-hdp-notebooks: Jupyter notebooks for an initial data exploration

Requirements

Encryption between filebeat and elk

CA certificates need to be manually stored in folder filebeat/certs. The present files are used for development-purposes.

Add these virtual host entries in your /etc/hosts file

127.0.0.1 ehrbase.local.dh.unimaas.nl
127.0.0.1 transform.local.dh.unimaas.nl
127.0.0.1 fhir-bridge.local.dh.unimaas.nl
127.0.0.1 jupyter.local.dh.unimaas.nl
127.0.0.1 openehrtool.local.dh.unimaas.nl

Template variables

For each new template, add its template_id and semantic version (sem_ver) as variables into the corresponding env file (for the ZIB, use env_files/etl_zib.env). api_route and filename variables can be auto-generated by running the bash script ./env_files/append_template_variables.sh.

Run the stack

Clone the external repositories

./dh.sh externals clone
./dh.sh externals checkout 2024.1

Run the setup requirements

./dh.sh setup

Start the Jupyter notebook for data exploration and live demo

./dh.sh jupyter-zib

Open your browser and try http://jupyter.local.dh.unimaas.nl using the following token:

SERVER_APP_TOKEN=aa3ca297f81ed69a3fcab71ff886d5cf3207be09960f6de7

Start the EHRbase backend

./dh.sh backend

Open your browser and try http://ehrbase.local.dh.unimaas.nl/ehrbase/swagger-ui/index.html with the following credentials:

SECURITY_AUTHUSER=user
SECURITY_AUTHPASSWORD=foobar

Credentials can be updated in ./ehrbase/.env.ehrbase

Run the ETL

Extract data from csv files, Transform the data into valid openEHR compositions by using a REST API and Load the compositions into EHRbase.

ETL workflows specific to ZIB templates.

./dh.sh zib

Run the tests

Start the dev environment and execute all the tests

./dh.sh test

Execute all the tests

./dh.sh run --rm --entrypoint pytest etl-zib --verbose --verbosity=5

Execute a specific class test

./dh.sh run --rm --entrypoint pytest etl-zib --verbose --verbosity=5 tests/test_burgerlijke_staat.py::TestBurgerlijkeStaat2017

Execute a single test

./dh.sh run --rm --entrypoint pytest etl-zib --verbose --verbosity=5 tests/test_all_zib_pipelines.py::TestAllZibPipelines::test_number_of_templates

Recreate the ETL stack

./dh.sh up -d --force-recreate ehrdb ehrbase etl-zib

Kill the whole stack

./dh.sh down

Start FHIR-bridge

Convert FHIR messages into openEHR compositions and them into EHRbase

Before starting, build the image in dh-hdp-fhir-bridge (check the README file in dh-hdp-fhir-bridge for the command)

./dh.sh fhir

Open your browser and try http://fhir-bridge.local.dh.unimaas.nl/fhir-bridge.

Run the following command to run both fhir-bridge and etl-zib:

./dh.sh up -d fhir-bridge && ./dh.sh up -d etl-zib && ./dh.sh logs -f etl-zib fhir-bridge

Start openehr-Tools [for DEV environment only]

Tool for interacting with the EHRbase server with a basic dashboard integrated.

./dh.sh up -d openehrtool

Open your browser and try http://openehrtool.local.dh.unimaas.nl