Skip to content

Commit

Permalink
README.md update
Browse files Browse the repository at this point in the history
Added an Example usage
  • Loading branch information
GMW99 committed Nov 16, 2023
1 parent 82ca938 commit b8aa25c
Showing 1 changed file with 37 additions and 0 deletions.
37 changes: 37 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,40 @@ Please follow the [install guide](https://rosalindfranklininstitute.github.io/Di
More than likely, especially in the case where your application is I/O bound, as this will allow you to leverage the RAM on the HPC node as shared storage and thus will more than likely rate limited by the HPC network rate and not the storage write rate. However, DisTRaX can also potentially reduce the time for compute-bound operations that need to read data to do processing, such as [RELION](https://relion.readthedocs.io/), as having the data in the shared RAM will reduce the read time.

DisTRaX can also, in a cloud cluster setting, help reduce cost overheads as if the data fits into the RAM of the compute nodes, it can remove the need for a fast central filesystem.

In short, DisTRaX allows you to use existing compute infrastructure to speed up your software while requiring no changes to your existing software.

## Example

This example shows how easy it is to use DisTRaX with your existing software.

```bash
#! /bin/bash
#SBATCH --job-name=distrax_example
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=2
#SBATCH --cpus-per-task=1
#SBATCH --time=10:00

# Get hostnames
hostnames=`scontrol show hostnames| sed -n -e 'H;${x;s/\n/,/g;s/^,//;p;}'`

# Get number of hosts
number_of_hosts=`scontrol show hostnames | wc -l`

# Create Storage System
mpirun -np $number_of_hosts --host $hostnames distrax_mpi -c distrax.cfg -a create

# Move data to storage cluster
rsync input_data_location /mnt/distrax/

# Run Application
./Heavy-IO-application --input_dir /mnt/distrax --output_dir /mnt/distrax

# Move data off storage cluster
rsync /mnt/distrax permanent_output_data_location

# Remove Storage System
mpirun -np $number_of_hosts --host $hostnames distrax_mpi -c distrax.cfg -a remove

```

0 comments on commit b8aa25c

Please sign in to comment.