Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Singularity guide #120

Merged
merged 5 commits into from
Dec 14, 2024
Merged

Conversation

nathanweeks
Copy link
Contributor

Updates the Singularity guide:

  • Set rserver --server-user in job script (singularity on HPC with slurm rocker-versioned2#837)
  • Show where R_PROFILE_USER and R_LIBS_USER can be set
    (Specify .Rprofile and .Renviron when using singularity  rocker-versioned2#855)
  • Use rocker/rstudio:4.4.2
  • Specify python3 instead of python (the latter may be python 2 or absent on some hosts)
  • Mention Apptainer
  • mktemp is simpler and portable-enough (GNU coreutils) for creating temp dirs
  • Simplify creation of various writable directories in container with --scratch and --workdir
  • Ensure R_LIBS_USER directory exists
  • no longer need to create readable database.conf (sqlite seems to be the default)
  • SLURM_CPUS_ON_NODE technically more correct than SLURM_JOB_CPUS_PER_NODE
    (though shouldn't make a difference with a single-node allocation)

Copy link
Member

@cboettig cboettig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@eitsupi eitsupi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

use/singularity.md Show resolved Hide resolved
use/singularity.md Show resolved Hide resolved
use/singularity.md Show resolved Hide resolved
use/singularity.md Outdated Show resolved Hide resolved
use/singularity.md Show resolved Hide resolved
use/singularity.md Outdated Show resolved Hide resolved
export R_LIBS_USER=${HOME}/R/rocker-rstudio/4.2
export OMP_NUM_THREADS=${SLURM_CPUS_ON_NODE}
export R_LIBS_USER=${HOME}/R/rocker-rstudio/4.4.2
mkdir -p "\${R_LIBS_USER}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe typo?

Suggested change
mkdir -p "\${R_LIBS_USER}"
mkdir -p "${R_LIBS_USER}"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The \ was used so the resulting rsession.sh script that is generated (which is executed in the container) contains (literally):

mkdir -p "${R_LIBS_USER}"

instead of expanding R_LIBS_USER in the here-document to the value it has in the host environment. This is admittedly somewhat subtle---which isn't great for readability/maintainability---though there is another instance where a backslash escape is used in the here-document (exec /usr/lib/rstudio-server/bin/rsession "\${@}").

For more consistency, I guess there's no reason why there couldn't be a backslash escape in:

export R_LIBS_USER=\${HOME}/R/rocker-rstudio/4.4.2

as home directories are $HOME on the host should be the same as $HOME in the container (the user's home directory is mounted by default in a Singularity container).

I'm probably splitting hairs at this point :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The \ was used so the resulting rsession.sh script that is generated (which is executed in the container) contains (literally):

In that case, can't you use "END" instead of END?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value of ${SLURM_CPUS_ON_NODE} from the host was being used in the here-doc. However... it seems this should no longer be necessary for OpenBLAS, which should detect the number of CPUs in the cgroup when OMP_NUM_THREADS is unset (OpenMathLib/OpenBLAS#1155) --- I suspect the vast majority of Slurm clusters today use cgroups to constrain processes to resources allocated in the job script --- and (for other OpenMP applications) libgomp seems to use the number of CPUs in the cgroup for the default number of threads in a parallel region (tested with a small C program compiled/run in both rocker/rstudio:4.0.0 and rocker/rstudio:4.4.2 environments).

I'm going to go out on a limb and remove the explicit OMP_NUM_THREADS environment variable assignment in rsession.sh (and use the cleaner <<"END" here-document syntax you suggested).

Quote here-document word to disable shell expansion
Copy link
Member

@eitsupi eitsupi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, let's merge for now.

@eitsupi eitsupi merged commit 25b0f2c into rocker-org:master Dec 14, 2024
2 checks passed
@nathanweeks nathanweeks deleted the singularity branch December 14, 2024 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants