Add example offloading template #1573

stognini · 2025-01-13T16:08:59Z

This PR adds a new example that acts more as a template structure for users to learn how to either add Celeritas capabilities to their existing framework or build a new Geant4 application (with offloading capabilities) from scratch.

It is technically very similar to the existing simple-offload.cc, but it provides more directions to the user, as it is much more similar to the well known examples found in Geant4. The main reasons to add it, in my opinion, is that it provides:

A short, independent README with code directions and explanations;
A very simple and clear CMakeLists.txt that teaches the user how to link a Celeritas' install against their project; and
An equivalent and well known code structure organization, in terms of .hh/.cc files, as any Geant4 example, which should make direct comparisons to existing Geant4 codes more intuitive.

github-actions · 2025-01-13T17:03:13Z

Test summary

0 tests 0 ✅ 0s ⏱️
0 suites 0 💤
0 files 0 ❌

Results for commit 8895050.

♻️ This comment has been updated with latest results.

sethrj

Let's move the example up a directory, make sure it shows up in the manual, and add it to scripts/ci/test-examples.sh

example/accel/offload-template/src/Celeritas.cc

example/accel/offload-template/CMakeLists.txt

example/accel/offload-template/README.md

example/accel/offload-template/src/ActionInitialization.cc

…gManager instead of TrackingAction

whokion · 2025-01-14T15:48:57Z

A couple of general comments:

Use G4RunManagerFactory instead of G4RunManager (also use G4PhysListFactory if there is no custom physics list)
Use G4VTrackingManager (i.e, add some like OffloadingTrackingManager to implement an example of those pure/virtual methods - see an example at CMSEmStandardPhysicsTrackingManager)

whokion

A good start, but may need some refinement to make the example more general.

example/offload-template/CMakeLists.txt

example/offload-template/README.rst

whokion · 2025-01-14T16:11:57Z

example/offload-template/main.cc

+    }
+
+    // Construct run manager
+    G4RunManager run_manager;


Suggested change

G4RunManager run_manager;

auto runManager = G4RunManagerFactory::CreateRunManager(G4RunManagerType::Default);

Would this make it MT if Geant4 is built with threading?

I actually made this a G4RunManagerType::Serial to run a single-thread execution, and just added a comment on the MT case.

Edit: My reasoning to keep it single-threaded is so that we don't have to also require an install with GEANT4_BUILD_MULTITHREADED=ON as a condition just to test a tiny offload example.

GEANT4_BUILD_MULTITHREADED=ON is now the default (so, you do not need explicitly to set it), which then is able to run both sequential and tasking (default tasking option is the traditional old MT). So it will support all cases.

whokion · 2025-01-14T16:17:38Z

example/offload-template/src/ActionInitialization.cc

+    // Construct Celeritas offloading interface
+    CelerSimpleOffload().Build(
+        &CelerSetupOptions(), &CelerSharedParams(), &CelerLocalTransporter());


Should this be in BuildForMaster (assuming that the default run manager (tasking run manager) is used)?

Not really, because it is not an MT application. But good point, I may have to either make it an MT example, or add a few notes throughout it about the MT case.

It probably should support MT? Most new Geant4 apps should.

Yeah, I thought of that. But then we have to require a Geant4 install with GEANT4_BUILD_MULTITHREADED=ON for a tiny example. Fairly, that would be standard for most cases, but seemed simpler to run a single-threaded one, so any compilation works, and explain how MT should be handled in a comment.

The BuildForMaster member function can be supplied always. If Geant4 is not MT enabled, it'll never be called, but by being there both cases can be supported.

example/offload-template/src/DetectorConstruction.cc

example/offload-template/CMakeLists.txt

whokion · 2025-01-14T19:17:48Z

2. Use G4VTrackingManager (i.e, add some like OffloadingTrackingManager to implement an example of those pure/virtual methods - see an example at [CMSEmStandardPhysicsTrackingManager](https://cmssdt.cern.ch/lxr/source/SimG4Core/PhysicsLists/src/CMSEmStandardPhysicsTrackingManager.cc))

Sorry that I did not know that TrackingManagerOffload already implemented and you already used the interface. Your approach is good.

example/offload-template/main.cc

drbenmorgan · 2025-01-15T15:12:22Z

example/offload-template/src/ActionInitialization.cc

+    // Construct Celeritas offloading interface
+    CelerSimpleOffload().Build(
+        &CelerSetupOptions(), &CelerSharedParams(), &CelerLocalTransporter());


The BuildForMaster member function can be supplied always. If Geant4 is not MT enabled, it'll never be called, but by being there both cases can be supported.

example/offload-template/src/ActionInitialization.cc

sethrj

Looks good, but please take a look at the couple remaining suggestions.

example/offload-template/src/DetectorConstruction.cc

scripts/ci/test-examples.sh

sethrj · 2025-01-17T11:45:00Z

Hey @drbenmorgan , I've been trying to understand some weird behavior in the multiple cmake package scripts we use and thought you might know... https://github.com/celeritas-project/celeritas/actions/runs/12812027794 shows @stognini 's example failing for Geant4 11.1 and 11.2 .

I finally realized we'd already "solved" this issue before: https://github.com/celeritas-project/celeritas/blob/eb9b0d721354503f12d3fbf18be0171225dcfdb8/cmake/CeleritasConfig.cmake.in#L70--L72 is due to #1152 , which notes "issues with the new geant4 cmake variable cache break[ing] xercesc importing for 11.1–11.2". For some reason though our previous fix fails when Celeritas isn't marked REQUIRED: see e4eed68. (Maybe it invalidates the find_dependency cache?)

Do you have any ideas how to better fix this? Thanks!

drbenmorgan · 2025-01-17T12:26:27Z

Argh, this looks like the same issue as in apt-sim/AdePT#267, the ~~hack~~ fix there was to explicitly find Xerces before anything else. I'll have to go back to that and the reproducer to try and properly track this down upstream...

sethrj · 2025-01-17T12:41:10Z

Yeah, we added that and it worked as long as celeritas and Geant were both required 😅

drbenmorgan · 2025-01-17T13:03:20Z

O.k., no better ideas from my side at the moment - will try and take a look on the VecGeom etc side next week...

sethrj · 2025-01-17T15:17:22Z

Aha, I see in the vecgeom config file:

if(NOT DEFINED XercesC_INCLUDE_DIR)
  set(XercesC_INCLUDE_DIR "/opt/spack/opt/spack/sequoia/xerces-c/3.2.5/3zdtn2m/include")
endif()

... which I introduced in VecGeom 52f767d29f, which maybe is conflicting with something in Geant4 and/or XercesC...

This reverts commit e4eed68.

stognini added 2 commits January 10, 2025 16:23

Draft Celeritas offload template example

d7d8c23

Use simple offload interface

3522121

stognini requested review from sethrj and whokion January 13, 2025 16:09

stognini added the documentation Documentation, examples, tests, and CI label Jan 13, 2025

stognini added 3 commits January 13, 2025 11:12

Minor improvements

c38da6b

Add missing copyright note

93344a7

Fixup

7cda4be

sethrj reviewed Jan 13, 2025

View reviewed changes

stognini added 7 commits January 13, 2025 16:59

Add offload-template to doc

cf3a6c8

Print error message in CMakelists if Celeritas not found; Use Trackin…

7ab16f6

…gManager instead of TrackingAction

Move to example/

1e0c3bd

Update README and doxygen comments

71fca31

Add offload-template to test-examples.sh

136923c

Update readme

2b0c86a

Fixup

691db71

whokion reviewed Jan 14, 2025

View reviewed changes

sethrj reviewed Jan 14, 2025

View reviewed changes

example/offload-template/CMakeLists.txt Outdated Show resolved Hide resolved

example/offload-template/CMakeLists.txt Outdated Show resolved Hide resolved

stognini and others added 4 commits January 14, 2025 14:34

Remove extra variables in CMakeLists; Add comments for MT execution

e60bfeb

Merge branch 'develop' into example-template

4a24b74

Run offload-template only with Geant4 v11

d52259d

Tweak documentation

cd760cd

drbenmorgan reviewed Jan 15, 2025

View reviewed changes

stognini added 3 commits January 15, 2025 10:35

Change to multithread

5995dc0

Fixup

bfa3330

Improve documentation

149a02f

sethrj approved these changes Jan 15, 2025

View reviewed changes

example/offload-template/src/DetectorConstruction.cc Outdated Show resolved Hide resolved

example/offload-template/src/DetectorConstruction.cc Outdated Show resolved Hide resolved

scripts/ci/test-examples.sh Outdated Show resolved Hide resolved

stognini and others added 5 commits January 15, 2025 14:42

Fix 11.1 to 11.3 build issues, maybe

66211f0

Address comments

cc74293

Just use auto*, since G4 functions are not marked as const

e51c9c6

Change cmake policy placement

6f6e132

REVERTME: minimize CI

4ce08ad

sethrj force-pushed the example-template branch from f7e6e40 to 4ce08ad Compare January 16, 2025 11:28

sethrj added 6 commits January 16, 2025 06:37

Update policy and include FindCeleritas

2f75fcc

Try switching order for now

0af9cfa

Fix/add ans colors to ci script

6e5adf7

Make 'required' match between geant4 and celeritas

e85e2a7

Check that celeritas uses geant4

e2a6722

OK fucking make them required then

e4eed68

sethrj added 7 commits January 17, 2025 10:48

Revert "OK fucking make them required then"

beea6de

This reverts commit e4eed68.

Separtely set GDML variable to look for exact XercesC

8d993c2

Merge remote-tracking branch 'upstream/develop' into example-template

dfbe718

Wrap Celeritas config script in policy

186a1b0

Export cache variables as cache

eaf6c30

Save cache variables as cache

c15f9c6

Reapply policy

3024e9e

sethrj marked this pull request as draft January 17, 2025 19:42

This was referenced Jan 17, 2025

Use cmake policy instead of minimum jrmadsen/PTL#53

Open

Fix weird edge cases when trying to find_package(Celeritas) #1586

Open

Add minor comment

8895050

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add example offloading template #1573

Add example offloading template #1573

stognini commented Jan 13, 2025 •

edited

Loading

github-actions bot commented Jan 13, 2025 •

edited

Loading

sethrj left a comment

whokion commented Jan 14, 2025 •

edited

Loading

whokion left a comment

whokion Jan 14, 2025

sethrj Jan 14, 2025

stognini Jan 14, 2025 •

edited

Loading

whokion Jan 14, 2025

whokion Jan 14, 2025

stognini Jan 14, 2025

sethrj Jan 14, 2025

stognini Jan 14, 2025

drbenmorgan Jan 15, 2025

whokion commented Jan 14, 2025

drbenmorgan Jan 15, 2025

sethrj left a comment

sethrj commented Jan 17, 2025

drbenmorgan commented Jan 17, 2025

sethrj commented Jan 17, 2025

drbenmorgan commented Jan 17, 2025

sethrj commented Jan 17, 2025 •

edited

Loading

	G4RunManager run_manager;
	auto runManager = G4RunManagerFactory::CreateRunManager(G4RunManagerType::Default);

Add example offloading template #1573

Are you sure you want to change the base?

Add example offloading template #1573

Conversation

stognini commented Jan 13, 2025 • edited Loading

github-actions bot commented Jan 13, 2025 • edited Loading

Test summary

sethrj left a comment

Choose a reason for hiding this comment

whokion commented Jan 14, 2025 • edited Loading

whokion left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stognini Jan 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whokion commented Jan 14, 2025

Choose a reason for hiding this comment

sethrj left a comment

Choose a reason for hiding this comment

sethrj commented Jan 17, 2025

drbenmorgan commented Jan 17, 2025

sethrj commented Jan 17, 2025

drbenmorgan commented Jan 17, 2025

sethrj commented Jan 17, 2025 • edited Loading

stognini commented Jan 13, 2025 •

edited

Loading

github-actions bot commented Jan 13, 2025 •

edited

Loading

whokion commented Jan 14, 2025 •

edited

Loading

stognini Jan 14, 2025 •

edited

Loading

sethrj commented Jan 17, 2025 •

edited

Loading