intel_debug: introduction: Add information about modular SOF release …

[ujfalusi]

…content

The firmware supports library loading starting with Meteor Lake and the release system is prepared to offer modular SOF releases starting with Panther Lake (but can be supported from MTL onward).

Update the documentation of the SOF release content to reflect this.

[ujfalusi]

Changes since v1:

• drop extra lines from between the library type descriptions

[ujfalusi]

Changes since v2:

• Drop Panther Lake from commit message

[kv2019i]

Excellent, thanks @ujfalusi !

[jsarha]

Looks good to me, just wondering how its decided, what symbols should be loaded from *-openmodules.ri and *-debug.ri. So far I only understand (sort of) how the uuid based symbol resolution works.

[jsarha]

So these are just libraries with symbols in them, and the fw core will load the symbols as needed? No UUID based loading, symlink or otherwise, required?

[ujfalusi]

In case of modular SOF release the basefw and modules are in different libraries (note that the basefw itself is a library file).
The full SOF is basefw+openmodules+debug, this is what we have now as monolithic single basefw, it includes all modules.
Iow, to get the full SOF you need to boot the basefw and then load the modules from the external libraries.

The UUID based loading is still supported, but the full SOF is defined in case of modular release as basefw+openmodules+debug.

[jsarha]

Ok. And all those three base-release libraries are loaded to SRAM at boot time, like the the older monolithic FW binary, no?

[ujfalusi]

The basefw loaded (to SRAM) and booted, then we load the openmodules followed by the debug library, these will be loaded to DRAM. The modules that are needed for the audio stream will be loaded (by the firmware) from DRAM to SRAM for execution.
The monolithic firmware with all the modules are stored in SRAM.

The basefw is still special, it is the one that can boot and have all the needed infra (like the kernel in Linux). Instead of individual modules we can have module bundles.

At the end these form the SOF package either way: monolithic or modular.

[jsarha]

I finally read the long mail thread about the subject from just before my vacation, and it answered some of my questions. But still, the modules in the open-modules-bundle are loaded as needed based on uuid's even if they are not needed for getting access to the library file itself, don't they?

[ujfalusi]

No, sof-bin release is not planned to have UUID symlinks to the openmodules.ri, it is going to be loaded unconditionally if it exists in filesystem.
We can only load library into firmware, the library can contain one or more modules (bundle). One can freely move modules between basefw and openmodules between releases, the whole package will contain all the needed modules.

I want to define this as simple as possible:
If sof-bin is monolithic for a platform then only the basefw will be on filesystem.
If sof-bin is modular for a platform then basefw+openmodules(+debug) will be on the filesystem.

The kernel shall load these unconditionally, which means that the kernel will treat the base libraries as optional, if they don't exist it is not an error since it is the monolithic release.

The kernel change will come after the agreement on the documentation and how the sof-bin is going to be released.

The kernel will still going to look based on UUID for missing modules but for a complete sof-bin release it is not really going to happen (apart from LNL which have one module outside as library).

[lyakh]

@jsarha let me try to add a bit more to what @ujfalusi already explains quite nicely.

• those additional libraries (in our case openmodules and debug) are really "bundles" as Peter calls them of individual modules. You can also think of them as of .ar or .tar or .cpio archives of ELF objects. They really pack multiple modules together and add headers like a signature and a set of manifests to the archive. That allows the firmware later load individual modules from those library archives into SRAM
• libraries are first loaded to DRAM. Because of the manifest header the kernel knows what is contained in each library. Now, I have to make one more distinction, and the terminology is going to be a bit shady, sorry. Libraries contain modules, and modules can contain multiple "drivers," each of which has its own UUID. That way the "volume" module implements PEAKVOL and GAIN drivers, the "src" module implements SRC and SRC_LITE, "mixin_mixout" implements MIXIN and MIXOUT,... So, when a topology needs PEAKVOL, it requests its UUID, but also gets GAIN "free of charge" because it's the same module, the same ELF object.
• So, the following happens when the kernel loads openmodules (Peter, correct me if I'm wrong somewhere):

1. the kernel reads in and stores all the UUIDs, and sends the whole library to the firmware
2. the library is sent to the firmware via DMA
3. the library is stored in DRAM
4. then the kernel reads in a topology and checks if it has all the UUIDs
5. now normally all UUIDs should already be available, if not, the kernel can additionally load individual modules following UUID names
6. now the loaded topology has all the modules available - in DRAM
7. then the user starts some audio activity
8. the kernel creates a pipeline and instantiates each "component" in it (using library and module IDs, yes, yet another identification mechanism, which we won't explain here in detail...)
9. the firmware then checks if the driver for that component is already in SRAM (e.g. because another pipeline already uses it), if not, it is loaded from DRAM. If this is the first time for this module, the ELF linking is performed
10. when the last user of a module is freed, the module is removed from SRAM

Peter, feel free to add any of this to the document, if you find it useful

[ujfalusi]

@lyakh, really nice writeup, but I don't think it belongs to this page, probably to the LLEXT or some other dedicated page. This would be nice to preserve as reference!

[jsarha]

@lyakh Thanks a lot for the bigger picture!