scripts: west_commands: add the west patch command

[cfriedt]

In smaller projects and organizations, forking Zephyr is a tenable solution for development continuity, in the case that bug-fixes or enhancements need to be applied to Zephyr to unblock things.

However, in other organizations, in the absence of healthy patch management, technical debt management, and open-source policies, and active scrutiny on the subject of internal forks, things can get out of hand quickly.

In other cases, it may just be preferable to have a zero forking / upstream first policy.

Regardless of the reason, this change adds a west patch command that enables users to manage patches locally in their modules, under version control, with complete transparence.

The format of the YAML file includes fields for filename, checksum, author, email, dates, pr and issue links. There are fields indicating whether the patch is upstreamble or whether it has been merged upstream already. Workflows can be created to notify maintainers when a merged patch may be discarded after a version or a commit bump,

In Zephyr modules, the YAML file resides conventionally under zephyr/patches.yml, and patch files reside under zephyr/patches/. Since the command to apply patches is flexible, patches can even come in the form of scripts.

Sample usage applying patches (the -v argument for additional detail):

west -v patch apply
reading patch file zephyr/run-tests-with-rtt-console.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/twister-rtt-support.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/multiple_icntl.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/move-bss-to-end.patch
checking patch integrity... OK
patching zephyr... OK
4 patches applied successfully \o/

Clean previously applied patches, leaving each module in a pristine state.

west patch clean

Behaviour after manually corrupting a patch file and passing the -r option is that all patches are automatically rolled-back.

west -v patch apply -r
reading patch file zephyr/run-tests-with-rtt-console.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/twister-rtt-support.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/multiple_icntl.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/move-bss-to-end.patch
checking patch integrity... FAIL
ERROR: sha256 mismatch for zephyr/move-bss-to-end.patch:
expect: 00e42e5d89f68f8b07e355821cfcf492faa2f96b506bbe87a9b35a823fd719cb
actual: b9900e0c9472a0aaae975370b478bb26945c068497fa63ff409b21d677e5b89f
Cleaning zephyr
FATAL ERROR: failed to apply patch zephyr/move-bss-to-end.patch

Fixes #71137

[cfriedt]

need to write a couple of tests at some point

[cfriedt]

• improved patch-schema.yml with sub-schema and field descriptions

[kartben]

fixes #71137, I guess

[kartben]

I have been using Golioth's west patch command quite successfully in the past
https://github.com/golioth/golioth-zephyr-sdk/blob/main/scripts/west_commands/patch.py

AFAIK it has a few downstream users already (, so might be worthwhile having a closer look / adopting if their approach makes sense. I quite like this doesn't require any custom yaml and "regular" patches can conveniently be dropped in a folder, but admittedly I haven't really looked in details at the implementation proposed in this PR :) Just thought I would mention of the existence of this alternative implementation that I know does the job :)
Note: Golioth SDK is Apache licensed.
cc @mniestroj @beriberikix @ubieda

[cfriedt]

I have been using Golioth's west patch command quite successfully in the past golioth/golioth-zephyr-sdk@main/scripts/west_commands/patch.py

Oh wow - I didn't even realize that existed.

AFAIK it has a few downstream users already (, so might be worthwhile having a closer look / adopting if their approach makes sense. I quite like this doesn't require any custom yaml and "regular" patches can conveniently be dropped in a folder, but admittedly I haven't really looked in details at the implementation proposed in this PR :) Just thought I would mention of the existence of this alternative implementation that I know does the job :) Note: Golioth SDK is Apache licensed. cc @mniestroj @beriberikix @ubieda

I'll take a look. Maybe we can have the best of both worlds?

[kartben]

I'll take a look. Maybe we can have the best of both worlds?

Deal! :)

[cfriedt]

I'll push again momentarily, just running some tests locally.

[ubieda]

We do use this downstream and it's pretty useful. Thanks for bringing this to Zephyr!

[cfriedt]

• updated to only execute checkout-command or clean-command if they are true-ish

[cfriedt]

• corrected comment for top-level schema in patch-schema.yml

[cfriedt]

• added missing break

[cfriedt]

I still need to add some tests 😅 , lol

Might not get to that today because I have a few loose ends to tie up at the office still.

[kartben]

cc @mbolivar (can't add you as reviewer yet until you're a collaborator)

[ycsin]

Definitely helpful for orgs with access to west and have multiple teams sharing the same fork of Zephyr but need to apply different patches.

Glad to see that Golioth has something similar, so the use case is definitely there.

[cfriedt]

• switched merge-commitback to only matching 40-digit SHA-1 hashes in patch-schema.yml