Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support executing Pulley in Wasmtime #9744

Merged
merged 12 commits into from
Dec 6, 2024

Conversation

alexcrichton
Copy link
Member

This commit is the initial implementation of executing the Pulley
interpreter from the wasmtime crate. This gives access to all of the
wasmtime crate's runtime APIs backed by execution of bytecode in
Pulley. This builds on the previous PRs I've been making for support in
Pulley to culminate in testing on CI in this PR. This PR handles some
final tidbits related to producing a runnable image that can be
interpreted by the wasmtime crate such as:

  • Pulley compilation artifacts are no longer marked as natively
    executable, just read-only.
  • Pulley compilation artifacts include wasm-to-array trampolines like
    normal platforms (removes a pulley special-case).
  • Dispatch of host calls from Pulley to the Wasmtime runtime are
    implemented.
  • Pulley's list of panicking wasm features is slimmed down as most are
    covered by "this lowering isn't supported" errors.
  • Execution of wasm code now has an if to see whether Pulley is
    enabled within a Store or not.
  • Traps and basic "unwinding" of the pulley stack are implemented (e.g.
    a "pulley version" of setjmp and longjmp, sort of)
  • Halting the interpreter has been refactored to help shrink the size of
    ControlFlow<Done> and handle metadata with each done state.

Some minor refactorings are also included here and there along with a
few fixes here and there necessary to get tests passing.

The next major part of this commit is updates to our wast test suite
and executing all *.wast files. Pulley is now executed by default for
all files as a new execution engine. This means that all platforms in CI
will start executing Pulley tests. At this time almost all tests are
flagged as "expected to fail" but there are a small handful of
allow-listed tests which are expected to pass. This exact list will
change over time as CLIF lowerings are implemented and the interpreter
is extended.

Follow-up PRs will extend the testing strategy further such as:

  • Extending #[wasmtime_test] to include Pulley in addition to Winch.
  • Getting testing set up on CI for 32-bit platforms.

@alexcrichton alexcrichton requested a review from fitzgen December 5, 2024 19:54
@alexcrichton
Copy link
Member Author

I'll note that this is in draft form as it currently is based on #9743. I'm going ahead and opening this up to see what CI has to say about this.

@github-actions github-actions bot added cranelift Issues related to the Cranelift code generator wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime labels Dec 5, 2024
Copy link

github-actions bot commented Dec 5, 2024

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

  • If you added a new Config method, you wrote extensive documentation for
    it.

    Our documentation should be of the following form:

    Short, simple summary sentence.
    
    More details. These details can be multiple paragraphs. There should be
    information about not just the method, but its parameters and results as
    well.
    
    Is this method fallible? If so, when can it return an error?
    
    Can this method panic? If so, when does it panic?
    
    # Example
    
    Optional example here.
    
  • If you added a new Config method, or modified an existing one, you
    ensured that this configuration is exercised by the fuzz targets.

    For example, if you expose a new strategy for allocating the next instance
    slot inside the pooling allocator, you should ensure that at least one of our
    fuzz targets exercises that new strategy.

    Often, all that is required of you is to ensure that there is a knob for this
    configuration option in wasmtime_fuzzing::Config (or one
    of its nested structs).

    Rarely, this may require authoring a new fuzz target to specifically test this
    configuration. See our docs on fuzzing for more details.

  • If you are enabling a configuration option by default, make sure that it
    has been fuzzed for at least two weeks before turning it on by default.


To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

This commit is the initial implementation of executing the Pulley
interpreter from the `wasmtime` crate. This gives access to all of the
`wasmtime` crate's runtime APIs backed by execution of bytecode in
Pulley. This builds on the previous PRs I've been making for support in
Pulley to culminate in testing on CI in this PR. This PR handles some
final tidbits related to producing a runnable image that can be
interpreted by the `wasmtime` crate such as:

* Pulley compilation artifacts are no longer marked as natively
  executable, just read-only.
* Pulley compilation artifacts include wasm-to-array trampolines like
  normal platforms (removes a pulley special-case).
* Dispatch of host calls from Pulley to the Wasmtime runtime are
  implemented.
* Pulley's list of panicking wasm features is slimmed down as most are
  covered by "this lowering isn't supported" errors.
* Execution of wasm code now has an `if` to see whether Pulley is
  enabled within a `Store` or not.
* Traps and basic "unwinding" of the pulley stack are implemented (e.g.
  a "pulley version" of `setjmp` and `longjmp`, sort of)
* Halting the interpreter has been refactored to help shrink the size of
  `ControlFlow<Done>` and handle metadata with each done state.

Some minor refactorings are also included here and there along with a
few fixes here and there necessary to get tests passing.

The next major part of this commit is updates to our `wast` test suite
and executing all `*.wast` files. Pulley is now executed by default for
all files as a new execution engine. This means that all platforms in CI
will start executing Pulley tests. At this time almost all tests are
flagged as "expected to fail" but there are a small handful of
allow-listed tests which are expected to pass. This exact list will
change over time as CLIF lowerings are implemented and the interpreter
is extended.

Follow-up PRs will extend the testing strategy further such as:

* Extending `#[wasmtime_test]` to include Pulley in addition to Winch.
* Getting testing set up on CI for 32-bit platforms.

prtest:full
Enable pulley for wasmtime/wasmtime-cli and also enable all features for
wasmtime-environ
@alexcrichton alexcrichton marked this pull request as ready for review December 5, 2024 22:24
@alexcrichton alexcrichton requested review from a team as code owners December 5, 2024 22:24
@alexcrichton
Copy link
Member Author

@fitzgen ok should be rebased and updated, I believe s390x CI is failing but I think most of the rest should be passing. Once results come in I'll figure out how to best ignore s390x for now.

Copy link
Member

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Super excited to see this progress!

A few nitpicks and suggestions below, but nothing major

(rule 1 (lower (trapz (icmp cc a b @ (value_type (fits_in_64 ty))) code))
(rule 1 (lower (trapz (icmp cc a b @ (value_type (ty_32_or_64 ty))) code))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious: why was this change needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is due to some other rule only accepting 32 or 64-bit inputs which causes ISLE to panic as something not a partial constructor. Basically tests panic unless this is here, so a future refactoring will need to flesh out more lowering rules for 8/16-bit types.

crates/cranelift/src/obj.rs Outdated Show resolved Hide resolved
crates/wasmtime/src/runtime/store.rs Show resolved Hide resolved
crates/wasmtime/src/runtime/vm.rs Show resolved Hide resolved
crates/wasmtime/src/runtime/vm/interpreter.rs Outdated Show resolved Hide resolved
crates/wast-util/src/lib.rs Outdated Show resolved Hide resolved
pulley/src/interp.rs Outdated Show resolved Hide resolved
pulley/src/interp.rs Outdated Show resolved Hide resolved
pulley/src/interp.rs Outdated Show resolved Hide resolved
pulley/src/interp.rs Outdated Show resolved Hide resolved
@alexcrichton alexcrichton requested a review from a team as a code owner December 6, 2024 20:06
@alexcrichton alexcrichton added this pull request to the merge queue Dec 6, 2024
Merged via the queue into bytecodealliance:main with commit 70a3793 Dec 6, 2024
130 checks passed
@alexcrichton alexcrichton deleted the pulley-in-wasmtime branch December 6, 2024 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift Issues related to the Cranelift code generator wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants