Running on macOS aarch64 - Lima with Ubuntu 24 - failed with "Illegal instruction"

[georgik]

Idea: Run Hyperion on macOS aarch64 using Lima.

Build:

cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

Failed with: Illegal instruction

Diagnostics:

gdb --args target/release/tag --ip 0.0.0.0 --port 35565
(gdb) run
Starting program: .../hyperion/target/release/tag --ip 0.0.0.0 --port 35565
Unable to fetch SVE/SSVE vector length: Invalid argument.

According cat /proc/info only sve2 is available.

Workaround: build with following options

RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

Result: After using target-feature modifier and portForward in lima, it's possible to run tag, hyperion-proxy and connect Minecraft Java client.

Working commands:

RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin hyperion-proxy --  0.0.0.0:8080
RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

[Tebarem]

Hey. Whats Lima?

[Kumpelinus]

Hey. Whats Lima?

I guess this one: https://github.com/lima-vm/lima

[georgik]

@Kumpelinus that is correct link. Unfortunatelly Hyperion is not possible to build directly on macOS aarch64, due to some header issues in flecs_ecs_sys crate, so the Lima is the next option.
Failure looks like this:

error: failed to run custom build command for `flecs_ecs_sys v0.1.2 (https://github.com/Indra-db/Flecs-Rust#9a020d4c)`
...
.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_types/_rsize_t.h:50:9: error: unknown type name '__darwin_size_t'\nfatal error: too many errors emitted, stopping now [-ferror-limit=]\n")

[Tebarem]

Can we get the full error?

[TestingPlant]

In gdb, after running hyperion and it gives the error, could you run the following commands and post the output from both of them?

backtrace
x/50i $pc-50

[andrewgazelka]

@Kumpelinus that is correct link. Unfortunatelly Hyperion is not possible to build directly on macOS aarch64, due to some header issues in flecs_ecs_sys crate, so the Lima is the next option. Failure looks like this:

error: failed to run custom build command for `flecs_ecs_sys v0.1.2 (https://github.com/Indra-db/Flecs-Rust#9a020d4c)`
...
.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_types/_rsize_t.h:50:9: error: unknown type name '__darwin_size_t'\nfatal error: too many errors emitted, stopping now [-ferror-limit=]\n")

Hyperion should be able to be built on macOS aarch64... if it isn’t this is a regression in flecs_ecs_sys and I will downgrade to an earlier version for the time being.

I am looking into it right now.

[andrewgazelka]

Idea: Run Hyperion on macOS aarch64 using Lima.

Build:

cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

Failed with: Illegal instruction

Diagnostics:

gdb --args target/release/tag --ip 0.0.0.0 --port 35565
(gdb) run
Starting program: .../hyperion/target/release/tag --ip 0.0.0.0 --port 35565
Unable to fetch SVE/SSVE vector length: Invalid argument.

According cat /proc/info only sve2 is available.

Workaround: build with following options

RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

Result: After using target-feature modifier and portForward in lima, it's possible to run tag, hyperion-proxy and connect Minecraft Java client.

Working commands:

RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin hyperion-proxy --  0.0.0.0:8080
RUSTFLAGS="-C target-feature=-sve" cargo run --release --bin tag -- --ip 0.0.0.0 --port 35565

I am able to do a fresh build

cargo build --release

bounty-bot on  cato/landingpage2 is 󰏗 v1.0.0
❯ uname -a
Darwin hydra.local 24.1.0 Darwin Kernel Version 24.1.0: Thu Oct 10 21:03:11 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T6020 arm64

bounty-bot on  cato/landingpage2 is 󰏗 v1.0.0
❯ arch
arm64

[andrewgazelka]

@georgik can I see the output of uname -a? Which hardware are you running? I can’t reproduce on my M2 Max MacBook Pro.

also @Indra-db (flecs-ecs maintainer) do you have any thoughts?

[andrewgazelka]

adding #801 which will hopefully allow for more reproducible builds

[georgik]

Thank you @andrewgazelka for taking a look at the problem.
I've tried the build with the latest changes, but the problem on M4 still persist.
I've created a new issue to track M4 build: #802
Looks like a regression in flecs_ecs_sys

[georgik]

Let me add information to Lima - Ubuntu 24:

uname -a
Linux lima-default 6.11.0-13-generic #14-Ubuntu SMP PREEMPT_DYNAMIC Sun Dec  1 00:22:04 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

[georgik]

I've tried the complete rebuild and the problem still persist
Illegal instruction (core dumped)

So the only workaround for Lima - Ubuntu 24 is to set the environment variable to disable SVE and keep only SVE2.
RUSTFLAGS="-C target-feature=-sve"