Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Invocable and prevent concurrent access into orc #75

Closed
wants to merge 3 commits into from

Conversation

PhilippGrulich
Copy link
Member

@PhilippGrulich PhilippGrulich commented Dec 29, 2024

Whenever invoked, the CallableFunction wrapper locates the executable within the LLVM context.
This makes calling a function not thread-safe as the underlying LLVM implementation is not thread-safe and introduces an unnecessary overhead.

For small functions, this lookup dominates the runtime. Note: This benchmark only measures the invocation. Compilation is done beforehand.

static nautilus::val<int> smallFunction(nautilus::val<int> i) {
	return i * 2;
}
----------------------------------------------------------------------
Benchmark                            Time             CPU   Iterations
----------------------------------------------------------------------
BM_smallFunctionCompiler          1187 ns         1183 ns       565135
BM_smallFunctionInterpreter       62.7 ns         62.5 ns     11918221

This PR caches the Invocable produced by the Compiler during construction of the CallableFunction and thus skips the lookup within the invocation.

----------------------------------------------------------------------
Benchmark                            Time             CPU   Iterations
----------------------------------------------------------------------
BM_smallFunctionCompiler          3.65 ns         3.65 ns    191092600
BM_smallFunctionInterpreter       63.1 ns         63.1 ns     11229368

Note: we can remove the release mode and benchmark commit and only pick the relevant commit

I am not sure why, but I cannot compile nautilus in
release mode locally.
Apparently the internals of the orc jit are not thread-safe:
https://lists.llvm.org/pipermail/llvm-dev/2017-November/119108.html

The thread sanitizer in nebulastream complaints about a data-race deep within orc. The mailing lists suggests using a lock around access into orc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants