Skip to content

Commit

Permalink
Improve bench/README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
lyrm committed Dec 2, 2024
1 parent f370411 commit 9bbb999
Show file tree
Hide file tree
Showing 2 changed files with 67 additions and 7 deletions.
70 changes: 65 additions & 5 deletions bench/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,67 @@
Benchmarks for Saturn
# Benchmarks for Saturn

# General usage
Benchmarks are written using [multicore-bench](https://github.com/ocaml-multicore/multicore-bench).

Execute `make bench` from root of the repository to run the standard set of
benchmarks. The output is in JSON, as it is intended to be consumed by
[current-bench](https://bench.ci.dev/ocaml-multicore/saturn/branch/main/benchmark/default).
## General Usage

To execute benchmarks, you can run:
```shell
make bench
```

Alternatively, you can use:
```shell
dune exec -- ./bench/main.exe
```

It is recommended to run the benchmarks with a budget of at least `1` second (as done with `make bench`):
```shell
dune exec -- ./bench/main.exe -budget 1
```

You can also print a brief version of the benchmarks with the `-brief` option. Additionally, it is possible to run only selected benchmarks by providing a part of the benchmark names. You can get the list of available benchmarks with the `--help` option.

For example, running:
```shell
dune exec -- ./bench/main.exe --help
```
returns:

```
Usage: main.exe <option>* filter*
The filters are regular expressions for selecting benchmarks to run.
Benchmarks:
Saturn Queue
Saturn Queue_unsafe
Saturn Bounded_Queue
Saturn Bounded_Queue_unsafe
Saturn Single_prod_single_cons_queue
Saturn Size
Saturn Skiplist
Saturn Htbl
Saturn Htbl_unsafe
Saturn Stack
Saturn Work_stealing_deque
Saturn Bounded_Stack
Options:
-budget seconds Budget for a benchmark
-debug Print progress information to help debugging
-diff path.json Show diff against specified base results
-brief Show brief human-readable results
-help Show this help message
--help Show this help message
```

For example, if you want to run only the `Htbl` benchmarks to compare the performance of `Htbl` and its unsafe version `Htbl_unsafe`, you can run:
```shell
dune exec -- ./bench/main.exe -budget 1 -brief Htbl
```

## Current-bench

The output is in JSON format, as it is intended to be consumed by [current-bench](https://bench.ci.dev/ocaml-multicore/saturn/branch/main/benchmark/default).
4 changes: 2 additions & 2 deletions bench/main.ml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
let benchmarks =
[
("Saturn Queue", Bench_queue.Safe.run_suite);
("Saturn Queue_unsafe", Bench_queue.Unsafe.run_suite);
("Saturn Queue (MS)", Bench_queue.Safe.run_suite);
("Saturn Queue_unsafe (MS)", Bench_queue.Unsafe.run_suite);
("Saturn Bounded_Queue", Bench_bounded_queue.Safe.run_suite);
("Saturn Bounded_Queue_unsafe", Bench_bounded_queue.Unsafe.run_suite);
("Saturn Single_prod_single_cons_queue", Bench_spsc_queue.run_suite);
Expand Down

0 comments on commit 9bbb999

Please sign in to comment.