Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks #12

Open
8 tasks
amirouche opened this issue Nov 27, 2021 · 1 comment
Open
8 tasks

Add benchmarks #12

amirouche opened this issue Nov 27, 2021 · 1 comment
Labels
idea The issue is still in ideation phase

Comments

@amirouche
Copy link

amirouche commented Nov 27, 2021

Have a way to run benchmarks:

  • use linux perf-tools to gather statistics
  • benchmark compilation separatly
  • aggregation for:
    • memory use
    • wall-clock time
  • benchmark safe and unsafe
  • score board
  • add a github actions that can only be run via the interface (ie. no triggers)

ref: https://github.com/benchmark-action/github-action-benchmark
ref: ecraven/r7rs-benchmarks#60
ref: ecraven/r7rs-coverage#4
ref: https://github.com/larcenists/larceny/tree/master/test/Benchmarking/R6RS/src
ref: https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html
ref: https://github.com/gothinkster/realworld
ref: https://www.techempower.com/benchmarks/

@amirouche amirouche modified the milestone: 2021.0 Nov 27, 2021
@amirouche
Copy link
Author

re github-action-benchmark is not what is needed. We would need something that can compare several implementations, and then aggregate the results of various metrics, and eventually get together a scoreboard.

@amirouche amirouche added the idea The issue is still in ideation phase label Dec 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
idea The issue is still in ideation phase
Projects
None yet
Development

No branches or pull requests

1 participant