Skip to content

Add in-process Fortran benchmark utility and implement all benchmarks#375

Merged
PEZ merged 14 commits intobddicken:mainfrom
PEZ:in-process-fortran
Jan 25, 2025
Merged

Add in-process Fortran benchmark utility and implement all benchmarks#375
PEZ merged 14 commits intobddicken:mainfrom
PEZ:in-process-fortran

Conversation

@PEZ
Copy link
Collaborator

@PEZ PEZ commented Jan 25, 2025

I have:

  • Read the project README, including the benchmark descriptions
  • Read the PR template instructions before I deleted them
  • Understood that if I have changed something that could impact performance of one or more contributions, I should provide results benchmark runs, using the run.sh script, from before and after the change.

Description of changes

This adds Fortran to the list of languages ported to the new in-process runner.

Benchmark results:

(Select the M4 Max run from the dropdown.)

The raw CSV:

benchmark,timestamp,commit_sha,is_checked,user,model,ram,os,arch,language,run_ms,mean_ms,std-dev-ms,min_ms,max_ms,runs
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Babashka,10000,4932.429097333333,19.558807582136737,4904.777583,4946.859625,3
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,C,10000,49.626525,1.197406,47.808000,53.878000,202
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure,10000,49.86259725870647,10.513245406417887,47.251917,197.52725,201
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure Native,10000,49.84420378606965,1.0570703469045586,48.031167,54.965,201
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Fortran,10000,49.843781,1.148286,48.014000,54.348000,201
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java,10000,49.104576,1.066798,47.233708,53.500709,204
loops,2025-01-25T22:34:12Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java Native,10000,50.052993,0.940394,48.424709,54.072500,200

benchmark,timestamp,commit_sha,is_checked,user,model,ram,os,arch,language,run_ms,mean_ms,std-dev-ms,min_ms,max_ms,runs
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Babashka,10000,10.8032,0.644375,9.66471,11.7805,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,C,10000,2.1302,0.313791,1.76792,2.88833,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure,10000,312.343,3.31949,304.985,319.047,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure Native,10000,4.98707,0.612648,4.14313,6.39454,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Fortran,10000,3.16003,0.382805,2.54942,3.84154,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java,10000,48.1888,2.65731,45.5346,57.1655,25
hello-world,2025-01-25T22:39:15Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java Native,10000,4.73181,0.600208,3.78654,6.04833,25

benchmark,timestamp,commit_sha,is_checked,user,model,ram,os,arch,language,run_ms,mean_ms,std-dev-ms,min_ms,max_ms,runs
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Babashka,10000,4208.603403,16.693095493461023,4185.345667,4223.739042,3
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,C,10000,25.253205,0.621768,24.369000,27.912000,396
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure,10000,47.03717979812207,1.3824813318915894,44.573166,50.967541,213
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure Native,10000,43.46573142857143,1.216267624512897,41.035125,48.560542,231
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Fortran,10000,.124543,.024082,.103000,1.581000,80294
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java,10000,42.044485,1.212892,39.179459,47.019875,238
fibonacci,2025-01-25T22:31:34Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java Native,10000,38.524551,1.014170,36.684375,42.212000,260

benchmark,timestamp,commit_sha,is_checked,user,model,ram,os,arch,language,run_ms,mean_ms,std-dev-ms,min_ms,max_ms,runs
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Babashka,10000,23214.701,0.0,23214.701,23214.701,1
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,C,10000,31.673044,0.602644,30.624000,34.882000,316
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure,10000,56.42315778651685,1.0903235486343315,55.100291,61.138917,178
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Clojure Native,10000,60.27484936746988,1.401040380520535,58.32725,68.93675,166
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Fortran,10000,33.700690,.888740,32.598000,42.921001,297
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java,10000,54.696480,1.600836,51.650167,59.846041,183
levenshtein,2025-01-25T22:36:38Z,fdbd28c,true,PEZ,Apple M4 Max,64GB,darwin24,arm64,Java Native,10000,61.097392,4.235434,52.652875,73.485792,164

@PEZ PEZ added levenshtein Anything regarding the levenshtein benchmark/test loops Anything regarding the loops benchmark/test fibonacci Anything regarding the fibonacci benchmark/test hello-world Anything regarding the hello-world benchmark/test in-process-runner When working with the tooling for the in-process runner labels Jan 25, 2025
@PEZ PEZ changed the title Add in-process Fortran benchmark utility and benchmark all implementations Add in-process Fortran benchmark utility and implement all benchmarks Jan 25, 2025
@PEZ PEZ requested a review from bddicken January 25, 2025 23:37
@PEZ PEZ merged commit ce56a44 into bddicken:main Jan 25, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fibonacci Anything regarding the fibonacci benchmark/test hello-world Anything regarding the hello-world benchmark/test in-process-runner When working with the tooling for the in-process runner levenshtein Anything regarding the levenshtein benchmark/test loops Anything regarding the loops benchmark/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fortran in-process runner

1 participant