Conversation
- code.py is native python - cythonized.py is compiled to c using cython, a python compiler
| from functools import cache | ||
|
|
||
|
|
||
| @cache |
There was a problem hiding this comment.
Using a cache is "cheating" in this case. It allows the program to skip huge amounts of the work relative to other implementations. I mean, it's cool that python supports this, but a similar cache could be implemented in other languages giving them huge boosts also.
There was a problem hiding this comment.
Hm, in this case all JIT and compiler optimizations should not be allowed either, because they are essentially "cheating" too, right? Otherwise the benchmark is comparing compilers, not languages.
|
|
||
| def parallel(fn, iterable): | ||
| # helper for parallel processing as a transparent fallback in case memoization is not effective | ||
| from multiprocessing import Pool |
There was a problem hiding this comment.
multi threading and multi processing are not allowed for this particular benchmark. Though, if you have an idea for a multi-process-specific benchmark, feel free to make a PR to add a new one!
There was a problem hiding this comment.
I thougt the objective is to benchmark 1 billion operations? I get that you want some ground rules, but why does it matter how that is achieved?
| for j in range(100000): # 100k inner loop iterations, per outer loop iteration | ||
| a[i] += j % u # Simple sum | ||
| a[i] += r # Add a random value to each element in array | ||
| print(a[r]) # Print out a single element from the arra |
There was a problem hiding this comment.
I think I already merged a PR for cython?
There was a problem hiding this comment.
There is PR #222, however that is treating Cython as a separate language (because it is using Cython-specific syntax). This PR however uses Cython's pure python mode, i.e. same syntax as original code.
There was a problem hiding this comment.
I think I already merged a PR for cython?
Actually, PR #222 hasn't been merged yet :)
|
Closing this for now. As stated above already, we're not allowing memoization, caching, parallellism, or anything like that in the current benchmarks. I'd be interested in a cython entry. There's a new runner and any contributions need to target that. |
4 participants
I'm submitting this for a Python implementation that's both on par with C, using the original code in terms of code structure and logic (in particular, a nested for-loop), and is using only the Python standard library.
This is particularly addressing the concern raised in #213, #217 where you mention that the amount of (computation) work should be the same regardless of the language, and hence pre-computing
u % jis not valid. Similarly, this is not using an external library (numpy), as you argued in #63 that only built-in / standard library features are allowed.Approach
code.pyis native python code, using the standard library; leveraging compute-parallelism and memoizationcythonized.pyis compiled to c using Cython, a python compilerResults
code.pyruns at approx. 10x C speed (with memoization) to at 1/10x C speed (compute-parallel, without memoization). That's a 4x to 40x speed-up v.v. the original code.cythonized.pyruns at roughly C speed, that's a 30x speed-up v.v. the original code.Rationale
One could argue that memoization reduces the amount of work done; however, this is up to the program itself, i.e. the work specified by the program is the same regardless (unlike code-specified optimization, such as pre-computation or explicit caching). In some sense, this is semantically equivalent to other languages' compiler optimizations and use of just in time compilation (JIT).
The use of Cython to compile the code to native code is the same approach used by other languages. The program used for this (
cythonized.py) is otherwise identical with the original code.