Python 3.13 RC1 - a Quick CPU Benchmark

Maxim Saplin - Aug 25 - - Dev Community

Python 3.13 is due to be released in October, yet the first release candidate was published earlier in August. Last year, I did a quick CPU bound benchmark of version 3.12 using Mandelbrot set calculation.

With all files in place, I downloaded version 3.13RC from the official web site and tested 3 versions of Python on my M1 Mac Book Pro:

user@users-MacBook-Pro py_chat_ui % python --version
Python 3.11.9
user@users-MacBook-Pro py_chat_ui % python3 --version
Python 3.12.5
user@users-MacBook-Pro py_chat_ui % python3.13 --version
Python 3.13.0rc1
Enter fullscreen mode Exit fullscreen mode

Here're the execution time (in seconds) for the implementation relying on NumPy (versions 3.11, 3.12 and 3.13RC1 correspondingly):

user@users-MacBook-Pro mandelbrot % python mandelbrot.py   
1 Execution Time: 6.305059909820557                   78513419
2 Execution Time: 6.282307863235474                   78513419
3 Execution Time: 6.473501920700073                   78513419
user@users-MacBook-Pro mandelbrot % python3 mandelbrot.py
1 Execution Time: 5.418003082275391                   78513419
2 Execution Time: 5.412122011184692                   78513419
3 Execution Time: 5.434246778488159                   78513419
user@users-MacBook-Pro mandelbrot % python3.13 mandelbrot.py
1 Execution Time: 7.197513818740845                   78513419
2 Execution Time: 7.212265968322754                   78513419
3 Execution Time: 7.200297832489014                   78513419
Enter fullscreen mode Exit fullscreen mode

And the results for the pure Python implementation:

user@users-MacBook-Pro mandelbrot % python mandelbrot_pure.py
1 Execution Time: 41.18416976928711                   78513425
2 Execution Time: 41.16466403007507                   78513425
3 Execution Time: 41.148504972457886                   78513425
4 Execution Time: 41.55486297607422                   78513425
user@users-MacBook-Pro mandelbrot % python3 mandelbrot_pure.py
1 Execution Time: 49.806406021118164                   78513425
2 Execution Time: 49.485753774642944                   78513425
3 Execution Time: 49.52305006980896                   78513425
4 Execution Time: 49.57118225097656                   78513425
user@users-MacBook-Pro mandelbrot % python3.13 mandelbrot_pure.py
1 Execution Time: 41.07340693473816                   78513425
2 Execution Time: 41.08624267578125                   78513425
3 Execution Time: 41.09266400337219                   78513425
4 Execution Time: 41.1431610584259                   78513425
Enter fullscreen mode Exit fullscreen mode

Frankly speaking, I am confused with the inconsistency in the results - newer doesn't mean better :)

P.S. >

For the reference, running Mandelbrot calculation with JiT compiled Dart program and AoT compiled C version:

user@users-MacBook-Pro mandelbrot % dart mandelbrot.dart
1  Execution Time: 0.476                       78513425
2  Execution Time: 0.484                       78513425
3  Execution Time: 0.475                       78513425
user@users-MacBook-Pro mandelbrot % gcc -o mandelbrot mandelbrot.c -Ofast
user@users-MacBook-Pro mandelbrot % ./mandelbrot 
1 Execution Time: 0.256706           79394433
2 Execution Time: 0.234396           79394433
3 Execution Time: 0.234862           79394433
Enter fullscreen mode Exit fullscreen mode

P.P.S. > Here's the GitHub repo with the same benchmark implemented in different languages.

P.P.P.S. > Here's last year's post comparing Python, Numba and Mojo.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .