Python is incredibly popular because it’s easy to learn, versatile, and has thousands of useful libraries for data science. But one thing it isn’t is fast.

That’s about to change in Python 3.11, currently in its early beta preview (version 3.11.0b1) ahead of its stable release later this year. Core Python (CPython) developer Mark Shannon shared details about the project to make Python faster at the PyCon 2022 conference this weekwhere the developers also showed progress on the goal of running Python code in the browser.

Last year, Microsoft funded a project for the Python Software Foundation (PSF), led by Python creator Guido van Rossum and Shannon, to make Python twice as fast as the current stable 3.10 series. The vision is to push Python towards the performance of C.

SEE: How to get promoted: Five ways to climb the ladder and succeed in your career

Microsoft hired van Rossum in 2020 and gave him carte blanche to choose any project. At last year’s PyCon 2021 conference, he said he “chose to go back to my roots” and would work on Python’s famous lack of performance.

Performance may not have been a top priority for Python, as adoption has been fueled by machine learning and data science through Tensor Flow, Numpy, Pandas, and many other platforms, such as AWS Boto3 SDK for Python. These platforms are downloaded tens of millions of times per month and used in often non-hardware constrained environments.

The Faster CPython Project provided updates on the performance of CPython 3.11 over the past year. Ahead of PyCon 2022, the project released more results comparing preview beta 3.11 to 3.10 across dozens of performance metrics, showing 3.11 to be 1.25 times faster overall than 3.10.

Shannon is realistic about the project’s ability to improve Python’s performance, but thinks the improvements can extend Python’s viable use to more virtual machines.

“Python is widely known to be slow. While Python will never match the performance of low-level languages ​​like C, Fortran, or even Java, we’d like it to be competitive with fast scripting language implementations, like V8 for JavaScript. or luajit for lua,” he wrote last year in Python Enhancement Proposal (PEP) 659.

“Specifically, we want to achieve these performance goals with CPython to benefit all Python users, including those who cannot use PyPy or other alternative virtual machines.”

The key approach detailed in PEP 659 is a “specialized, adaptive interpreter that specializes code aggressively, but over a very small region, and is able to quickly and cheaply adapt to poor specialization”.

As noted, optimizations for virtual machines are “expensive”, often requiring a long “warm-up” time. To avoid this expenditure of time, the virtual machine must “assume that specialization is justified even after a few executions of a function”, so the interpreter must optimize and deoptimize continuously and very cheaply.

This should result in a faster interpreter for CPython that tracks individual bytecodes as a program runs. Work on the new interpreter is nearly complete but still requires completion of dynamic specializations for loops and binary operations, according to PSF.

Also, memory consumption in version 3.11 has not changed since version 3.10.

SEE: Developers face burnout. Here’s how companies are trying to fix it

On the question of a just-in-time (JIT) compiler for Python performance, Shannon suggested that it’s not a priority and likely won’t arrive until Python 3.13, according to the coverage of the article. event by the Python Software Foundation.

Anaconda, the creator of the Anaconda Python distribution for data science, supports the Pyston project, an implementation of Python that promises speed improvements over Python.

One of Anaconda’s oldest efforts to speed up Python was the Numba project, an LLVM-based JIT compiler for CPython, which speeds up numerical Python functions running on the CPU or GPU, but cannot optimize entire programs. and does not address the broader use of Python. case. Another is Pythonan implementation of CPython with a JIT compiler for faster performance.

According to Faster Python implementation planCPython 3.12 might gain a “simple JIT compiler for small regions” that compiles small regions of specialized code, while 3.13 would improve the compiler to extend regions for compilation.