Majority of new Python work is Python 3

(This post was originally from June 2017).

There is considerable additional effort required to support Python < 3.6 in general while using concurrent and other performant Python programming with the latest common modules like Numpy, Xarray, Matplotlib, etc.

Python 3 is used by a large and growing majority of new and active Python developers in science, engineering, medical research and education. Python 3 was released in December 2008. While initially there was pushback over backward incompatibilities, the performance, efficiencies and features of Python 3 have won out.

The most popular Python packages have supported Python 3 for some time now, including Amazon AWS and Google Cloud Platform.

The main holdouts are of the same nature as those that hang on to COBOL systems. Those with static, unchanging requirements in large proprietary codebases that few people are familiar with. Some programmers thrive and make a decent living servicing those legacy COBOL and Python environments. The majority of STEM coders, analysts and educators have been writing Python 3 code. The Python 3 objections were mostly written before 2016 and almost all were before 2017. Some of their complaints were addressed in Python 3.6 (released December 23, 2016).

A main issue over Python 3 is over the separation between bytes and strings. Applications with IoT and embedded systems distinguish between bytes and strings, so I appreciate the separation of bytes and strings in Python 3. For the global environment I write for, I appreciate that strings are Unicode in Python 3.

Python 3 efficiencies in terms of programmer time come in more efficient syntax. The Python 3 core itself is as much as 10% faster in several aspects, with some standard modules like re processing regular expressions as much as 20x faster. The modernize tool and six and __future__ modules smooth over most of these changes to make backward compatible code. Some of the most enticing changes from Python ≥ 3.6 are not easily backportable. These features simplify syntax and make the code more self-documenting.

asyncio brings core Python features that used to require Tornado, twisted, etc. Asynchronous execution is required for applications that need to scale massively. IoT applications where remote sensors report in are a perfect use case for asyncio. asyncio is a convenient, relatively safe way to thread baked right into Python.

Least-recently used caching is enabled by a decorator to functions. For example, you have a computationally-expensive function where you sometimes use the same arguments, but don’t want to bother tracking the output by saving in a variable. LRU caching is as simple as:

import functools

@functools.cache
def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

print([fib(n) for n in range(16)])

print(fib.cache_info())

results in:

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]

CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)

Pip uses pyproject.toml to completely describe package metadata for installation.

Python type hinting, is used by some IDEs to give code warnings while not actually enforcing strict typing (unless you want to).

import math

def two_sinxy(x:float, y:float) -> float:
    return 2*math.sin(x*y)

This function would not crash if you fed int in on most interpreters, but PyCharm IDE and other can emit warnings when conflicting variable types are passed in/out.

Python argument unpacking, unpacks iterables into functions requiring multiple arguments, expanded with *iterable. Multiple iterables can be unpacked into a single function.

ipaddress is a useful standard library feature for manipulating IP addresses, used in the findssh program to scan for active servers in IP address ranges.

Object-oriented pathlib is standard library and replaces most os.path functions.

f-strings allow f'This is {weight} kg for {price} dollars. instead of 'This is {} kg for {} dollars'.format(weight,price)

Python 3.7 adds several more compelling reasons to upgrade.

Patreon transitioned from PHP → Python 3 in 2015. Key appeals for Patreon to switch to Python 3 included:

  • future-proofing
  • appeal to developer hiring, working with latest language
  • lower risk than porting to static typed language like Scala

Instagram runs fully on Python 3 as noted at the 2017 PyCon keynote at the 13 minute mark.

Starting in 2010, Arch Linux defaulted to Python 3. Ubuntu 17.10 defaulted to Python 3. Ubuntu 18.04 requires Python 3 from programs in the main Ubuntu repository with default Python 3.6. The goal is to demote Python from the main repository.

Executable Python scripts should continue to have the first line

#!/usr/bin/env python

so that users can configure their desired Python version. Many users install a third party Python distribution such as Anaconda Python, PyCharm, Intel Python, etc. that have high performance math libraries such as Cuda-based CuPy.

  • Very detailed notes from Python Software Foundation Fellow Nick Coghlan on why, when, what, how, where of Python 3 transition with fascinating historical notes.
  • ActiveState saw majority of downloads being Python 3 since January 2017.