One thing every programmer will agree on is that Python is the language for the win due to its wide use in almost all sectors from Natural Language Processing to Deep Learning and even Data Analysis. But, with the release of Python 3.9, it has just made coding even easier and efficient in many terms. Let us go through these new features.

With every new update of Python, there have been several improved, new, and even deprecated features. Although there has been work on updating Python for almost a year now, Beta versions have been around for a while. So, finally, on 5 October 2020, the new version, Python 3.9 is officially out. So, let’s dive right into what the new features are and how to use them.

PEG Parser

This is a major change you can see with Python 3.9, although this is one that you don’t usually notice in your daily coding. Ever since day 1 of Python, the LL(1) parser has been used, which analyzes one character at a time and does not believe in backtracking. A benefit of this is that these parsers are simple to implement, but the problem with these there may be a few hard cases that need to be circumvented using special hacks. For this reason, Parsing Expression Grammar (PEG) parsers are created. The chief aim of this is to an abstract syntax tree (AST) the same as that of the LL(1) parser. So, the latest Python version, 3.9, works on both these parsers, the default being the PEG parser. Even though both these parsers produce the same results, the PEG is slightly faster with a downside of using more memory. However, this change should not be very visible in practice but is a major one.

Time Zone Support

Python provides us with a module ‘datetime’ to work through date and time in the standard library, but not enough support for working with time zones. Due to this, incorporating time zone based has been tedious because the programmer was required to implement the rules manually. The ‘datetime’ module supports only UTC for setting the time zone, however, other time zones also need to be implemented.

To use this module, install using the following code:

python -m pip install tzdata

And can be implemented as follows: 

from datetime import datetime

from zoneinfo import ZoneInfo

datetime.now(tz=ZoneInfo(“Europe/Oslo”))

which gives the following output:

datetime.datetime(2020, 9, 8, 17, 12, 0, 939001, tzinfo=zoneinfo.ZoneInfo(key=’Europe/Oslo’))

This adds a new ‘zoneinfo’ standard library that helps make time-zone based working convenient. ‘zoneinfo’ helps gain access to IANA (Internet Assigned Numbers Authority) Time Zone database, which is updated several times every year. This database is huge. 

This library can help in listing all available timezones using the following code:

import zoneinfo

zoneinfo.available_timezones()

len (zoneinfo.available_timezones())

On running this, the output obtained is:

{‘America/St_Lucia’, ‘Asia/Aqtau’, … ‘Asia/Beirut’}

599

This number representing time zones varies as per installation. One important point to note here is that you must keep your local time zone database properly synced with IANA, as it is regularly updated, especially if your application is sensitive to the time zone value. Using names like “Asia/Beirut” ensure unambiguous access for any time zone.

Simple updating for dictionaries

Dictionaries are a widely used data structure in Python, having been largely used with optimization over time. To merge two dictionaries, the syntax used until now has been very tedious. Besides being tedious, it also provides undesired results by changing the original dictionary or overlapping the values.

To overcome this, Python 3.9 brings 2 new operators; known as the union (|), used to merge dictionaries, and the in-place union (|=) that will update the dictionary in place. Another difference is that in-place union can work with any data structure that is like the dictionary whereas, for the union operator, both dictionaries need to belong to the same dictionary type.

pycon = {2016: “Delhi”, 2018: “Mumbai”}

europython = {2017: “Paris”, 2018: “London”, 2019: “Rome”}

pycon | europython

pycon |= europython

pycon

Output:

{2016: “Delhi”, 2018: “London”, 2017: “Paris”, 2019: “Rome”}

Flexible Decorators

Decorators are a useful tool in Python as it allows you to modify function behavior. It also allows wrapping another function to extend the wrapped function behavior. This poses a challenge if you wish to use a decorator for connecting a button to a slot. And for doing this, earlier versions restricted the use of square brackets. This is no longer an issue in Python 3.9 as it allows any expressions, even those including dictionary items.

@buttons[“hello”].clicked.connect

def say_hello():

    message.setText(“Hello, World!”)

This may not look like a major change, but it helps in writing cleaner codes and makes dynamic choosing of decorators easy. 

Annotation of type hints

Annotation of functions was first introduced in Python 3.0 which supported adding metadata arbitrarily to functions. These were only used for documentation purposes to the user. As annotations have gained popularity for type hints, Python 3.9 introduces “typing.Annotated” that can be used to combine hints with other information. The ‘Annotated’ method takes at least two arguments, first is the regular type and the remaining are the arbitrary metadata. Here, the type checker checks only the first argument and leaves the interpretation of that metadata to the user. 

from typing import Annotated

def velocity(

    displacement: Annotated[float, “feet”], time: Annotated[float, “seconds”]

) -> Annotated[float, “km per hour”]:

      fps2kmph = 3600 / 5280  

      return displacement / time * fps2kmph

This leads to quite a lengthy code. However, to maintain a short code type aliases can be used where you could define new variables that represent annotated types. This might require some work but this makes a code short and readable. For extensive use of annotation, an annotation factory can also be used. 

Affix removal

Before Python 3.9, removal of a suffix or prefix was done using the “.strip()” method. However, its results are not always what was desired and has induced a lot of bug reports. Naturally, .strip(“ Code”) must remove the substring “Code”, but what happens instead is it removes individual characters like “ ”, “C”, “o”, “d”, and ”e”. And in case the suffix is an empty string, the whole text string is removed. To prevent all these errors from happening, Python 3.9 came up with two string methods for solving this issue: .removeprefix() and .removesuffix() for the removal of the beginning of the end of the string, respectively. A good thing about this is that in case the suffix or prefix is not seen in the string, the string is not altered at all. And, these functions remove at most one occurrence of the affix in one analysis, so, to delete multiple occurrences, a while loop can be used.

“This is a code for Python”.removesuffix(” Python”)

‘this is a code for’

“this is a code for Python’.removeprefix(“this “)

‘is a code for Python’

“this is a code for Python”.removeprefix(“Other”)

‘this is a code for Python’

Ability to type hints and dictionaries directly 

For basic types like str, bool, and int, it is easy to add type hints, but that is not the case for generics. It is basically a container that can process parameters like a list of numbers. In the earlier versions, types hints like list[float] were not readily available and needed to import a different object for the list from the ‘typing’ module

But with Python 3.9, it is not necessary to maintain this parallel hierarchy and enables the use of ‘list’ for type hints as well.

digits: list[float]

Topological sorting

 To install packages such that all its dependencies are always fulfilled, you perform what is known as the topological sort that finds a total order of all the dependencies. For this task, Python 3.9 comes with a new module called graphlib that takes care of topological sorting. Besides helping with the total ordering of a set, this module can also be used to consider tasks than enable parallelization when doing advance scheduling. The feature of the ‘TopologicalSorter’ module enables you to add edges and nodes representing dependencies easily using the .add() method.

from graphlib import TopologicalSorter

ts = TopologicalSorter(dependencies)

list(ts.static_order())

Output:

[‘html2text’, ‘sgmllib3k’, ‘feedparser’, ‘realpython-reader’]

This output indicates that you need to first download the ‘html2text’ library then the ‘sgmllib3k’ library and so on.

HTTP status codes

The IANA server seen earlier helps coordinate several key resources for the internet infrastructure, one of which is the HTTP Status Code Registry. Previous versions did support some status codes in the ‘HTTP’ standard library, but Python 3.9 adds new status codes as Early Hint (103) and (Too Early (425), accessible using the number or name.

from HTTP import HTTPStatus

HTTPStatus(425).phrase

Output: ‘Too Early’

HTTPStatus.EARLY_HINTS.value

Output: 103

IPv6 scoped address

Python 3.9 also introduced the ability to specify the scope as per requirement for an IPv6 address which is used to determine the part of the internet to which a particular IP belongs. This can be done using the % sign at the end of the address.

Modules are improved

A variety of modules have been updated in Python 3.9, some of which are as follows:

1. asyncio: removed the support for reuse_address from asyncio.loop.create_datagram_endpoint()
2. ast: enables indentation in dump()
3. compileall: use of hardlinks enabled in duplicated .pyc files
4. concurrent.features: cancel_futures parameter added to concurrent.futures.Executor.shutdown() enabling the cancellation of all pending futures rather than wait for them
5. curses: added functions like curses.set_tabsize(), curses.get_tabsize(), curses.get_escdelay() and curses.set_escdelay()
6. datetime: the function isocalendar() from datetime.date and datetime.datetime both return namedtuple()
7. http: as seen earlier, codes 103, 425 and 418 have been added 
8. random: new method random.Random.randbytes has been added.
9. smtplib: if the timeout for their constructor is 0, SMTP and SMTP_LIB return a value error
10. xml: when performing serialisation using xml.etree.ElementTree to the XML file, white space characters arr preserved within attributes
11. gcd and lcm: Python 3.9 defines 2 new methods .gcd() and .lcm() to calculate the Greatest Common Divisor and Least Common Multiple of numbers, respectively. 

So, should you switch to Python 3.9? 

Despite all these cool and new features, Python 3.9 still has a few things missing, the list of which is as follows:

Docker images are missing

Python 3.9 does not provide support for Docker images to gather preconfigured server environment and applications, instead, you will have to use python:3.9-rc-slim to install the 3.9.0-rc2 version 

Missing packages

A highly used package like Pandas doesn’t install properly with Python 3.9. What happens here is, generally to install packages, a compiled version of these packages knows as the wheel is uploaded to PyPI and you just download the binary wheel without the need to compile. But with Python 3.9, modules such as Numpy and Pandas do not have wheels, an alternative to install from source can be followed but it demands a lot more effort.

Bugs from Python

Like Python 3.8.1 which was released 2 months after Python 3.8.0 with a long list of bugfixes, Python 3.9 also needs a lot of bugfixes to use all the code that will be written using the new version else it will be of no use.

Toolchain support is lacking

With almost every new version, Python modifies the syntax in some way and so it is with Python 3.9 where you can use decorators with arbitrary expressions. However, other tools like linters and autoformatters need to support new syntax too, as until that doesn’t happen, the new syntax will hardly be of any use.