Who's Afraid of the Big Bad 3

Lennart Regebro

PyCon PL, Szczyrk, 2014

<Cue dramatic music!>

Who wants Python 3 anyway?

Python 2 is good enough!

You will have to spend hundreds of man-hours!

There are no third-party modules!

Python 3 was a mistake!

No other language ever break backwards compatibility!

Everything breaks!

Python 3 is really a new language!

The core developers don't listen!

Or?

Time to Third-party!

• 165 of the 200 top packages on the Cheeseshop support Python 3
• Over 4000 Python 3 packages on the Cheeseshop.

You want Python 3!

Although you might not know it yet

Extended Iterable Unpacking

>>> first, second, *rest, last = \
...     "a b c d e f".split()
>>> first, second, last
('a', 'b', 'f')

Keyword only arguments

>>> def foo(a, *args, b, **kw):
...   print(a, args, b, kw)

>>> foo(1, 2, 3, b=4, c=5)
1 (2, 3) b {'c': 5}

Chained exceptions

>>> raise KeyError("wut?") from ZeroDivisionError()
ZeroDivisionError

The above exception was the direct cause of
the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'wut?'

Better OS Exceptions

       BlockingIOError ChildProcessError
       ConnectionError BrokenPipeError
ConnectionAbortedError ConnectionRefusedError
  ConnectionResetError FileExistsError
     FileNotFoundError InterruptedError
     IsADirectoryError NotADirectoryError
       PermissionError ProcessLookupError
          TimeoutError

File handle warnings

__main__:1: ResourceWarning: unclosed file

Yield from

>>> def my_generator():
...     yield from range(1,5)
...     yield from range(10,15)
...
>>> list(my_generator())
[1, 2, 3, 4, 10, 11, 12, 13, 14]

Simply super

Python 2

super(ClassName, self).method(foo, bar)

Python 3

super().method(foo, bar)

asyncio

Supporting Python 3 is not so bad

Many changes are handled by 2to3

• Exception syntax
• print is a function
• xrange is gone
• Standard library reorganisation
• etc...

Some changes need no handling at all

• dict.keys() no longer returns a list
• Indentation is stricter
• Long and Int are merged

If you need Python 2 compatibility

>>> from __future__ import division
>>> from __future__ import print_function
>>> print("Three halves is written", 3/2, "with decimals.")
Three halves is written 1.5 with decimals.

u'' is back!

So what IS hard?

API changes

Example 1: zope.interface

class TheComponent(object):
    implements(ITheInterface)

Example 1: zope.interface

@implementor(ITheInterface)
class TheComponent(object):
    pass

Example 2: icalendar

ical = str(icalendarobject)

Example 2: icalendar

ical = icalendarobject.to_ical()

Bytes/Strings/Unicode

You gotta keep'em separated!

Practical Experiences

Diazo

Tool 1: caniusepython3

https://caniusepython3.com/

$ caniusepython3 --project diazo

You need 3 projects to transition to Python 3.
Of those 3 projects, 2 has no direct
dependencies blocking its transition:

  repoze.xmliter (which is blocking diazo)
  experimental.cssselect (which is blocking diazo)

Adding Python 3 support to collective.checkdocs

Tool 2: 2to3

$ 2to3 -w .

collective.checkdocs

Time spent: ~4h

Adding Python 3 support to repoze.xmliter

Tool 3a: Tox

$ tox

__________________ summary __________________
  py26: commands succeeded
  py27: commands succeeded
ERROR:   py34: commands failed
  pep8: commands succeeded

Tool 3b: Virtualenv + bash

$ virtualenv-2.7 .venv/py27
$ virtualenv-3.4 .venv/py34

And a small script:

#!/bin/bash
.venv/py27/bin/python setup.py test
.venv/py34/bin/python setup.py test

Tool 3c: Spiny

The Unicode problem

if sys.version_info > (3,):
    unicode = str

doctype_re_b = re.compile(
    b"^<!DOCTYPE\\s[^>]+>\\s*", re.MULTILINE)
doctype_re_u = re.compile(
    u"^<!DOCTYPE\\s[^>]+>\\s*", re.MULTILINE)

if isinstance(result, unicode):
    result, subs = doctype_re_u.subn(
        self.doctype, result, 1)
else:
    result, subs = doctype_re_b.subn(
        self.doctype.encode(), result, 1)

repoze.xmliter

Time spent: < 6h

Adding Python 3 support to Diazo

Tool 4: Futurize

$ pip install future

Running a single fixer

futurize -w -f <thespecificfixer> .

Updating the Diazo documentation

Updating the documentation

1. Replace the WSGI server with gearbox
2. Replace Paster#urlmap with rutter
3. Replace Paster#static with ???
4. Replace Paster#proxy with ???

Introducing webobentrypoints

1. Replace the WSGI server with gearbox
2. Replace Paster#urlmap with rutter
3. Replace Paster#static with webobentrypoints#staticdir
4. Replace Paster#proxy with webobentrypoints#proxy

Diazo

Adding Python 3 support to Diazo: ~3h

Switching from Paster: < 6h

Total time: Less than 20 hours!

General workflow

1. caniusepython3
2. Make sure you have tests
3. Fix one problem at a time

Fixing problems

1. Run fixers one by one
2. 2to3 or Futurize
3. Review the changes

Conclusions

• No, Python 3 is not a mistake!
• Yes, you want to use Python 3.
• No, it's really not that hard (mostly).

Just Do It!

Yes We Can!

YOLO!

Been there, done that!

You can do it!

Thanks!

Props to

• Aaron Meurer's talk "10 awesome features of Python that you can't use because you refuse to upgrade to Python 3"
• Loads of People on Twitter
• PyCon Ireland for inviting me and hence forcing me to write a talk

Questions?