Reading Code: Python's requests library

In preparation for the latest PizzaPY I browsed the code of Python's requests library. (You've got to love open source.) I'm of the opinion that must read lots of code in order to be able to write decent code yourself. The more diverse you read, the better for you. Basically the old adage from traditional writing:

If you want to be a writer, you must do two things above all others: read a lot and write a lot.

—Stephen King

I stumbled over a shockingly mondane Makefile, was reminded of Pipfiles, felt better at seeing Kenneth commenting out tests for three months, too (possibly even great devs such as Kenneth forget things?) and squirmed at the output of flake8:

requests$ flake8 requests | grep -v requests/packages | wc -l

And then I visited the text property of the Response object in and saw this:

def text(self):
    # ...
    if not self.content:
        return str('')

Hm? return str('')? Why not return ''? Got to be about Python 2 vs 3, no? But how? In both major Python versions, this is true:

>>> type('') == type(str(''))

So, … why? Well, as it turns out requests uses the dynamic nature of Python quite a bit.

As you may know, str is part of the builtins. And builtins, while being essential and imported automatically, is merely adding names to the current module like any other import statement would. And you can overwrite them, as you can with any other name. And that's exactly what requests is doing at the top of the

from .compat import (
    cookielib, urlunparse, urlsplit, urlencode, str, bytes, StringIO,
    is_py2, chardet, builtin_str, basestring)

In we find the resolution to the riddle (leaving out the cluttering code around):

if is_py2:
    str = unicode
elif is_py3:
    str = str

So, in requests, str does not mean, what it means in the standard library. It always mean “a Unicode string”. I guess that's very helpful for Kenneth, but seems to violate the principle of least surprise

Wonder why unicode wasn't used for the purpose. I guess because the requests authors wanted to make their code as much Python 3 as possible, so when they drop support for 2.x, it's straightforward.

Anyway, reading code is fun. You always find gems and quirks and learn from them.