[{"_1":2,"_115":-5,"_116":-5},"loaderData",{"_3":4,"_38":-5,"_39":40,"_98":99,"_113":-5,"_114":-5},"root",{"_5":-5,"_6":-5,"_7":8,"_22":23,"_24":25,"_26":27,"_30":31,"_36":37},"user","shoppingCartId","shoppingCart",{"_9":10,"_11":12,"_13":14,"_15":16,"_17":18,"_19":20,"_21":14},"id","no-cart","count",0,"free",true,"updatedAt","2026-02-17T06:24:23.266Z","paymentGateway","STRIPE","cartItems",[],"loading","locale","en","readerTheme","system","constants",{"_28":29},"paypalClientId","AX53Cwz8jR6rVVzzZiZNWWXl0dGagi-GZrjG-S3Ua14E5nrKMn7DxDQ730O4R8LusLPp49ALn6aRVaEy","publishers",{"_32":33,"_34":35},"companies",[],"universities",[],"needsLocaleCookieSync",false,"routes/_store+/_readers+/read_/route","routes/_store+/_readers+/read_.$slug/route",{"_41":42,"_67":68,"_87":88,"_96":37,"_97":-7},"book",{"_43":44,"_45":46,"_47":48,"_49":50,"_53":54,"_56":57,"_65":66},"title","Python 201","slug","python201","metaText","Python 201 is a book for people who want to enhance their understanding of the Python programming language","coverFeatured",{"_51":52},"url","https://d2sofvawe08yqg.cloudfront.net/python201/s_featured?1620522847&1620522847","coverFeatured2x",{"_51":55},"https://d2sofvawe08yqg.cloudfront.net/python201/s_featured2x?1620522847&1620522847","authors",[58],{"_9":59,"_60":61,"_5":62},"Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6UHJvZHVjdEF1dGhvci8yNTcxNg","name","Michael Driscoll",{"_63":64},"username","mikedriscoll","showReadLinkString","sample","parts",[69,79,83],{"_43":70,"_71":72,"_73":74,"_75":76,"_77":78},"Chapter 1 - An Intro to Argument Parsing using argparse","description","","fileCount",1,"path","#leanpub-auto-chapter-1---an-intro-to-argument-parsing-using-argparse","chapters",[],{"_43":80,"_71":72,"_73":74,"_75":81,"_77":82},"Chapter 2 - The collections module","#leanpub-auto-chapter-2---the-collections-module",[],{"_43":84,"_71":72,"_73":74,"_75":85,"_77":86},"Chapter 3 - Context Managers","#leanpub-auto-chapter-3---context-managers",[],"toc",[89,94,95],{"_90":74,"_43":70,"_71":-5,"_75":76,"_73":74,"_91":-5,"_92":-5,"_93":-5},"level","quizType","sectionNum","youtubeId",{"_90":74,"_43":80,"_71":-5,"_75":81,"_73":74,"_91":-5,"_92":-5,"_93":-5},{"_90":74,"_43":84,"_71":-5,"_75":85,"_73":74,"_91":-5,"_92":-5,"_93":-5},"isUnavailableOnline","unavailableMessage","routes/_store+/_readers+/read_.$slug.$chapterId/route",{"_43":44,"_100":-5,"_41":101,"_110":-5,"_111":112},"content",{"_102":103,"_43":44,"_47":48,"_45":46,"_65":66,"_104":105,"_109":-5},"previewContent","

\n Chapter 1 - An Intro to Argument Parsing using argparse\n
- \n \n
  - \n Getting Started\n
  - \n Adding Arguments\n
  - \n Short Options and Long Options\n
  - \n Options that Conflict\n
  - \n Wrapping Up\n
  \n
\n
\n Chapter 2 - The collections module\n
- \n \n
  - \n ChainMap\n
  - \n Counter\n
  - \n defaultdict\n
  - \n deque\n
  - \n namedtuple\n
  - \n OrderedDict\n
  - \n Wrapping Up\n
  \n
\n
\n Chapter 3 - Context Managers\n
- \n \n
  - \n Creating a Context Manager class\n
  - \n Creating a Context Manager using contextlib\n
  - \n contextlib.closing(thing)\n
  - \n contextlib.suppress(*exceptions)\n
  - \n contextlib.redirect_stdout / redirect_stderr\n
  - \n ExitStack\n
  - \n Reentrant Context Managers\n
  - \n Wrapping Up\n
  \n
\n

Chapter 1 - An Intro to Argument Parsing using argparse

\n\n

Have you ever wondered how to process command line arguments in Python?\nYeah, there’s a module for that. It’s called argparse, which is a\nreplacement for optparse. In this article, we’ll be taking a whirlwind\ntour of this helpful module. Let’s start with something simple!

\n\n

Getting Started

\n\n

I have always found the simplest way to explain a coding concept is to\nshow some code. So that’s what we’re going to do. Here’s a super simple\nexample that doesn’t do much of anything:

\n\n

>>> import argparse\n>>> parser = argparse.ArgumentParser(\n...         description=\"A simple argument parser\",\n...         epilog=\"This is where you might put example usage\"\n...     )\n... \n>>> parser.print_help()\nusage: _sandbox.py [-h]\n\nA simple argument parser\n\noptional arguments:\n  -h, --help  show this help message and exit\n\nThis is where you might put example usage\n

\n\n\n\n

Here we just import argparse and give it a description and set up a\nusage section. The idea here is that when you ask the program you are\ncreating for help, it will tell you how to use it. In this case, it\nprints out a simple description, the default optional arguments (“-h” in\nthis case) and example usage.

\n\n

Now let’s make this example a bit more concrete. You won’t normally be\nparsing arguments from the command-line after all. So we’ll move the\ncode into a Python function inside of a Python file:

\n\n

 1 # arg_demo.py\n 2 \n 3 import argparse\n 4 \n 5 \n 6 def get_args():\n 7     \"\"\"\"\"\"\n 8     parser = argparse.ArgumentParser(\n 9         description=\"A simple argument parser\",\n10         epilog=\"This is where you might put example usage\"\n11     )\n12     return parser.parse_args()\n13 \n14 if __name__ == '__main__':\n15     get_args()\n

\n\n\n\n

Now let’s call the script from the command line:

\n\n

1 python arg_demo.py -h\n

\n\n\n\n

This will print out the help text like we saw earlier. Now let’s learn\nabout how to add some of our own custom arguments.

\n\n

Adding Arguments

\n\n

Let’s write some code that adds three new arguments that our parser can\nunderstand. We’ll add an argument that is required and two that are not.\nWe’ll also look at adding a default and a required type. Here’s the\ncode:

\n\n

 1 # arg_demo2.py\n 2 \n 3 import argparse\n 4 \n 5 \n 6 def get_args():\n 7     \"\"\"\"\"\"\n 8     parser = argparse.ArgumentParser(\n 9         description=\"A simple argument parser\",\n10         epilog=\"This is where you might put example usage\"\n11     )\n12 \n13     # required argument\n14     parser.add_argument('-x', action=\"store\", required=True,\n15                         help='Help text for option X')\n16     # optional arguments\n17     parser.add_argument('-y', help='Help text for option Y', default=False)\n18     parser.add_argument('-z', help='Help text for option Z', type=int)\n19     print(parser.parse_args())\n20 \n21 if __name__ == '__main__':\n22     get_args()\n

\n\n\n\n

Now let’s run it a few times so you can see what happens:

\n\n

 1 mike@pc:~/py/argsparsing$ python arg_demo2.py\n 2 usage: arg_demo2.py [-h] -x X [-y Y] [-z Z]\n 3 arg_demo2.py: error: argument -x is required\n 4 \n 5 mike@pc:~/py/argsparsing$ python arg_demo2.py -x something\n 6 Namespace(x='something', y=False, z=None)\n 7 \n 8 mike@pc:~/py/argsparsing$ python arg_demo2.py -x something -y text\n 9 Namespace(x='something', y='text', z=None)\n10 \n11 mike@pc:~/py/argsparsing$ python arg_demo2.py -x something -z text\n12 usage: arg_demo2.py [-h] -x X [-y Y] [-z Z]\n13 arg_demo2.py: error: argument -z: invalid int value: 'text'\n14 \n15 mike@pc:~/py/argsparsing$ python arg_demo2.py -x something -z 10\n16 Namespace(x='something', y=False, z=10)\n

\n\n\n\n

As you can see, if you run the code without passing it any arguments,\nyou will get an error. Next we pass it just the required argument so you\ncan see what the defaults are for the other two. Then we try passing\n“text” to the ‘-y’ argument and that gets stored, so we know it doesn’t\nrequire a Boolean. The last two examples show what happens when you pass\nan invalid and a valid value to the ‘-z’ argument.

\n\n

By the way, the argument names do not have to be one character in\nlength. You can change those something more descriptive, like ‘arg1` or\n‘simulator’ or whatever you want.

\n\n

Short Options and Long Options

\n\n

Let’s take a look at how we might use a short option versus a long a\none. We actually are already using the short option here:

\n\n

1 parser.add_argument('-x', action=\"store\", required=True,\n2                     help='Help text for option X')\n

\n\n\n\n

If we wanted a long option, then we’d just need to add it right after\nthe short one. Here’s an example:

\n\n

1 parser.add_argument('-x', '--execute', action=\"store\", required=True,\n2                     help='Help text for option X')                        \n

\n\n\n\n

You will note that a long option is more than one character in length\nand that it must start with two dashes instead of one.

\n\n

Options that Conflict

\n\n

What do you do if you have options that conflict with each other? A\ncommon example would be running your application in verbose mode versus\nquiet mode. You can run it in either mode, but not both. How do we\nprevent the user from running it that way though? It’s actually quite\neasy via the mutually_exclusive_group function. Let’s pretend that\noptions x and y cannot run at the same time and modify our code\naccordingly:

\n\n

 1 import argparse\n 2 \n 3 \n 4 def get_args():\n 5     \"\"\"\"\"\"\n 6     parser = argparse.ArgumentParser(\n 7         description=\"A simple argument parser\",\n 8         epilog=\"This is where you might put example usage\"\n 9     )\n10 \n11     group = parser.add_mutually_exclusive_group()\n12     group.add_argument('-x', '--execute', action=\"store\",\n13                         help='Help text for option X')\n14     group.add_argument('-y', help='Help text for option Y', default=False)\n15 \n16     parser.add_argument('-z', help='Help text for option Z', type=int)\n17     print(parser.parse_args())\n18 \n19 if __name__ == '__main__':\n20     get_args()\n

\n\n\n\n

You will note that we have to create a mutually exclusive group. Then we\nadd the options that need to be mutually exclusive to that group. The\nrest go into the regular parser group. Let’s try running the code with\nboth options like this:

\n\n

1 python arg_demo3.py -x 10 -y 2\n

\n\n\n\n

When I did this command, I ended up getting the following output:

\n\n

1 usage: arg_demo3.py [-h] [-x EXECUTE | -y Y] [-z Z]\n2 arg_demo2.py: error: argument -y: not allowed with argument -x/--execute    \n

\n\n\n\n

Obviously that didn’t work and the argparse module told us why.

\n\n

Wrapping Up

\n\n

You now know how to create the basics of an argument parser. There are\nmany other aspects of this module that you might be interested in, such\nas defining an alternate destination name for the argument to be saved\nto, using different prefixes (i.e. ‘+’ instead of ‘-‘), creating\nargument groups and more. I recommend checking out the documentation for\nmore details.

\n\n\n

Chapter 2 - The collections module

\n\n

Python’s collections module has specialized container datatypes that can\nbe used to replace Python’s general purpose containers (dict, tuple,\nlist, and set). We will be studying the following parts of this fun\nmodule:

\n\n

\n
\n
ChainMap
\n
defaultdict
\n
deque
\n
namedtuple
\n
OrderedDict
\n
\n

\n\n

There is a sub-module of collections called abc or Abstract Base\nClasses. These will not be covered in this chapter.

\n\n

Let’s get started with the ChainMap container!

\n\n

ChainMap

\n\n

A ChainMap is a class that provides the ability to link multiple\nmappings together such that they end up being a single unit. If you look\nat the documentation, you will notice that it accepts **maps*,\nwhich means that a ChainMap will accept any number of mappings or\ndictionaries and turn them into a single view that you can update. Let’s\nlook at an example so you can see how this works:

\n\n

>>> from collections import ChainMap\n>>> car_parts = {'hood': 500, 'engine': 5000, 'front_door': 750}\n>>> car_options = {'A/C': 1000, 'Turbo': 2500, 'rollbar': 300}\n>>> car_accessories = {'cover': 100, 'hood_ornament': 150, 'seat_cover': 99}\n>>> car_pricing = ChainMap(car_accessories, car_options, car_parts)\n>>> car_pricing['hood']\n500\n

\n\n\n\n

Here we import ChainMap from our collections module. Next we create\nthree dictionaries. Then we create an instance of our ChainMap by\npassing in the three dictionaries that we just created. Finally, we try\naccessing one of the keys in our ChainMap. When we do this, the ChainMap\nwill go through each map in order to see if that key exists and has a\nvalue. If it does, then the ChainMap will return the first value it\nfinds that matches that key.

\n\n

This is especially useful if you want to set up defaults. Let’s pretend\nthat we want to create an application that has some defaults. The\napplication will also be aware of the operating system’s environment\nvariables. If there is an environment variable that matches one of the\nkeys that we are defaulting to in our application, the environment will\noverride our default. Let’s further pretend that we can pass arguments\nto our application. These arguments take precendence over the\nenvironment and the defaults. This is one place where a ChainMap can\nreally shine. Let’s look at a simple example that’s based on one from\nPython’s documentation:

\n\n

 1 import argparse\n 2 import os\n 3 \n 4 from collections import ChainMap\n 5 \n 6 \n 7 def main():\n 8     app_defaults = {'username':'admin', 'password':'admin'}\n 9 \n10     parser = argparse.ArgumentParser()\n11     parser.add_argument('-u', '--username')\n12     parser.add_argument('-p', '--password')\n13     args = parser.parse_args()\n14     command_line_arguments = {key:value for key, value \n15                               in vars(args).items() if value}\n16 \n17     chain = ChainMap(command_line_arguments, os.environ, \n18                      app_defaults)\n19     print(chain['username'])\n20 \n21 if __name__ == '__main__':\n22     main()\n23     os.environ['username'] = 'test'\n24     main()\n

\n\n\n\n

Let’s break this down a little. Here we import Python’s argparse\nmodule along with the os module. We also import ChainMap.Next we\nhave a simple function that has some silly defaults. I’ve seen these\ndefaults used for some popular routers. Then we set up our argument\nparser and tell it how to handle certain command line options. You will\nnotice that argparse doesn’t provide a way to get a dictionary object of\nits arguments, so we use a dict comprehension to extract what we need.\nThe other cool piece here is the use of Python’s built-in vars. If\nyou were to call it without an argument, vars would behave like Python’s\nbuilt-in locals. But if you do pass in an object, then vars is the\nequivalent to object’s __dict__ property.

\n\n

In other words, vars(args) equals args.__dict__. Finally\ncreate our ChainMap by passing in our command line arguments (if there\nare any), then the environment variables and finally the defaults. At\nthe end of the code, we try calling our function, then setting an\nenvironment variable and calling it again. Give it a try and you’ll see\nthat it prints out admin and then test as expected. Now let’s\ntry calling the script with a command line argument:

\n\n

1 python chain_map.py -u mike\n

\n\n\n\n

When I ran this, I got mike back twice. This is because our command\nline argument overrides everything else. It doesn’t matter that we set\nthe environment because our ChainMap will look at the command line\narguments first before anything else.

\n\n

Now that you know how to use ChainMaps, we can move on to the Counter!

\n\n

Counter

\n\n

The collections module also provides us with a neat little tool that\nsupports convenient and fast tallies. This tool is called Counter.\nYou can run it against most iterables. Let’s try it out with a string!

\n\n

>>> from collections import Counter\n>>> Counter('superfluous')\nCounter({'u': 3, 's': 2, 'e': 1, 'l': 1, 'f': 1, 'o': 1, 'r': 1, 'p': 1})\n>>> counter = Counter('superfluous')\n>>> counter['u']\n3\n

\n\n\n\n

In this example, we import Counter from collections and then pass it\na string. This returns a Counter object that is a subclass of Python’s\ndictionary. We then run the same command but assign it to the variable\ncounter so we can access the dictionary a bit easier. In this case,\nwe saw that the letter “u” occurs three times in the example string.

\n\n

The Counter provides a few methods that might interest you. For example,\nyou can call elements which will an iterator over the elements that\nare in the dictionary, but in an arbitrary order. You can kind of think\nof this function as a “scrambler” as the output in this case is a\nscrambled version of the string.

\n\n

>>> list(counter.elements())\n['e', 'l', 'f', 'o', 'r', 's', 's', 'p', 'u', 'u', 'u']\n

\n\n\n\n

Another useful method is most_common. You can ask the Counter what\nthe most common items are by passing in a number that represents what\nthe top recurring “n” items are:

\n\n

>>> counter.most_common(2)\n[('u', 3), ('s', 2)]\n

\n\n\n\n

Here we just ask our Counter what the top two most recurring items were.\nAs you can see, it produced a list of tuples that tells us “u” occurred\n3 times and “s” occurred twice.

\n\n

The other method that I want to cover is the subtract method. The\nsubtract method accepts an iterable or a mapping and the uses that\nargument to subtract. It’s a bit easier to explain if you see some code:

\n\n

>>> counter_one = Counter('superfluous')\n>>> counter_one\nCounter({'u': 3, 's': 2, 'e': 1, 'l': 1, 'f': 1, 'o': 1, 'r': 1, 'p': 1})\n>>> counter_two = Counter('super')\n>>> counter_one.subtract(counter_two)\nNone\n>>> counter_one\nCounter({'u': 2, 'l': 1, 'f': 1, 'o': 1, 's': 1, 'e': 0, 'r': 0, 'p': 0})\n

\n\n\n\n

So here we recreate our first counter and print it out so we know what’s\nin it. That we create our second Counter object. Finally we subtract the\nsecond counter from the first. If you look carefully at the output at\nthe end, you will notice the that number of letters for five of the\nitems has been decremented by one.

\n\n

As I mentioned at the beginning of this section, you can use the Counter\nagainst any iterable or mapping, so you don’t have to just use strings.\nYou can also pass it tuples, dictionaries and lists! Give it a try on\nyour own to see how it works with those other data types.

\n\n

Now we’re ready to move on to the defaultdict!

\n\n

defaultdict

\n\n

The collections module has a handy tool called defaultdict. The\ndefaultdict is a subclass of Python’s dict that accepts a\ndefault_factory as its primary argument. The default_factory is\nusually a Python type, such as int or list, but you can also use a\nfunction or a lambda too. Let’s start by creating a regular Python\ndictionary that counts the number of times each word is used in a\nsentence:

\n\n

 1 sentence = \"The red for jumped over the fence and ran to the zoo for food\"\n 2 words = sentence.split(' ')\n 3 \n 4 reg_dict = {}\n 5 for word in words:\n 6     if word in reg_dict:\n 7         reg_dict[word] += 1\n 8     else:\n 9         reg_dict[word] = 1\n10 \n11 print(reg_dict)\n

\n\n\n\n

If you run this code, you should see output that is similar to the\nfollowing:

\n\n

 1 {'The': 1,\n 2  'and': 1,\n 3  'fence': 1,\n 4  'food': 1,\n 5  'for': 2,\n 6  'jumped': 1,\n 7  'over': 1,\n 8  'ran': 1,\n 9  'red': 1,\n10  'the': 2,\n11  'to': 1,\n12  'zoo': 1}\n

\n\n\n\n

Now let’s try doing the same thing with defaultdict!

\n\n

 1 from collections import defaultdict\n 2 \n 3 \n 4 sentence = \"The red for jumped over the fence and ran to the zoo for food\"\n 5 words = sentence.split(' ')\n 6 \n 7 d = defaultdict(int)\n 8 for word in words:\n 9     d[word] += 1\n10 \n11 print(d)\n

\n\n\n\n

You will notice right away that the code is much simpler. The\ndefaultdict will automatically assign zero as the value to any key it\ndoesn’t already have in it. We add one so it makes more sense and it\nwill also increment if the word appears multiple times in the sentence.

\n\n

 1 defaultdict(<class 'int'>,\n 2             {'The': 1,\n 3              'and': 1,\n 4              'fence': 1,\n 5              'food': 1,\n 6              'for': 2,\n 7              'jumped': 1,\n 8              'over': 1,\n 9              'ran': 1,\n10              'red': 1,\n11              'the': 2,\n12              'to': 1,\n13              'zoo': 1})\n

\n\n\n\n

Now let’s try using a Python list type as our default factory. We’ll\nstart off with a regular dictionary first, as before.

\n\n

 1 my_list = [(1234, 100.23), (345, 10.45), (1234, 75.00),\n 2            (345, 222.66), (678, 300.25), (1234, 35.67)]\n 3 \n 4 reg_dict = {}\n 5 for acct_num, value in my_list:\n 6     if acct_num in reg_dict:\n 7         reg_dict[acct_num].append(value)\n 8     else:\n 9         reg_dict[acct_num] = [value]\n10 \n11 print(reg_dict)\n

\n\n\n\n

This example is based on some code I wrote a few years ago. Basically I\nwas reading a file line by line and needed to grab the account number\nand the payment amount and keep track of them. Then at the end, I would\nsum up each account. We’re skipping the summing part here. If You run\nthis code, you should get some output similar to the following:

\n\n

1 {345: [10.45, 222.66], 678: [300.25], 1234: [100.23, 75.0, 35.67]}\n

\n\n\n\n

Now let’s re-implement this code using defaultdict:

\n\n

 1 from collections import defaultdict\n 2 \n 3 \n 4 my_list = [(1234, 100.23), (345, 10.45), (1234, 75.00),\n 5            (345, 222.66), (678, 300.25), (1234, 35.67)]\n 6 \n 7 d = defaultdict(list)\n 8 for acct_num, value in my_list:\n 9     d[acct_num].append(value)\n10 \n11 print(d)\n

\n\n\n\n

Once again, this cuts out the if/else conditional logic and makes the\ncode easier to follow. Here’s the output from the code above:

\n\n

1 defaultdict(<class 'list'>,\n2             {345: [10.45, 222.66],\n3              678: [300.25],\n4              1234: [100.23, 75.0, 35.67]})\n

\n\n\n\n

This is some pretty cool stuff! Let’s go ahead and try using a lambda\ntoo as our default_factory!

\n\n

>>> from collections import defaultdict\n>>> animal = defaultdict(lambda: \"Monkey\")\n>>> animal['Sam'] = 'Tiger'\n>>> print (animal['Nick'])\nMonkey\n>>> animal\ndefaultdict(<function <lambda> at 0x7f32f26da8c0>, {'Nick': 'Monkey', 'Sam': 'Ti\\\nger'})\n

\n\n\n\n

Here we create a defaultdict that will assign ‘Monkey’ as the default\nvalue to any key. The first key we set to ‘Tiger’, then the next key we\ndon’t set at all. If you print the second key, you will see that it got\nassigned ‘Monkey’. In case you haven’t noticed yet, it’s basically\nimpossible to cause a KeyError to happen as long as you set the\ndefault_factory to something that makes sense. The documentation does\nmention that if you happen to set the default_factory to None, then you\nwill receive a KeyError. Let’s see how that works:

\n\n

>>> from collections import defaultdict\n>>> x = defaultdict(None)\n>>> x['Mike']\nTraceback (most recent call last):\n  Python Shell, prompt 41, line 1\nKeyError: 'Mike'\n

\n\n\n\n

In this case, we just created a very broken defaultdict. It can no\nlonger assign a default to our key, so it throws a KeyError instead. Of\ncourse, since it is a subclass of dict, we can just set the key to some\nvalue and it will work. But that kind of defeats the purpose of the\ndefaultdict.

\n\n

deque

\n\n

According to the Python documentation, deques “are a generalization\nof stacks and queues”. They are pronounced “deck” which is short for\n“double-ended queue”. They are a replacement container for the Python\nlist. Deques are thread-safe and support memory efficient appends and\npops from either side of the deque. A list is optimized for fast\nfixed-length operations. You can get all the gory details in the Python\ndocumentation. A deque accepts a maxlen argument which sets the\nbounds for the deque. Otherwise the deque will grow to an arbitrary\nsize. When a bounded deque is full, any new items added will cause the\nsame number of items to be popped off the other end.

\n\n

As a general rule, if you need fast appends or fast pops, use a deque.\nIf you need fast random access, use a list. Let’s take a few moments to\nlook at how you might create and use a deque.

\n\n

>>> from collections import deque\n>>> import string\n>>> d = deque(string.ascii_lowercase)\n>>> for letter in d:\n...     print(letter)\n

\n\n\n\n

Here we import the deque from our collections module and we also import\nthe string module. To actually create an instance of a deque, we\nneed to pass it an iterable. In this case, we passed it\nstring.ascii_lowercase, which returns a list of all the lower case\nletters in the alphabet. Finally, we loop over our deque and print out\neach item. Now let’s look at at a few of the methods that deque\npossesses.

\n\n

>>> d.append('bork')\n>>> d\ndeque(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', \n       'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'bork'])\n>>> d.appendleft('test')\n>>> d\ndeque(['test', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',\n       'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'bork'])\n>>> d.rotate(1)\n>>> d\ndeque(['bork', 'test', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l\\\n',\n       'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'])\n

\n\n\n\n

Let’s break this down a bit. First we append a string to the right end\nof the deque. Then we append another string to the left side of the\ndeque.. Lastly, we call rotate on our deque and pass it a one, which\ncauses it to rotate one time to the right. In other words, it causes one\nitem to rotate off the right end and onto the front. You can pass it a\nnegative number to make the deque rotate to the left instead.

\n\n

Let’s finish out this section by looking at an example that’s based on\nsomething from the Python documentation:

\n\n

 1 from collections import deque\n 2 \n 3 \n 4 def get_last(filename, n=5):\n 5     \"\"\"\n 6     Returns the last n lines from the file\n 7     \"\"\"\n 8     try:\n 9         with open(filename) as f:\n10             return deque(f, n)\n11     except OSError:\n12         print(\"Error opening file: {}\".format(filename))\n13         raise\n

\n\n\n\n

This code works in much the same way as Linux’s tail program does.\nHere we pass in a filename to our script along with the n number of\nlines we want returned. The deque is bounded to whatever number we pass\nin as n. This means that once the deque is full, when new lines are read\nin and added to the deque, older lines are popped off the other end and\ndiscarded. I also wrapped the file opening with statement in a\nsimple exception handler because it’s really easy to pass in a malformed\npath. This will catch files that don’t exist for example.

\n\n

Now we’re ready to move on and learn about the namedtuple.

\n\n

namedtuple

\n\n

The one that we’ll be focusing on in this section is the namedtuple\nwhich you can use to replace Python’s tuple. Of course, the\nnamedtuple is not a drop-in replacement as you will soon see. I have\nseen some programmers use it like a struct. If you haven’t used a\nlanguage with a struct in it, then that needs a little explanation. A\nstruct is basically a complex data type that groups a list of variables\nunder one name. Let’s look at an example of how to create a namedtuple\nso you can see how they work:

\n\n

1 from collections import namedtuple\n2 \n3 Parts = namedtuple('Parts', 'id_num desc cost amount')\n4 auto_parts = Parts(id_num='1234', desc='Ford Engine',\n5                cost=1200.00, amount=10)\n6 print(auto_parts.id_num)\n

\n\n\n\n

Here we import namedtuple from the collections module. Then we\ncalled namedtuple, which will return a new subclass of a tuple but with\nnamed fields. So basically we just created a new tuple class. you will\nnote that we have a strange string as our second argument. This is a\nspace delimited list of properties that we want to create.

\n\n

Now that we have our shiny new class, let’s create an instance of it! As\nyou can see above, we do that as our very next step when we create the\nauto_parts object. Now we can access the various items in our\nauto_parts using dot notation because they are now properties of our\nParts class.

\n\n

One of the benefits of using a namedtuple over a regular tuple is that\nyou no longer have to keep track of each item’s index because now each\nitem is named and accessed via a class property. Here’s the difference\nin code:

\n\n

>>> auto_parts = ('1234', 'Ford Engine', 1200.00, 10)\n>>> auto_parts[2]  # access the cost\n1200.0\n>>> id_num, desc, cost, amount = auto_parts\n>>> id_num\n'1234'\n

\n\n\n\n

In the code above, we create a regular tuple and access the cost of the\nvehicle engine by telling Python the appropriate index we want.\nAlternatively, we can also extract everything from the tuple using\nmultiple assignment. Personally, I prefer the namedtuple approach just\nbecause it fits the mind easier and you can use Python’s dir()\nmethod to inspect the tuple and find out its properties. Give that a try\nand see what happens!

\n\n

The other day I was looking for a way to convert a Python dictionary\ninto an object and I came across some code that did something like this:

\n\n

>>> from collections import namedtuple\n\n>>> Parts = {'id_num':'1234', 'desc':'Ford Engine',\n         'cost':1200.00, 'amount':10}\n>>> parts = namedtuple('Parts', Parts.keys())(**Parts)\n>>> parts\nParts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')\n

\n\n\n\n

This is some weird code, so let’s take it a piece at a time. The first\nline we import namedtuple as before. Next we create a Parts dictionary.\nSo far, so good. Now we’re ready for the weird part. Here we create our\nnamedtuple class and name it ‘Parts’. The second argument is a list of\nthe keys from our dictionary. The last piece is this strange piece of\ncode: (Parts)**. The double asterisk means that we are calling our\nclass using keyword arguments, which in this case is our dictionary. We\ncould split this line into two parts to make it a little clearer:

\n\n

>>> parts = namedtuple('Parts', Parts.keys())\n>>> parts\n<class '__main__.Parts'>\n>>> auto_parts = parts(**Parts)\n>>> auto_parts\nParts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')\n

\n\n\n\n

So here we do the same thing as before, except that we create the class\nfirst, then we call the class with our dictionary to create an object.\nThe only other piece I want to mention is that namedtuple also accepts a\nverbose argument and a rename argument. The verbose argument is\na flag that will print out class definition right before it’s built if\nyou set it to True. The rename argument is useful if you’re creating\nyour namedtuple from a database or some other system that your program\ndoesn’t control as it will automatically rename the properties for you.

\n\n

At this point you should be familiar enough with the namedtuple to use\nit yourself, so let’s check out the OrderedDict!

\n\n

OrderedDict

\n\n

Python’s collections module has another great subclass of dict known as\nOrderedDict. As the name implies, this dictionary keeps track of the\norder of the keys as they are added. If you create a regular dict, you\nwill note that it is an unordered data collection:

\n\n

>>> d = {'banana': 3, 'apple':4, 'pear': 1, 'orange': 2}\n>>> d\n{'apple': 4, 'banana': 3, 'orange': 2, 'pear': 1}\n

\n\n\n\n

Every time you print it out, the order may be different. There are times\nwhen you will need to loop over the keys of your dictionary in a\nspecific order. For example, I have had a use case where I needed the\nkeys sorted so I could loop over them in order. To do that, you can do\nthe following:

\n\n

>>> keys = d.keys()\n>>> keys\ndict_keys(['apple', 'orange', 'banana', 'pear'])\n>>> keys = sorted(keys)\n['apple', 'banana', 'orange', 'pear']\n>>> for key in keys:\n...     print (key, d[key])\n... \napple 4\nbanana 3\norange 2\npear 1\n

\n\n\n\n

Let’s create an instance of an OrderedDict using our original dict, but\nduring the creation, we’ll sort the dictionary’s keys:

\n\n

>>> from collections import OrderedDict\n>>> d = {'banana': 3, 'apple':4, 'pear': 1, 'orange': 2}\n>>> new_d = OrderedDict(sorted(d.items()))\n>>> new_d\nOrderedDict([('apple', 4), ('banana', 3), ('orange', 2), ('pear', 1)])\n>>> for key in new_d:\n...     print (key, new_d[key])\n... \napple 4\nbanana 3\norange 2\npear 1\n

\n\n\n\n

Here we create our OrderedDict by sorting it on the fly using Python’s\nsorted built-in function. The sorted function takes in the dictionary’s\nitems, which is a list of tuples that represent the key pairs of the\ndictionary. It sorts them and then passes them into the OrderedDict,\nwhich will retain their order. Thus when we go to print our the keys and\nvalues, they are in the order we expect. If you were to loop over a\nregular dictionary (not a sorted list of keys), the order would change\nall the time.

\n\n

Note that if you add new keys, they will be added to the end of the\nOrderedDict instead of being automatically sorted.

\n\n

Something else to note about OrderDicts is that when you go to compare\ntwo OrderedDicts, they will not only test the items for equality, but\nalso that the order is correct. A regular dictionary only looks at the\ncontents of the dictionary and doesn’t care about its order.

\n\n

Finally, OrderDicts have two new methods in Python 3: popitem and\nmove_to_end. The popitem method will return and remove a (key, item)\npair. The move_to_end method will move an existing key to either end\nof the OrderedDict. The item will be moved right end if the last\nargument for OrderedDict is set to True (which is the default), or to\nthe beginning if it is False.

\n\n

Interestingly, OrderedDicts support reverse iteration using Python’s\nreversed built-in function:

\n\n

>>> for key in reversed(new_d):\n...     print (key, new_d[key])\n... \npear 1\norange 2\nbanana 3\napple 4\n

\n\n\n\n

Pretty neat, although you probably won’t be needing that functionality\nevery day.

\n\n

Wrapping Up

\n\n

We’ve covered a lot of ground in this chapter. You learned how to use a\ndefaultdict and an OrderedDict. We also learned about a neat subclass of\nPython’s list, the deque. Finally we looked at how to use a namedtuple\nto do various activities. I hope you found each of these collections\ninteresting. They may be of great use to you in your own coding life.

\n\n\n

Chapter 3 - Context Managers

\n\n

Python came out with a special new keyword several years ago in Python\n2.5 that is known as the with statement. This new keyword allows a\ndeveloper to create context managers. But wait! What’s a context\nmanager? They are handy constructs that allow you to set something up\nand tear something down automatically. For example, you might want to\nopen a file, write a bunch of stuff to it and then close it. This is\nprobably the classic example of a context manager. In fact, Python\ncreates one automatically for you when you open a file using the with\nstatement:

\n\n

1 with open(path, 'w') as f_obj:\n2     f_obj.write(some_data)\n

\n\n\n\n

Back in Python 2.4, you would have to do it the old fashioned way:

\n\n

1 f_obj = open(path, 'w')\n2 f_obj.write(some_data)\n3 f_obj.close()\n

\n\n\n\n

The way this works under the covers is by using some of Python’s magic\nmethods: __enter__ and __exit__. Let’s try creating our\nown context manager to demonstrate how this all works!

\n\n

Creating a Context Manager class

\n\n

Rather than rewrite Python’s open method here, we’ll create a context\nmanager that can create a SQLite database connection and close it when\nit’s done. Here’s a simple example:

\n\n

 1 import sqlite3\n 2 \n 3 \n 4 class DataConn:\n 5     \"\"\"\"\"\"\n 6 \n 7     def __init__(self, db_name):\n 8         \"\"\"Constructor\"\"\"\n 9         self.db_name = db_name\n10 \n11     def __enter__(self):\n12         \"\"\"\n13         Open the database connection\n14         \"\"\"\n15         self.conn = sqlite3.connect(self.db_name)\n16         return self.conn\n17 \n18     def __exit__(self, exc_type, exc_val, exc_tb):\n19         \"\"\"\n20         Close the connection\n21         \"\"\"\n22         self.conn.close()\n23         if exc_val:\n24             raise\n25 \n26 if __name__ == '__main__':\n27     db = '/home/mdriscoll/test.db'\n28     with DataConn(db) as conn:\n29         cursor = conn.cursor()\n

\n\n\n\n

In the code above, we created a class that takes a path to a SQLite\ndatabase file. The __enter__ method executes automatically where\nit creates and returns the database connection object. Now that we have\nthat, we can create a cursor and write to the database or query it. When\nwe exit the with statement, it causes the __exit__ method to\nexecute and that closes the connection.

\n\n

Let’s try creating a context manager using another method.

\n\n

Creating a Context Manager using contextlib

\n\n

Python 2.5 not only added the with statement, but it also added the\ncontextlib module. This allows us to create a context manager using\ncontextlib’s contextmanager function as a decorator. Let’s try creating\na context manager that opens and closes a file after all:

\n\n

 1 from contextlib import contextmanager\n 2 \n 3 @contextmanager\n 4 def file_open(path):\n 5     try:\n 6         f_obj = open(path, 'w')\n 7         yield f_obj\n 8     except OSError:\n 9         print(\"We had an error!\")\n10     finally:\n11         print('Closing file')\n12         f_obj.close()\n13 \n14 if __name__ == '__main__':\n15     with file_open('/home/mdriscoll/test.txt') as fobj:\n16         fobj.write('Testing context managers')\n

\n\n\n\n

Here we just import contextmanager from contextlib and decorate\nour file_open function with it. This allows us to call file_open using\nPython’s with statement. In our function, we open the file and then\nyield it out so the calling function can use it.

\n\n

Once the with statement ends, control returns back to the file_open\nfunction and it continues with the code following the yield statement.\nThat causes the finally statement to execute, which closes the file. If\nwe happen to have an OSError while working with the file, it gets\ncaught and finally statement still closes the file handler.

\n\n

contextlib.closing(thing)

\n\n

The contextlib module comes with some other handy utilities. The first\none is the closing class which will close the thing upon the\ncompletion of code block. The Python documentation gives an example\nthat’s similar to the following one:

\n\n

1 from contextlib import contextmanager\n2 \n3 @contextmanager\n4 def closing(db):\n5     try:\n6         yield db.conn()\n7     finally:\n8         db.close()\n

\n\n\n\n

Basically what we’re doing is creating a closing function that’s wrapped\nin a contextmanager. This is the equivalent of what the closing class\ndoes. The difference is that instead of a decorator, we can use the\nclosing class itself in our with statement. Let’s take a look:

\n\n

1 from contextlib import closing\n2 from urllib.request import urlopen\n3 \n4 with closing(urlopen('http://www.google.com')) as webpage:\n5     for line in webpage:\n6         # process the line\n7         pass\n

\n\n\n\n

In this example, we open a url page but wrap it with our closing class.\nThis will cause the handle to the web page to be closed once we fall out\nof the with statement’s code block.

\n\n

contextlib.suppress(*exceptions)

\n\n

Another handy little tool is the suppress class which was added in\nPython 3.4. The idea behind this context manager utility is that it can\nsuppress any number of exceptions. Let’s say we want to ignore the\nFileNotFoundError exception. If you were to write the following\ncontext manager, it wouldn’t work:

\n\n

>>> with open('fauxfile.txt') as fobj:\n        for line in fobj:\n            print(line)\n\nTraceback (most recent call last):\n  Python Shell, prompt 4, line 1\nbuiltins.FileNotFoundError: [Errno 2] No such file or directory: 'fauxfile.txt'\n

\n\n\n\n

As you can see, this context manager doesn’t handle this exception. If\nyou want to ignore this error, then you can do the following:

\n\n

1 from contextlib import suppress\n2 \n3 with suppress(FileNotFoundError):\n4     with open('fauxfile.txt') as fobj:\n5         for line in fobj:\n6             print(line)\n

\n\n\n\n

Here we import suppress and pass it the exception that we want to\nignore, which in this case is the FileNotFoundError exception. If you\nrun this code, you will note that nothing happens as the file does not\nexist, but an error is also not raised. It should be noted that this\ncontext manager is reentrant. This will be explained later on in\nthis section.

\n\n

contextlib.redirect_stdout / redirect_stderr

\n\n

The contextlib library has a couple of neat tools for redirecting stdout\nand stderr that were added in Python 3.4 and 3.5 respectively. Before\nthese tools were added, if you wanted to redirect stdout, you would do\nsomething like this:

\n\n

1 path = '/path/to/text.txt'\n2 \n3 with open(path, 'w') as fobj:\n4     sys.stdout = fobj\n5     help(sum)\n

\n\n\n\n

With the contextlib module, you can now do the following:

\n\n

1 from contextlib import redirect_stdout\n2 \n3 path = '/path/to/text.txt'\n4 with open(path, 'w') as fobj:\n5     with redirect_stdout(fobj):\n6         help(redirect_stdout)\n

\n\n\n\n

In both of these examples, we are redirecting stdout to a file. When we\ncall Python’s help, instead of printing to stdout, it gets saved\ndirectly to the file. You could also redirect stdout to some kind of\nbuffer or a text control type widget from a user interface toolkit like\nTkinter or wxPython.

\n\n

ExitStack

\n\n

ExitStack is a context manager that will allow you to easily\nprogrammatically combine other context managers and cleanup functions.\nIt sounds kind of confusing at first, so let’s take a look at an example\nfrom the Python documentation to help us understand this idea a bit\nbetter:

\n\n

>>> from contextlib import ExitStack\n>>> with ExitStack as stack:\n        file_objects = [stack.enter_context(open(filename))\n            for filename in filenames]\n                    ]\n

\n\n\n\n

This code basically creates a series of context managers inside the list\ncomprehension. The ExitStack maintains a stack of registered callbacks\nthat it will call in reverse order when the instance it closed, which\nhappens when we exit the the bottom of the with statement.

\n\n

There are a bunch of neat examples in the Python documentation for\ncontextlib where you can learn about topics like the following:

\n\n

\n
\n
Catching exceptions from __enter__ methods
\n
Supports a variable number of context managers
\n
Replacing any use of try-finally
\n
and much more!
\n
\n

\n\n

I highly recommend checking it out so you get a good feel for how\npowerful this class is.

\n\n

Reentrant Context Managers

\n\n

Most context managers that you create will be written such that they can\nonly be used once using a with statement. Here’s a simple example:

\n\n

>>> from contextlib import contextmanager\n>>> @contextmanager\n... def single():\n...     print('Yielding')\n...     yield\n...     print('Exiting context manager')\n>>> context = single()\n>>> with context:\n...     pass\n... \nYielding\nExiting context manager\n>>> with context:\n...     pass\n... \nTraceback (most recent call last):\n  Python Shell, prompt 9, line 1\n  File \"/usr/local/lib/python3.5/contextlib.py\", line 61, in __enter__\n    raise RuntimeError(\"generator didn't yield\") from None\nbuiltins.RuntimeError: generator didn't yield\n

\n\n\n\n

Here we create an instance of our context manager and try running it\ntwice with Python’s with statement. The second time it runs, it raises a\nRuntimeError.

\n\n

But what if we wanted to be able to run the context manager twice? Well\nwe’d need to use one that is “reentrant”. Let’s use the\nredirect_stdout context manager that we used before!

\n\n

>>> from contextlib import redirect_stdout\n>>> from io import StringIO\n>>> stream = StringIO()\n>>> write_to_stream = redirect_stdout(stream)\n>>> with write_to_stream:\n...     print('Write something to the stream')\n...     with write_to_stream:\n...         print('Write something else to stream')\n... \n>>> print(stream.getvalue())\nWrite something to the stream\nWrite something else to stream\n

\n\n\n\n

Here we create a nested context manager where they both write to\nStringIO, which is an in-memory text stream. The reason this works\ninstead of raising a RuntimeError like before is that redirect_stdout\nis reentrant and allows us to call it twice. Of course, a real world\nexample would be much more complex with more functions calling each\nother. Please also note that reentrant context managers are not\nnecessarily thread-safe. Read the documentation before trying to use\nthem in a thread.

\n\n

Wrapping Up

\n\n

Context managers are a lot of fun and come in handy all the time. I use\nthem in my automated tests all the time for opening and closing dialogs,\nfor example. Now you should be able to use some of Python’s built-in\ntools to create your own context managers. Be sure to take the time to\nread the Python documentation on contextlib as there are lots of\nadditional information that is not covered in this chapter. Have fun and\nhappy coding!

\n\n

\n\n","tocStructure",[106,107,108],{"_90":74,"_43":70,"_71":-5,"_75":76,"_73":74,"_91":-5,"_92":-5,"_93":-5},{"_90":74,"_43":80,"_71":-5,"_75":81,"_73":74,"_91":-5,"_92":-5,"_93":-5},{"_90":74,"_43":84,"_71":-5,"_75":85,"_73":74,"_91":-5,"_92":-5,"_93":-5},"publicChapterContent","nextChapter","sections",[],"routes/_store+/_layout","routes/_store+/_readers+/_layout","actionData","errors"]

Table of Contents

Chapter 1 - An Intro to Argument Parsing using argparse

Getting Started

Adding Arguments

Short Options and Long Options

Options that Conflict

Wrapping Up

Chapter 2 - The collections module

ChainMap

Counter

defaultdict

deque

namedtuple

OrderedDict

Wrapping Up

Chapter 3 - Context Managers

Creating a Context Manager class

Creating a Context Manager using contextlib

contextlib.closing(thing)

contextlib.suppress(*exceptions)

contextlib.redirect_stdout / redirect_stderr

ExitStack

Reentrant Context Managers

Wrapping Up