What is Jupyter and how do you use it?¶
Project Jupyter exists to develop software and standards for interactive computing. Jupyter, which you are using now, provides an interactive notebook which runs on a cloud-server, and with which you interact using a web browser.
-As installed, Jupyter provides two things:
--
-
- An interactive Python notebook, which you are using now (this is an interactive notebook!) -
- An interactive bash terminal (we will use this later) -
What is an interactive Python notebook? It is a notebook that mixes documentation (like this!) with Python code (like below), with the output of that code.
-For example, below is the Python code to print "Hello World". Click on the below code and then press "SHIFT+Return". You should see that the code is run, and below "Hello World" will be printed.
- -print("Hello World")
-The above code is interactive because it runs when you pressed "SHIFT+Return". It is also interactive because you can change it. For example, click again on the code and change "Hello World" to "Hello Jupyter". Press "SHIFT+Return" again and you should see "Hello Jupyter" printed.
-A code cell can have as much or little Python in it as you want. The below cell defines a function and then runs it in a loop. What do you think will be printed out when you select it and press "SHIFT+Return"? Have a go. Are you right?
- -def countDown(start):
- for i in range(start, 0, -1):
- print("%d..." % i)
- print("Lift off!")
-
-countDown(10)
-Each cell can use the variables and functions defined in previous cells. For example, the next cell sets the value of the variable x to 5, while the following cell then uses that to call the countDown function.
x = 5
-countDown(x)
-You can go back up and edit the value of x. For example, set x to 7 and press SHIFT+Return. This has changed x. However, it doesn't re-run any code that depends on x. That is for you to do. You need to click on the next cell countDown(x) and press SHIFT+Return again. You should then see that the code is run and updated.
Because you can change any cell at any time, a notebook can get into a confused state. If we change x again to 21, then the cell countDown(x), is now out of date, as it depends on the old value of x. So, how can we know what state each cell is in? The state is based on the order in which cells are executed. This is shown by the little number next to each cell, e.g. see below.
The number "10" shows that the line x = 21 was the 10th cell executed by Python, while the line countDown(x) was the 9th cell executed. As countDown(x) was executed before x = 21, it was not affected by x = 21. If you want it to be affected, you need to execute that cell again (so it is now the 11th cell executed), e.g.
If you find yourself getting confused, then click "Kernel | Restart & Clear Output" from the menu above to clear all outputs, clear the state, and reset back to a clean notebook.
-
Interactive Graphics¶
In addition to mixing documentation with interactive Python, the notebook also allows Python modules to embed interactive graphics. For example, the BioSimSpace Project provides a easy wrapper around the Python molecular viewer nglview. This can be used to view molecules from within a notebook.
-Execute the cell below by selecting it and pressing SHIFT+Return. This will import the viewMolecules function from BioSimSpace, which will load the molecules contained in the file data/complex.pdb and will display it below.
from BioSimSpace import viewMolecules
-v = viewMolecules("data/complex.pdb")
-You can rotate and interact with molecules once they have loaded (the "[*]" next to a cell indicates that it is processing. It will turn into a number once processing is complete)
-You can change molecular representations and interact with the 3D molecule view. Later on we will go through some examples showing what you can do and how you can select parts of the molecule (or click here if you want to jump ahead and learn now).
-In addition to viewing molecules, you can also draw graphs. This is achieved using a combination of the matplotlib, numpy and pandas modules. For example, the below code draws a sine and cosine curve.
- -# First, import pandas, numpy and matplotlib
-import pandas as pd
-from pandas import Series, DataFrame
-import numpy as np
-import matplotlib.pyplot as plt
-%config InlineBackend.figure_format = 'svg' #Â helps make things look better in Jupyter :-)
-
-# now define a sine and cosine curve
-X = np.linspace(-np.pi, np.pi, 256, endpoint=True)
-data = {'cos': np.cos(X), 'sin': np.sin(X)}
-trig = DataFrame(index=X, data=data)
-
-trig.plot()
-Later on we will go through pandas, matplotlib and numpy and you will learn how to draw lots of different and interesting graphs (or click the links to jump ahead and learn now).
- -Mixing documentation and code¶
In addition to interactivity, the real benefit of a Jupyter notebook is that it encourages you to include lots of documentation with your code. In effect, a notebook is a combination of code, documentation, and (once it has been run) analysis and data.
-You can create documentation by changing the type of a cell. The type of a cell is indicated in the drop-down at the top, under the menu. If you click on the graph above, you will see that the type of a code cell is "Code".
-
If you now click on this documentation, you should see the cell type change to "Markdown"
-
Creating your own cells¶
You can create your own cells in two different ways;
--
-
- either by pressing SHIFT+Return on the last cell (which creates a new cell at the bottom of the notebook), -
- or by clicking "Insert | Cell Above" or "Insert | Cell Below" from the menu to insert a new cell above or below the currently selected cell. -
You can choose whether this should be a code or documentation (markdown) cell by selecting the appropriate value from the drop-down at the top.
-Code cells can contain any valid Python. Documentation (markdown) cells can contain any text. The text can be formatted, using the markdown standard.
-Have a go by creating some new documentation and code cells below. If you want some inspiration for documentation, take a look at the documentation cells above by double-clicking on them. You should see how I have inserted images, lists, headings etc.
-You can then render the documentation using SHIFT+Return (just like running a code cell)
- -
-
-Controlling the Python kernel¶
The Python process that is interpreting and executing your Python code is referred to as the "kernel" of the notebook (hence the "Kernel" menu above). When you execute a code cell it sends the code to the Python kernel, and then waits for the kernel to send a result back. The Python kernel can only execute one thing at a time. This means that, for slow functions, you could be executing cells too quickly and leave the Python kernel behind.
-For example, here is a function that fakes a slow function as it goes to sleep for the specified number of seconds.
- -def sleep(n):
- import os
- os.system("sleep %d" % n)
-Below we will have three code cells that each sleep for a long time. Execute all three quickly one after another...
- -sleep(30)
-sleep(30)
-sleep(30)
-You should see that there is a [*] next to each code cell. This is because they are all waiting for the Python kernel to finish executing and return.
-
If you wait you will see each sleep finishing and control then passing to the next.
-Sometimes you don't want to be patient, and you want to stop the Python kernel from running a function. To stop (interupt) the kernel, either click the "Stop" icon at the top of the screen, or select "Kernel | Interupt" from the menu.
-
Other useful buttons up there are "restart the kernel" (same as the menu Kernel | Restart & Clear Output) and the arrows that can move the selected cell up and down the notebook.
-Have a play. Explore the buttons and menu items. Take a look at the "Help" in the menu. Remember that, if everything goes wrong, you can always click "Stop" or select "Kernel | Restart & Clear Output" from the menu.
- -
-Running external programs in a code cell¶
You can do more than just execute Python in the code cells. You can run any command you want! Just put an exclamation mark first, and then type the command normally as you would in a terminal. For example, the below cell is running ls
! ls
-You can run any command that you want, exactly as you would typing in a terminal. For example, lets use cat to read the LICENSE file...
! cat LICENSE
-You can work with directories as well. For example, here we will copy the dioxin.pdb file from data into the current directory and will then view it using viewMolecules...
! cp data/dioxin.pdb ./
-from BioSimSpace import viewMolecules #Â importing again in case we restarted the kernel
-viewMolecules("dioxin.pdb")
-Saving the notebook¶
At the top of the screen is the name of the notebook, and its current save state.
-
The current save state tells you when the last checkpoint was saved, and whether or not the notebook contains any unsaved changes. Checkpoints provide points in the notebook that you can refer back to, using the menu item "File | Revert to Checkpoint". You are always able to revert to the last checkpoint, meaning that if you make a mistake, you can go back to a previous safe version.
-If you click "File | Save and Checkpoint", or click the "Save" icon (floppy disk icon on the far left), then the current state of the notebook will be saved. This includes not just the code and documentation you have written, but also all of the outputs and graphs that you have created. This is really useful, as it lets you save your work, and then come back later.
-For example, what don't you save now, and then close this window and then open the notebook again? You should see that everything you have done, and everything that was output is restored. Did it work?
-The notebook was saved to a file called 01_jupyter_howto.ipynb. The .ipynb signifies that the file is a Jupyter notebook (formerly called an interactive python notebook). You can change the name by clicking on the name of the notebook above, and then choosing a different name in the window that opens.
Alternatively, you can save a copy of this notebook under a different name by clicking "File | Make a Copy..." in the menu. You can then copy this file or send it to someone else. As a notebook contains the code, data, outputs and analysis, sharing notebooks is the closest thing we have to interactive papers. This is a really powerful concept and, I believe, will fundementally change how we report and share the results of computational and data analysis in science.
- -
-The Jupyter Hub¶
When you first logged into this website you launched this notebook by clicking on its name in the Jupyter Hub. This was the page that looked like a file browser and that (hopefully) is still open in one of the tabs in your browser. The Jupyter Hub is not just a file brower. It is the gateway to do a lot more than just interact with a single notebook.
-The first thing to note is that this Jupyter Hub is running on the cloud and consumes resources. It is important that you click "Logout" (top right) when you have finished, as this will free up the hub to be used by someone else. There is a timer that will automatically log you out 40 minutes after your web browser is closed. However, please remember to log out if you can.
-
The main view of the hub is the file browser that shows all of your files. You can navigate through the files and directories like any normal file browser. For example, you can see above the 01_jupyter_howto.ipynb file that contains this notebook. You should see that the file has a green icon next to it, as well as a Running indicator on the right. This indicates that the notebook is currently running.
Running notebooks consume cloud resources, which are limited. You can only have 1.8 GB of data in memory and have limited CPU. You should thus shutdown any running notebooks that you aren't using any more. To do this, click the checkbox to the left of the notebook filename. This will bring up some buttons at the top of the screen. Click the "Shutdown" button to shut down the notebook.
-
Alternatively, you can go to the "Running" tab to see everything of your's that is currently running in the hub. Click the "Shutdown" button on the right for anything that you aren't using and want to shut down.
-
Back in the "Files" tab, clicking the checkbox next to a file brings up buttons that let you download files. You can use the hub to download files by selecting them and clicking the "Download" button that will appear. Other buttons will appear depending on the type of file, e.g. "Duplicate", "View", "Edit" etc. The bin icon will delete the file.
-
You can create new notebooks or files by clicking on the "New" button on the right. This will open a drop-down menu for you to select the type of thing to create.
-
As well as letting you create a new jupyter notebook (by clicking "New | Python3"), you can also create new text files ("New | Text File") or new folders ("New | Folder"). Most interestingly, is that you can also create a new bash terminal. If you click "New | Terminal" it will open a new tab in which you will have a fully functional bash terminal shell. This is running on the same cloud server as your notebook. It is 100% completely a fully functional terminal within your web browser. You could even use it to run the normal command-line python ;-)
-
The cloud server on which this is running comes with lots of useful software that you can run from within the bash shell. For example you have;
--
-
update_workshops: This command will update all of the software and workshop material to the latest version. Useful if you delete anything by accident or the tutor needs to make a last-minute fix
-git: Git is installed. The workshop material is downloaded and updated from git repositories, e.g. https://github.com/ccpbiosim/python_and_data. You could also use git to push things back to the cloud, e.g. if you want to download lots of files.
-anaconda python3: A full anaconda python3 is installed, with pandas, matplotlib, numpy etc. etc.
-wget: This tool lets you quickly download (or should that be sideload?) files onto the cloud server
-top: Process monitor. You can use this to see which processes are consuming lots of resources. Any process that uses more than 1.8 GB of memory will automatically be killed by the cloud server. You can kill them yourself using thekillcommand.
-tarandbzip2: useful for packing/unpacking and compressing/uncompressing files
-AmberTools: The full AmberTools suite is installed in directory$AMBERHOME
-ProtoMS: The full ProtoMS package is installed in directory$PROTOMSHOME
-whamandwham-2d: Tools used for weighted histogram analysis
-SireandBioSimSpace: The Sire and BioSimSpace python simulation framework and workflow node packages.
-
The cloud server was build with docker. If you want to download it yourself to run on your own computer at home after the workshop then please install docker and type
- -docker run -it --rm -p 8888:8888 chryswoods/bss-workshop:latestThis will download and run the image from dockerhub and will make it available at the web address that will be printed to the screen (e.g. it will look something like http://localhost:8888/?token=641396480e6421eae8b18261d82a75f958fe166e1c8b20a8). Simply open that address in your browser :-). You can see and download the DockerFile used to generate this image from here.
Thanks to the Microsoft Azure Kubernetes Service on which this Jupyter image is running.
- -
-Lists¶
Lists provide a simple way to hold a collection of values. You create a list using square brackets. For example, here we can create a list that contains the values "cat", "dog" and "horse"
-a =[ "cat", "dog", "horse" ]
-a = ["cat", "dog", "horse"]
-print(a)
-You can access the items in the list by placing the index of the item in square brackets. The first item is at index 0
- -a[0]
-The second item is at index 1, and the third item at index 2.
- -a[2]
-What do you think will happen if we access the item at index 3?
- -a[3]
-You can also access the items in the list from the back. What do you think is at index -1?
- -a[-1]
-What about index -2, -3 or -4?
- -a[-3]
-You can add items onto a list by using the .append function. You can find this using tab completion and Python help...
help(a.append)
-a.append("fish")
-a
-a[3]
-You can put whatever you want into a list, including other lists!
- -b = [ 42, 15, a, [7,8,9] ]
-b[3]
-Putting lists inside lists allows for multidimensional lookup, e.g. can you work out why b[3][2] equals 9?
b[3][2]
-You can loop over the items in a list using a for loop, e.g.
- -for x in a:
- print(x)
-You can get the number of items in the list using the len function.
len(a)
-You can use this as an alternative way of looping over the elements of a list
- -for i in range(0,len(a)):
- print(a[i])
-A string behaves like a list of letters. For example, if we have the string s = "Hello World", then s[0] is "H" and s[-1] is d.
s = "Hello World"
-s[-1]
-You can loop over every letter in a string in the same way that you can loop over every item in a list.
- -for letter in s:
- print(letter)
-Exercises¶
Exercise 1¶
Create two Python lists called a and b. Put into these lists the values [2, 4, 6, 8], and [10, 20, 30, 40].
-Check that a[2] equals 6 and b[-1] equals 40. (note that you will need to use the menu "Insert | Insert Cell Below" to insert more cells below to create space for your code)
-Exercise 2¶
Now create a loop that loops over each item in a and b and that calculates and prints out the product a[i] * b[i].
-Exercise 3¶
Modify your code to create a list called c. Use the .append function to set c[i] = a[i] * b[i]. Check your code by making sure that c[-1] equals 320.
-Dictionaries¶
Dictionaries are another type of Python container. Instead of storing values by index, they store them associated with a key.
-You create dictionaries using curly brackets, assiging values to their keys using a colon, e.g.
-a = { "cat" : "mieow", "dog" : "woof", "horse" : "neigh" }
-a = { "cat" : "mieow", "dog" : "woof", "horse" : "neigh"}
-a
-You can look up values in the dictionary by placing the key in square brackets. For example, we can look up the value associated with the key "cat" using a["cat"].
a["cat"]
-What happens if the key does not exist?
- -a['fish']
-You insert items into the dictionary by assigning values to keys, e.g.
- -a["fish"] = "bubble"
-a
-You can list all of the keys or values of a dictionary using the keys or values functions (which you can find using tab completion and Python help)
help(a.values)
-a.keys()
-a.values()
-You can loop over the dictionary by looping over the keys and looking up the values in a for loop, e.g.
- -for key in a.keys():
- print("A %s goes %s" % (key, a[key]))
-You can put anything as a value into a dictionary, including other dictionaries and even lists. The keys should be either numbers or strings.
- -b = { "a" : ["aardvark", "anteater", "antelope"], "b" : ["badger", "beetle"], 26.5: a}
-What do you think is at b["a"][-1]? What about b[26.5]["fish"]?
b[26.5]["fish"]
-Exercise¶
Below you have a dictionary that contains the full mapping of every letter to its Morse-code equivalent.
- -letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-Exercise 1¶
Use the morse code dictionary to look up the morse code for the letters "s" and "o". What is the morse code for "SOS" (the international emergency distress signal)?
- -
-Exercise 2¶
Here is a string that contains a message that must be converted to morse code. Write a loop that converts each letter into the morse code equivalent, and stores it into a list. Print the list out to see the full morse code message that must be sent. Note that you will need to use the .lower() function to get the lower case of capital letters.
message = "SOS We have hit an iceberg and need help quickly"
-
-Exercise 3¶
The inverted form of a dictionary is one where the keys are now looked up by value. For example, the below code inverts letter_to_morse such that the morse code is the key, and the letter is the value.
morse_to_letter = {}
-for letter in letter_to_morse.keys():
- morse_to_letter[ letter_to_morse[letter] ] = letter
-Check that this code works by verifying that morse_to_letter["..."] equals "s".
Next, loop through the morse code message you created in exercise 2 and see if you can convert it back to english. Note that you can join a list of letters together into a string using the code "".join(letters).
-Functions¶
Functions provide a way to package often-used code into reusable and easy to use components. For example, here is some code that multiplies together two lists
- -list1 = [2, 4, 6, 8]
-list2 = [10, 20, 30, 40]
-list3 = []
-for x, y in zip(list1,list2):
- list3.append(x * y)
-list3
-We don't want to keep typing out the above code every time we want to multiply the numbers in two lists. Instead, we can collect that code together into a function
- -def multiply(a, b):
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-We can now call this function directly on our lists, e.g.
- -list3 = multiply(list1, list2)
-list3
-The function is called using its name, and passing in the values as two arguments, e.g.
- -list4 = multiply( [1,2,3], [4,5,6] )
-list4
-The arguments are placed inside the round brackets. These are copied, in order, to the function. For example, [1,2,3] is copied into a, and [4,5,6] is copied as b. The code in the function is then executed. We can watch this code being run by adding in print statements, e.g.
def multiply(a, b):
- print("a = %s" % a)
- print("b = %s" % b)
- c = []
- for x,y in zip(a,b):
- print("%s times %s equals %s" % (x,y,x*y))
- c.append(x*y)
- print("c = %s" % c)
- return c
-You must pass the right number of arguments into a function. For example, this is what happens if you get the number of arguments wrong...
- -list5 = multiply(list1)
-You can write functions that take as many (or as few) arguments as you want. For example, here is a function that takes no arguments, and then a function that takes lots
- -def func0():
- return "no arguments to this function"
-def func1(a, b, c, d, e=5):
- return a+b+c+d+e
-func0()
-func1(1, 2, 3, 4, 5)
-func1(1, 2, 3, 4)
-Note that with the last function we have set a default value of the argument e. This is given the value of 5 if it is not specified. This allows us to pass 4 arguments instead of 5. Changing the default value by editing the definition of the function above will thus change the output of func1 when it is called with only four arguments.
Exercise¶
Here is the morse code dictionary from the last session, together with the code that converts a message from english into morse code.
- -letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-message = "SOS We have hit an iceberg and need help quickly"
-morse = []
-for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
-print(morse)
-Exercise 1¶
Create a function called encode that takes a message and returns the morse code equivalent. Test this function by encodig the message SOS We have hit an iceberg and need help quickly and check that you get the same result as in the last session. Now try using your function to encode other messages.
-Exercise 2¶
Using the answer from Exercise 2 in the dictionaries lesson, write a function called decode that converts a morse code message back to english. Check that you can decode the above morse code message back to english.
-Exercise 3¶
Below is a list of messages. Loop over the messages and check that encode( decode(message) ) equals the original message. Do any of the messages fail to encode and decode correctly? If so, why? How can your check be modified to account for the limitations of your encode and decode functions?
- -messages = [ "hello world", "this is a long message", "Oh no this may break", "This message is difficult to encode." ]
-
-Objects¶
In the last session you learned how to package up useful code into functions. This is a really useful idea, as it lets you re-use useful code in your own scripts, and to then share useful code with other people.
-However, it is normal for functions to rely on data. For example, consider the Morse code encode and decode functions in the last lesson. These only work because of the data contained in the letter_to_morse dictionary. The functions would break if anyone changes the data in this dictionary.
letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-def encode(message):
- morse = []
- for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
- return morse
-encode("Hello")
-The above encode("Hello") has worked. However, if we change the data in letter_to_morse, e.g. swapping l from .-.. to -.--, then we get ['....', '.', '-.--', '-.--', '---'], which is wrong. We can make even larger changes, which would completely break the function...
While such changes are easy to spot in this example, they become more difficult to find in larger programs. In addition, as you share code, you will find that people using your code will do weird things to the data on which it depends, which can introduce weird bugs and problems.
-The solution is to package a function together with the data on which it depends into a single object. This idea is the foundation of object orientated programming. To explore this, let us start with a simple example that packages the encode function together with the letter_to_morse dictionary on which it depends.
class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-Above, we have packaged the data (letter_to_morse) together with the encode function into what we call a class. A Class describes how data and functions are combined together. An instance of a class is called an object, which we can create by calling Morse().
m = Morse()
-m is an object of the class Morse. It has its own copy of letter_to_morse within it, and its own copy of the encode function. We can call m's copy of the encode function by typing m.encode(...), e.g.
m.encode("Hello World")
-To create a new class, you use the class keyword, followed by the name of your class. In this case, class Morse defined a new class called Morse. You then add a colon, and write all of the functions that should be part of the class indented below. At a minimum, you must define one function, called the constructor. This function has the signature def __init__(self, arguments...). The first argument, self, is a special variable that allows an object of the class to access the data that belongs to itself. It is the job of the constructor to set up that data. For example, let's now create a new class that provides a simple guessing game.
class GuessGame:
- def __init__(self, secret):
- self._secret = secret
-
- def guess(self, value):
- if (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- print("Try again...")
-In this class, the constructor __init__(self, secret) takes an extra argument after self. This argument is saved as the _secret variable that is part of the self of the object. Note that we always name variables that are part of a class with a leading underscore. We can construct different object instances of GuessGame that have different secrets, e.g.
g1 = GuessGame("cat")
-g2 = GuessGame("dog")
-Here, the self._secret for g1 equals "cat". The self._secret for g2 equals "dog".
When we call the function g1.guess(value), it compares value against self._secret for g1.
g1.guess("dog")
-g1.guess("cat")
-When we call the function g2.guess(value) it compares value against self._secret for g2.
g2.guess("cat")
-g2.guess("dog")
-Exercise¶
Exercise 1¶
Edit the below GuessGame example so that it records how many unsuccessful guesses have been performed. Add a function called nGuesses() that returns the number of unsuccessful guesses. Once you have made the changes, check your class by creating an object of your class and using it to make some successful and unsuccessful guesses.
class GuessGame:
- def __init__(self, secret):
- self._secret = secret
-
- def guess(self, value):
- if (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- print("Try again...")
-
-Exercise 2¶
Edit the constructor of your GuessGame class so that the user can optionally specify a maximum number of allowable guesses. If the maximum number of guesses is not supplied, then set the default value to 5.
Create a maxGuesses() function that returns the maximum number of allowable guesses.
Finally, edit the guess() function so that it will not let you make more than the maximum number of guesses (e.g. if the number of guesses exceeds the maximum number, then print out "Sorry, you have run out of guesses.").
Check that you code works by creating an object of GuessGame that only allows three guesses, and see what happens if you guess incorrectly more than three times.
- -
-Classes¶
Classes allow you to define how to package data with functions to create objects. An object is an instance of a class, which contains its own data, and its own copy of functions that can operate on that data.
-You use classes to define objects that represent the concepts and things that your program will work with. For example, if your program managed exam results of students, then you may create one class that represents an Exam, and another that represents a Student.
- -class Exam:
- def __init__(self, max_score=100):
- self._max_score = max_score
- self._actual_score = 0
-
- def percent(self):
- return 100.0 * self._actual_score / self._max_score
-
- def setResult(self, score):
- if (score < 0):
- self._actual_score = 0
- elif (score > self._max_score):
- self._actual_score = self._max_score
- else:
- self._actual_score = score
-
- def grade(self):
- if (self._actual_score == 0):
- return "U"
- elif (self.percent() > 90.0):
- return "A"
- elif (self.percent() > 80.0):
- return "B"
- elif (self.percent() > 70.0):
- return "C"
- else:
- return "F"
-class Student:
- def __init__(self):
- self._exams = {}
-
- def addExam(self, name, exam):
- self._exams[name] = exam
-
- def addResult(self, name, score):
- self._exams[name].setResult(score)
-
- def result(self, exam):
- return self._exams[exam].percent()
-
- def grade(self, exam):
- return self._exams[exam].grade()
-
- def grades(self):
- g = {}
- for exam in self._exams.keys():
- g[exam] = self.grade(exam)
- return g
-We can now create a student, and give them a set of exams that they need to complete.
- -s = Student()
-s.addExam( "maths", Exam(20) )
-s.addExam( "chemistry", Exam(75) )
-At this point, the student has not completed any exams, so the grades are all 'U'
- -s.grades()
-However, we can now add the results...
- -s.addResult("maths", 15)
-s.addResult("chemistry", 62)
-s.grades()
-Programming with classes makes the code easier to read, as the code more closely represents the concepts that make up the program. For example, here we have a class that represents a full school of students.
- -class School:
- def __init__(self):
- self._students = {}
- self._exams = []
-
- def addStudent(self, name):
- self._students[name] = Student()
-
- def addExam(self, exam, max_score):
- self._exams.append(exam)
-
- for key in self._students.keys():
- self._students[key].addExam(exam, Exam(max_score))
-
- def addResult(self, name, exam, score):
- self._students[name].addResult(exam, score)
-
- def grades(self):
- g = {}
- for name in self._students.keys():
- g[name] = self._students[name].grades()
- return g
-We can now create a whole school of students and manage the exams and results for all of them with some reasonably readable code :-)
- -school = School()
-school.addStudent("Charlie")
-school.addStudent("Matt")
-school.addStudent("James")
-school.addExam( "maths", 20 )
-school.addExam( "physics", 50 )
-school.addExam( "english literature", 30 )
-school.grades()
-We can now add in the results of the exams, which have been returned to us by the exam markers...
- -englit_results = { "Charlie" : 10, "Matt" : 25, "James" : 3 }
-phys_results = { "Matt" : 48, "James" : 3 }
-maths_results = { "James" : 20, "Matt" : 18, "Charlie" : 4 }
-Indeed, we will do this by using a function...
- -def add_results(school, exam, results):
- for student in results.keys():
- school.addResult(student, exam, results[student])
-add_results(school, "english literature", englit_results)
-add_results(school, "physics", phys_results)
-add_results(school, "maths", maths_results)
-school.grades()
-class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-
-Exercise 2¶
Below is a copy of the School class, together with a copy of the code needed to populate an object of that class with students and exam results. Edit the School class to add in the following functions:
-
-
.resits(): this should return the list of exams that each student should resit if they get a "F" or "U" grade.
-.prizeStudent(): this should return the name of the student who scored the highest average percent across all of the exams.
-.reviseCourse(threshold): this should return the name of the exam that gets the lowest average score across all students, if the average score is belowthreshold.
-
Use these functions to find out which students need to resit which exams, which student should be awarded the annual school prize, and which courses should be revised as the average mark is less than 50%.
- -class School:
- def __init__(self):
- self._students = {}
- self._exams = []
-
- def addStudent(self, name):
- self._students[name] = Student()
-
- def addExam(self, exam, max_score):
- self._exams.append(exam)
-
- for key in self._students.keys():
- self._students[key].addExam(exam, Exam(max_score))
-
- def addResult(self, name, exam, score):
- self._students[name].addResult(exam, score)
-
- def grades(self):
- g = {}
- for name in self._students.keys():
- g[name] = self._students[name].grades()
- return g
-students = ["Charlie", "James", "Matt"]
-exams = { "maths" : 20, "physics" : 50, "english literature" : 30 }
-results = { "maths" : { "James" : 20, "Matt" : 18, "Charlie" : 4 },
- "physics" : { "Matt" : 48, "James" : 3 },
- "english literature" : { "Charlie" : 10, "Matt" : 25, "James" : 3 } }
-school = School()
-for student in students:
- school.addStudent(student)
-for exam in exams.keys():
- school.addExam(exam, exams[exam])
-for exam in results:
- add_results(school, exam, results[exam])
-school.grades()
-
-Documentation¶
Python has great in-built documentation that is available via the help function. For example
l = ["cat", "dog", "fish"]
-help(l)
-You can add similar documentation to the functions that you write. You do this by adding in a documentation string as the first string after defining the function e.g.
- -def multiply(a, b):
- """This function returns the element-wise multiplication of the passed lists 'a' and 'b'"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-multiply( [1,2,3], [4,5,6] )
-help(multiply)
-The documentation string should be placed between two sets of triple quotes ("""). This is a convention that makes it easier to expand the documentation later, and that ensures that nothing you write in the documentation will be expanded or interpreted as Python.
Documentation should provide an easy to understand, and brief description of what the function does. It should not give information that is obvious by reading the function signature. For example, this is a bad piece of documentation.
- -def multiply(a, b):
- """function multiply(a,b) -> list"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-help(multiply)
-It is much better to say what the function does, and then what it returns (as this can't be seen from the signature). Good documentation would be
- -def multiply(a, b):
- """Calculates the element-wise multiplication of a and b, returning a list of the results"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-help(multiply)
-Your documentation can span over multiple lines. If you are describing the arguments, then you should use one line per argument, for example
- -def make_complex(real, imag=0):
- """Create and return a complex number
-
- Keyword arguments:
-
- real -- the real part of the number
- imag -- the imaginary part of the number
- """
- return (real,imag)
-help(make_complex)
-By convention, you will notice above that the last """ is placed on its own line if the documentation spans multiple lines. It is on the same line if the documentation is short.
In general, keep your documentation short, to the point, and avoid repeating obvious information. However, be precise, as this may be the only part of your code that somebody else reads before they use your function in their program.
-A good suggestion is to look at documentation you like and try to copy that style. Also, look for code that you think is poorly documented, and try to avoid their mistakes.
- -Exercise¶
Below is a series of undocumented functions. Take a look through the functions and try to work out what they do. Once you understand the functions, write some documentation for each function. Get your neighbour to read your documentation. Do they understand what the function does based on what you have written? Do the function names -combined with your documentation accurately convey the result of calling the function?
-Note that you may have to use help(...) yourself if you don't recognise some of the code in the functions. Also try to play with the function to see how it behaves.
def add(a, b):
- c = []
- for x,y in zip(a,b):
- c.append(x+y)
- return c
-
-def subtract(a, b):
- c = []
- for x,y in zip(a,b):
- c.append(x / y)
- return c
-
-def capitalise(message):
- words = message.split(" ")
- for i in range(0,len(words)):
- words[i] = "%s%s" % (words[i][0].upper(), words[i][1:])
- return " ".join(words)
-
-def surprise(x):
- import random
- if x < random.random():
- print("Surprise!")
-
-For this last function, try calling it via list_interface("ipynb").
def list_interface(x):
- import glob
- f = glob.glob("*.%s" % x)
- l = []
- for x in f:
- if x.startswith("0"):
- l.append(x)
- return l
-
-Documenting Classes¶
It is almost as easy to document a class as it is to document a function. Simply add docstrings to all of the classes functions, and also below the class name itself. For example, here is a simple documented class
- -class Demo:
- """This class demonstrates how to document a class.
-
- This class is just a demonstration, and does nothing.
-
- However the principles of documentation are still valid!
- """
-
- def __init__(self, name):
- """You should document the constructor, saying what it expects to
- create a valid class. In this case
-
- name -- the name of an object of this class
- """
- self._name = name
-
- def getName(self):
- """You should then document all of the member functions, just as
- you do for normal functions. In this case, returns
- the name of the object
- """
- return self._name
-d = Demo("cat")
-help(d)
-Often, when you write a class, you want to hide member data or member functions so that they are only visible within an object of the class. For example, above, the self._name member data should be hidden, as it should only be used by the object.
You control the visibility of member functions or member data using an underscore. If the member function or member data name starts with an underscore, then it is hidden. Otherwise, the member data or function is visible.
-For example, we can hide the getName function by renaming it to _getName
class Demo:
- """This class demonstrates how to document a class.
-
- This class is just a demonstration, and does nothing.
-
- However the principles of documentation are still valid!
- """
-
- def __init__(self, name):
- """You should document the constructor, saying what it expects to
- create a valid class. In this case
-
- name -- the name of an object of this class
- """
- self._name = name
-
- def _getName(self):
- """You should then document all of the member functions, just as
- you do for normal functions. In this case, returns
- the name of the object
- """
- return self._name
-d = Demo("cat")
-help(d)
-Member functions or data that are hidden are called "private". Member functions or data that are visible are called "public". You should document all public member functions of a class, as these are visible and designed to be used by other people. It is helpful, although not required, to document all of the private member functions of a class, as these will only really be called by you. However, in years to come, you will thank yourself if you still documented them... ;-)
-While it is possible to make member data public, it is not advised. It is much better to get and set values of member data using public member functions. This makes it easier for you to add checks to ensure that the data is consistent and being used in the right way. For example, compare these two classes that represent a person, and hold their height.
- -class Person1:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self.height = 0
-class Person2:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self._height = 0
-
- def setHeight(self, height):
- """Set the person's height to 'height', returning whether or
- not the height was set successfully
- """
- if height < 0 or height > 300:
- print("This is an invalid height! %s" % height)
- return False
- else:
- self._height = height
- return True
-
- def getHeight(self):
- """Return the person's height"""
- return self._height
-The first example is quicker to write, but it does little to protect itself against a user who attempts to use the class badly.
- -p = Person1()
-p.height = -50
-p.height
-p.height = "cat"
-p.height
-The second example takes more lines of code, but these lines are valuable as they check that the user is using the class correctly. These checks, when combined with good documentation, ensure that your classes can be safely used by others, and that incorrect use will not create difficult-to-find bugs.
- -p = Person2()
-p.setHeight(-50)
-p.getHeight()
-p.setHeight("cat")
-p.getHeight()
-class GuessGame:
- def __init__(self, secret, max_guesses=5):
- self._secret = secret
- self._nguesses = 0
- self._max_guesses = max_guesses
-
- def guess(self, value):
- if (self.nGuesses() >= self.maxGuesses()):
- print("Sorry, you have run out of guesses")
- elif (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- self._nguesses += 1
- print("Try again...")
-
- def nGuesses(self):
- return self._nguesses
-
- def maxGuesses(self):
- return self._max_guesses
-Exercise 2¶
Below is a poorly-written class that uses public member data to store the name and age of a Person. Edit this class so that the member data is made private. Add get and set functions that allow you to safely get and set the name and age.
class Person:
- """Class the represents a Person, holding their name and age"""
- def __init__(self, name="unknown", age=0):
- """Construct a person with unknown name and an age of 0"""
- self.name = name
- self.age = age
-Exercise 3¶
Add a private member function called _splitName to your Person class that breaks the name into a surname and first name. Add new functions called getFirstName and getSurname that use this function to return the first name and surname of the person.
-Exceptions¶
Mistakes and errors happen in computer programs as much as in real life. Like life, how you handle an error in your program shows your level of professionalism, and gives others evidence that they can trust that you have written a program that will work well.
-In the last section we indicated errors in the Person.setHeight function by printing a message to the screen and returning False to indicate that the call to setHeight had failed.
class Person:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self._height = 0
-
- def setHeight(self, height):
- """Set the person's height to 'height', returning whether or
- not the height was set successfully
- """
- if height < 0 or height > 300:
- print("This is an invalid height! %s" % height)
- return False
- else:
- self._height = height
- return True
-
- def getHeight(self):
- """Return the person's height"""
- return self._height
-p = Person()
-p.setHeight(-20)
-This is not a good way of indicating an error. The issues with this are;
--
-
- How does the person calling
getHeightknow to check whether the call returnsTrueorFalse
- - What if we wanted to return something else? Should we return the error state and the value we want together? -
- If the error state is not checked, and nobody reads the error message printed to the screen, then the program is broken, as the person has been created with a height of 0. -
The solution is to send something to the programmer that they cannot ignore, which indicates that there is an error. That something is called an "exception".
-Take a look at this simple code that sets the height...
- -def setHeight(height):
- if height < 0 or height > 300:
- raise ValueError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(-5)
-When we try to use an invalid value for the height, we raise (or throw) a ValueError exception. This stops the function from continuing, and gives us a very helpful print out of what went wrong, and where.
ValueError is just a class. The name of the class provides us with useful information (there was an error with a value in the program). You choose what error you want to raise. Python provides a set of usefully named error classes that you can use:
-
-
IOError: Error raised when you have a problem with IO, e.g. opening or closing files
-ZeroDivisionError: Error raised when you divide by zero
-TypeError: Error raised when you are using the wrong type, e.g. maybe setting the height to a string
-IndexError: Error raised when you are using an invalid index to access a list or other similar container
-KeyError: Error raised when you are using an invalid key to access a dictionary or other similar container
-
A full list of standard Python exceptions is available here.
-You are free to raise any exception class you want. It is your job as a programmer to choose the one that is most sensible, e.g.
- -def setHeight(height):
- if height < 0 or height > 300:
- raise ZeroDivisionError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(400)
-Using a ZeroDivisionError is a bad choice, as the error has nothing to do with division by zero. A ValueError is the right choice as the error relates to an invalid value passed to the function.
You are free to create your own exception classes.
- -class InvalidHeightError(Exception):
- pass
-def setHeight(height):
- if height < 0 or height > 300:
- raise InvalidHeightError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(-10)
-Your own exception classes must be declared as derived from type Exception, hence why you have to write class InvalidHeightError(Exception):. As the class doesn't need to do anything else, you can use pass to say that nothing else needs to be added. Note that you can call your error class anything you want. By convention, it is good to end the class name with Error so that other programmers know what it is for.
Exercise¶
Here is an extended copy of the Person code from above.
class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- return self.getWeight() / self.getHeight()**2
-Exercise 1¶
Edit the above copy of Person to ensure that the .setWeight function only accepts valid weights. A valid weight is any number that is between 0 and 500 kilograms. You should raise a ValueError if the weight is outside this range. For the moment, do not worry about the user supplying a non-numeric weight.
Also edit the above copy of Person to ensure that the .setHeight function only accepts valid heights. A valid height is any number that is between 0 and 2.5 meters. You should raise a ValueError if the height is outside this range. For the moment, do not worry about the user supplying a non-numeric height.
Check that a ValueError exception is correctly raised if invalid heights or weights are supplied. Also check that the ValueError exception is not raised if a valid height and weight are supplied.
-Exercise 2¶
If you run the following code;
-p = Person()
-p.bmi()
-it will raise a DivideByZero exception. This is because the calculation involves dividing by the height squared, which is zero in a default-constructed Person. While an exception has been raised, it is not very intuitive for another programmer to debug. A solution is to create your own named exception that provides more information.
Create a new exception called NullPersonError, and edit the .bmi() function so that this exception is raised if it is called on a Person whose height or weight is zero.
Check that the NullPersonError exception is raised if .bmi() is called on a default-constructed Person. Check that this exception is not raised if .bmi() is called on a properly constructed Person.
-Error Handling¶
Exceptions are useful for more than just signalling errors. They can also be used to help you handle the error, and potentially even fix the problem (true self-healing program!).
-Consider this cut down version of the .setHeight function from the last session...
def setHeight(height):
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-The code currently correctly detects if the user supplies a height that is below 0 or above 2.5. However, what about when the user tries to set the height to something that is not a number?
- -setHeight("cat")
-We get a weird error message that says we have a TypeError, as you cannot order a string and an integer.
One way to address this is to ask that height is converted to a float, using height = float(height)
def setHeight(height):
- height = float(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-However, this hasn't made the error any easier to understand, as we now get a ValueError raised...
setHeight("cat")
-The solution is for us to handle the exception, using a try...except block
def setHeight(height):
- try:
- height = float(height)
- except:
- raise TypeError("Invalid height: '%s'. You can only set the height to a numeric value" % height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-setHeight("cat")
-What's happened here? The try: line starts a try-block. The code that is in the try-block is run. If any of this code raises an exception, then execution stops in the try-block, and switches instead to the code in the except-block (everything within the except: block). In our case, float(height) raised an exception, so execution jumped to the except-block, in which we ran the raise TypeError(...) code.
Now the error is much more informative, allowing the user to better understand what has gone wrong. However, exception handling can do more than this. It can allow you to fix the problem. Consider this example...
- -setHeight("1.8 m")
-We as humans can see that this could be an acceptable input. However, the computer needs help to understand. We can add code to the except-block that can try to resolve the problem. For example, imagine we had a function that could interpret heights from strings...
- -def string_to_height(height):
- """This function tries to interpret the passed argument as a height
- in meters. The format should be 'X m', 'X meter' or 'X meters',
- where 'X' is a number
- """
- # convert height to a string - this always works
- height = str(height)
-
- words = height.split(" ")
-
- if len(words) == 2:
- if words[1] == "m" or words[1] == "meter" or words[1] == "meters":
- try:
- return float(words[0])
- except:
- pass
-
- # Getting here means that we haven't been able to extract a valid height
- raise TypeError("Cannot extract a valid height from '%s'" % height)
-We can now call this function from within the except-block of setHeight
def setHeight(height):
- try:
- height = float(height)
- except:
- height = string_to_height(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-setHeight("1.8 m")
-Exercise¶
Exercise 1¶
Here is a copy of the Person class from the last session. Edit the setHeight function so that it uses exception handling and the string_to_height function to correctly interpret heights such as "1.8 m", and so that it gives a useful error message if it is given something weird. Check that the function correctly responds to a range of valid and invalid inputs.
class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This shoud be between 0 and 2.5 meters" % height)
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- if weight < 0 or weight > 500:
- raise ValueError("Invalid weight: %s. This should be between 0 and 500 kilograms" % weight)
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- if (self.getHeight() == 0 or self.getWeight() == 0):
- raise NullPersonError("Cannot calculate the BMI of a person with zero "
- "height or weight (%s,%s)" % (self.getHeight(),self.getWeight()))
-
- return self.getWeight() / self.getHeight()**2
-
-Exercise 2¶
Create a string_to_weight function that interprets weights in kilograms (e.g. "5 kg", "5 kilos" or "5 kilograms"). Now edit the Person.setWeight function so that it uses exception handling and string_to_weight to to correctly interpret weights such as 35.5 kg and gives a useful error message if it is given something weird. Check that your function responds correctly to a range of valid and invalid inputs.
-Modules¶
You can turn any Python script that you write into a module that other people can import and use in their own code.
-For example;
- -import superhero
-What has happened here???
-There is a file in your current directory called superhero.py. The line import superhero will look in the current directory, to find a file called superhero.py. It then runs this file, just as if you had typed it into the screen.
This is just a simple Python script, which we can print out using
- -! cat superhero.py
-We can get help on the module using help
help(superhero)
-This documentation comes from the class and function documentation put into the file.
-You can also use the data, classes and functions in the file, e.g.
- -ironman = superhero.Superhero(name="Iron Man", weakness="rust")
-superhero.battle(ironman, superhero.lex)
-superhero.lex.steal("rust")
-superhero.battle(ironman, superhero.lex)
-One thing to note is that all of the classes, functions and data in the script has been imported into its own namespace, named after the script (e.g. superhero.). We can import the file and put all names into the current namespace using
from superhero import *
-battle(ironman, lex)
-While any python script can be imported as a module, there are a few conventions you should follow that will make your module easier for others to use.
--
-
- Add documentation to the module. As you can see, there is a docstring at the top of
superhero.py, which is the first thing written out by help(). This should provide an overview of the module.
- - Avoid actually running any code or creating any variables. The current
superhero.pyis bad as it does this, which is why you see "Is it a bird..." printed when you import it!
-
The way to avoid creating any variables or running code is to let the script detect when it is being imported, and to not create any variables if that is the case.
-You can detect if your Python script is not being imported using
-if __name__ == "__main__":
- print("I am not being imported.")
-if __name__ == "__main__":
- print("I am not being imported")
-To show how this works, there is a superhero2.py script, which is identical to superhero.py, except all code that should not be run on import is hidden inside the if __name__ == "__main__": block.
! cat superhero2.py
-import superhero2
-By using if __name__ == "__main__": we have prevented superhero2.py from printing anything out when it is imported, and have also prevented it from creating the variables lex and superman.
You can see this by running the superhero2.py script directory, e.g. using
! python superhero2.py! python superhero2.py
-Exercise¶
Exercise 1¶
Use the "New Text File" option in the Jupyter Home to create a new python text file called morse.py. Copy the below class into this file.
class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- self._morse_to_letter = {}
- for letter in self._letter_to_morse.keys():
- self._morse_to_letter[ self._letter_to_morse[letter] ] = letter
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-
- def decode(self, morse):
- message = []
- for code in morse:
- message.append( self._morse_to_letter[code] )
- return "".join(message)
-Add documentation to this class, and to the module. Next, import the module and get help using the commands
-import morse
-help(morse)
-Does your documentation make sense?
- -
-Exercise 2¶
Create some checks of your module that should not be run when the module is imported (i.e. only run directly). The checks should be, e.g.
-morse = Morse()
-
- for message in ["Hello world", "something to encode", "test message"]:
- test = morse.decode( morse.encode(message) )
-
- if message.lower() == test:
- print("Success: %s" % message)
- else:
- print("Failed: %s" % message)
-Validate that the check doesn't run on import using
-import morse
-Validate that the check runs from the command line using
- -! python morse.py
-Pandas¶
Pandas is a library providing high-performance, easy-to-use data structures and data analysis tools. The core of pandas is its dataframe which is essentially a table of data. Pandas provides easy and powerful ways to import data from a variety of sources and export it to just as many. It is also explicitly designed to handle missing data elegantly which is a very common problem in data from the real world.
-The offical pandas documentation is very comprehensive and you will be answer a lot of questions in there, however, it can sometimes be hard to find the right page. Don't be afraid to use Google to find help.
- -Pandas has a standard convention for importing it which you will see used in a lot of documentation so we will follow that in this course:
- -import pandas as pd
-from pandas import Series, DataFrame
-Series¶
The simplest of pandas' data structures is the Series. It is a one-dimensional list-like structure.
-Let's create one from a list:
Series([14, 7, 3, -7, 8])
-There are three main components to this output.
-The first column (0, 2, etc.) is the index, by default this is numbers each row starting from zero.
-The second column is our data, stored i the same order we entered it in our list.
-Finally at the bottom there is the dtype which stands for 'data type' which is telling us that all our data is being stored as a 64-bit integer.
-Usually you can ignore the dtype until you start doing more advanced things.
In the first example above we allowed pandas to automatically create an index for our Series (this is the 0, 1, 2, etc. in the left column) but often you will want to specify one yourself
s = Series([14, 7, 3, -7, 8], index=['a', 'b', 'c', 'd', 'e'])
-print(s)
-We can use this index to retrieve individual rows
- -s['a']
-to replace values in the series
- -s['c'] = -1
-or to get a set of rows
- -s[['a', 'c', 'd']]
-Exercise 1¶
-
-
- Create a Pandas
Serieswith 10 or so elements where the indices are years and the values are numbers.
- - Experiment with retrieving elements from the
Series.
- - Try making another
Serieswith duplicate values in the index, what happens when you access those elements?
- - How does a Pandas
Seriesdiffer from a Pythonlistordict?
-
-Series operations¶
A Series is list-like in the sense that it is an ordered set of values. It is also dict-like since its entries can be accessed via key lookup. One very important way in which is differs is how it allows operations to be done over the whole Series in one go, a technique often referred to as 'broadcasting'.
A simple example is wanting to double the value of every entry in a set of data. In standard Python, you might have a list like
- -my_list = [3, 6, 8, 4, 10]
-If you wanted to double every entry you might try simply multiplying the list by 2:
my_list * 2
-but as you can see, that simply duplicated the elements. Instead you would have to use a for loop or a list comprehension:
[i * 2 for i in my_list]
-With a pandas Series, however, you can perform bulk mathematical operations to the whole series in one go:
my_series = Series(my_list)
-print(my_series)
-my_series * 2
-As well as bulk modifications, you can perform bulk selections by putting more complex statements in the square brackets:
- -s[s < 0] # All negative entries
-s[(s * 2) > 4] # All entries which, when doubled are greater than 4
-These operations work because the Series index selection can be passed a series of True and False values which it then uses to filter the result:
(s * 2) > 4
-Here you can see that the rows a, b and e are True while the others are False. Passing this to s[...] will only show rows that are True.
Multi-Series operations¶
It is also possible to perform operations between two Series objects:
s2 = Series([23,5,34,7,5])
-s3 = Series([7, 6, 5,4,3])
-s2 - s3
-Exercise 2¶
-
-
- Create two
Seriesobjects of equal length with no specified index and containing any values you like. Perform some mathematical operations on them and experiment to make sure it works how you think.
- - What happens then you perform an operation on two series which have different lengths? How does this change when you give the series some indices? -
- Using the
Seriesfrom the first exercise with the years for the index, Select all entries with even-numbered years. Also, select all those with odd-numbered years.
-
-DataFrame¶
While you can think of the Series as a one-dimensional list of data, pandas' DataFrame is a two (or possibly more) dimensional table of data. You can think of each column in the table as being a Series.
data = {'city': ['Paris', 'Paris', 'Paris', 'Paris',
- 'London', 'London', 'London', 'London',
- 'Rome', 'Rome', 'Rome', 'Rome'],
- 'year': [2001, 2008, 2009, 2010,
- 2001, 2006, 2011, 2015,
- 2001, 2006, 2009, 2012],
- 'pop': [2.148, 2.211, 2.234, 2.244,
- 7.322, 7.657, 8.174, 8.615,
- 2.547, 2.627, 2.734, 2.627]}
-df = DataFrame(data)
-This has created a DataFrame from the dictionary data. The keys will become the column headers and the values will be the values in each column. As with the Series, an index will be created automatically.
df
-Or, if you just want a peek at the data, you can just grab the first few rows with:
- -df.head(3)
-Since we passed in a dictionary to the DataFrame constructor, the order of the columns will not necessarilly match the order in which you defined them. To enforce a certain order, you can pass a columns argument to the constructor giving a list of the columns in the order you want them:
DataFrame(data, columns=['year', 'city', 'pop'])
-When we accessed elements from a Series object, it would select an element by row. However, by default DataFrames index primarily by column. You can access any column directly by using square brackets or by named attributes:
df['year']
-df.city
-Accessing a column like this returns a Series which will act in the same way as those we were using earlier.
Note that there is one additional part to this output, Name: city. Pandas has remembered that this Series was created from the 'city' column in the DataFrame.
type(df.city)
-df.city == 'Paris'
-This has created a new Series which has True set where the city is Paris and False elsewhere.
We can use filtered Series like this to filter the DataFrame as a whole. df.city == 'Paris' has returned a Series containing booleans. Passing it back into df as an indexing operation will use it to filter based on the 'city' column.
df[df.city == 'Paris']
-You can then carry on and grab another column after that filter:
- -df[df.city == 'Paris'].year
-If you want to select a row from a DataFrame then you can use the .loc attribute which allows you to pass index values like:
df.loc[2]
-df.loc[2]['city']
-Adding new columns¶
New columns can be added to a DataFrame simply by assigning them by index (as you would for a Python dict) and can be deleted with the del keyword in the same way:
df['continental'] = df.city != 'London'
-df
-del df['continental']
-Exercise 3¶
-
-
- Create the
DataFramecontaining the census data for the three cities.
- - Select the data for the year 2001. Which city had the smallest population that year? -
- Find all the cities which had a population smaller than 2.6 million. -
-Reading from file¶
One of the msot common situations is that you have some data file containing the data you want to read. Perhaps this is data you've produced yourself or maybe it's from a collegue. In an ideal world the file will be perfectly formatted and will be trivial to import into pandas but since this is so often not the case, it provides a number of features to make your ife easier.
-Full information on reading and writing is available in the pandas manual on IO tools but first it's worth noting the common formats that pandas can work with:
--
-
- Comma separated tables (or tab-separated or space-separated etc.) -
- Excel spreadsheets -
- HDF5 files -
- SQL databases -
For this course we will focus on plain-text CSV files as they are perhaps the most common format. Imagine we have a CSV file like (you can download this file from city_pop.csv):
- -! cat data/city_pop.csv # Uses the IPython 'magic' !cat to print the file
-We can use the pandas function read_csv() to read the file and convert it to a DataFrame. Full documentation for this function can be found in the manual or, as with any Python object, directly in the notebook by putting a ? after the name:
help(pd.read_csv)
-pd.read_csv('data/city_pop.csv')
-We can see that by default it's done a fairly bad job of parsing the file (this is mostly because I;ve construsted the city_pop.csv file to be as obtuse as possible). It's making a lot of assumptions about the structure of the file but in general it's taking quite a naïve approach.
The first this we notice is that it's treating the text at the top of the file as though it's data. Checking the documentation we see that the simplest way to solve this is to use the skiprows argument to the function to which we give an integer giving the number of rows to skip:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
-)
-The next most obvious problem is that it is not separating the columns at all. This is controlled by the sep argument which is set to ',' by default (hence comma separated values). We can simply set it to the appropriate semi-colon:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';'
-)
-Reading the descriptive header of our data file we see that a value of -1 signifies a missing reading so we should mark those too. This can be done after the fact but it is simplest to do it at import-time using the na_values argument:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';',
- na_values='-1'
-)
-The last this we want to do is use the year column as the index for the DataFrame. This can be done by passing the name of the column to the index_col argument:
df3 = pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';',
- na_values='-1',
- index_col='year'
-)
-df3
-Exercise 4¶
-
-
- Alongside
data/city_pop.csvthere is another file calleddata/cetml1659on.dat. This contains some historical weather data for a location in the UK. Import that file as a PandasDataFrameusingread_csv(), making sure that you cover all the NaN values.
- - How many years had a negative average temperature in January? -
- What was the average temperature in June over the years in the data set? Tip: look in the documentation for which method to call. -
We will come back to this data set in a later stage.
- -
-Basic NumPy¶
NumPy ('Numerical Python') is the defacto standard module for doing numerical work in Python. Its main feature is its array data type which allows very compact and efficient storage of homogenous (of the same type) data.
-A lot of the material in this section is based on SciPy Lecture Notes (CC-by 4.0).
-As you go through this material, you'll likely find it useful to refer to the NumPy documentation, particularly the array objects section.
-As with pandas there is a standard convention for importing numpy, and that is as np:
import numpy as np
-Now that we have access to the numpy package we can start using its features.
Creating arrays¶
In many ways a NumPy array can be treated like a standard Python list and much of the way you interact with it is identical. Given a list, you can create an array as follows:
python_list = [1, 2, 3, 4, 5, 6, 7, 8]
-numpy_array = np.array(python_list)
-print(numpy_array)
-# ndim give the number of dimensions
-numpy_array.ndim
-# the shape of an array is a tuple of its length in each dimension. In this case it is only 1-dimensional
-numpy_array.shape
-# as in standard Python, len() gives a sensible answer
-len(numpy_array)
-nested_list = [[1, 2, 3], [4, 5, 6]]
-two_dim_array = np.array(nested_list)
-print(two_dim_array)
-two_dim_array.ndim
-two_dim_array.shape
-It's very common when working with data to not have it already in a Python list but rather to want to create some data from scratch. numpy comes with a whole suite of functions for creating arrays. We will now run through some of the most commonly used.
The first is np.arange (meaning "array range") which works in a vary similar fashion the the standard Python range() function, including how it defaults to starting from zero, doesn't include the number at the top of the range and how it allows you to specify a 'step:
np.arange(10) #0 .. n-1 (!)
-np.arange(1, 9, 2) # start, end (exclusive), step
-Next up is the np.linspace (meaning "linear space") which generates a given floating point numbers starting from the first argument up to the second argument. The third argument defines how many numbers to create:
np.linspace(0, 1, 6) # start, end, num-points
-Note how it included the end point unlike arange(). You can change this feature by using the endpoint argument:
np.linspace(0, 1, 5, endpoint=False)
-np.ones creates an n-dimensional array filled with the value 1.0. The argument you give to the function defines the shape of the array:
np.ones((3, 3)) # reminder: (3, 3) is a tuple
-Likewise, you can create an array of any size filled with zeros:
- -np.zeros((2, 2))
-The np.eye (referring to the matematical identity matrix, commonly labelled as I) creates a square matrix of a given size with 1.0 on the diagonal and 0.0 elsewhere:
np.eye(3)
-The np.diag creates a square matrix with the given values on the diagonal and 0.0 elsewhere:
np.diag([1, 2, 3, 4])
-Finally, you can fill an array with random numbers:
- -np.random.rand(4) # uniform in [0, 1]
-np.random.randn(4) # Gaussian
-Exercise 5¶
-
-
- Experiment with
arange,linspace,ones,zeros,eyeanddiag.
- - Create different kinds of arrays with random numbers. -
- Look at the function
np.empty. What does it do? When might this be useful?
-
Reshaping arrays¶
Behind the scenes, a multi-dimensional NumPy array is just stored as a linear segment of memory. The fact that it is presented as having more than one dimension is simply a layer on top of that (sometimes called a view). This means that we can simply change that interpretive layer and change the shape of an array very quickly (i.e without NumPy having to copy any data around).
This is mostly done with the reshape() method on the array object:
my_array = np.arange(16)
-my_array
-my_array.shape
-my_array.reshape((2, 8))
-my_array.reshape((4, 4))
-Note that if you check, my_array.shape will still return (16,) as reshaped is simply a view on the original data, it hasn't actually changed it. If you want to edit the original object in-place then you can use the resize() method.
You can also transpose an array using the transpose() method which mirrors the array along its diagonal:
my_array.reshape((2, 8)).transpose()
-my_array.reshape((4,4)).transpose()
-Exercise 6¶
Using the NumPy documentation at https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html, to create, in one line a NumPy array which looks like:
-[10, 60, 20, 70, 30, 80, 40, 90, 50, 100]
-Hint: you will need to use transpose(), reshape() and arange() as well as one new function from the "Shape manipulation" section of the documentation. Can you find a method which uses less than 4 function calls?
-There is still more numpy to learn
- -More Numpy¶
Carrying on from the last lesson we will continue learning how to manipulate data in numpy before using matplotlib to plot our data.
import numpy as np
-Basic data types¶
You may have noticed that, in some instances, array elements are displayed with a trailing dot (e.g. 2. vs 2). This is due to a difference in the data-type used:
a = np.array([1, 2, 3])
-a.dtype
-b = np.array([1., 2., 3.])
-b.dtype
-Different data-types allow us to store data more compactly in memory, but most of the time we simply work with floating point numbers. Note that, in the example above, NumPy auto-detects the data-type from the input but you can specify it explicitly:
- -c = np.array([1, 2, 3], dtype=float)
-c.dtype
-The default data type is floating point.
- -d = np.ones((3, 3))
-d.dtype
-There are other data types as well:
- -e = np.array([1+2j, 3+4j, 5+6*1j])
-type(1j)
-#e.dtype
-f = np.array([True, False, False, True])
-f.dtype
-g = np.array(['Bonjour', 'Hello', 'Hallo',])
-g.dtype # <--- strings containing max. 7 letters
-We previously came across dtypes when learing about pandas. This is because pandas uses NumPy as its underlying library. A pandas.Series is essentially a np.array with some extra features wrapped around it.
Exercise 1¶
Recreate some of the arrays we created in yesterday's session and look at what dtype they have.
- -Why NumPy¶
To show some of the advantages of NumPy over a standard Python list, let's do some benchmarking. It's an important habit in programming that whenever you think one method may be faster than another, you check to see whether your assumption is true.
-Python provides some tools to make this easier, particularly via the timeit module. Using this functionality, IPython provides a %timeit magic function to make our life easier. To use the %timeit magic, simply put it at the beginning of a line and it will give you information about how ling it took to run. It doesn't always work as you would expect so to make your life easier, put whatever code you want to benchmark inside a function and time that function call.
We start by making a list and an array of 10000 items each of values counting from 0 to 9999:
- -python_list = list(range(100000))
-numpy_array = np.arange(100000)
-We are going to go through each item in the list and double its value in-place, such that the list is changed after the operation. To do this with a Python list we need a for loop:
def python_double(a):
- for i, val in enumerate(a):
- a[i] = val * 2
-
-%timeit python_double(python_list)
-To do the same operation in NumPy we can use the fact that multiplying a NumPy array by a value will apply that operation to each of its elements:
def numpy_double(a):
- a *= 2
-
-%timeit numpy_double(numpy_array)
-As you can see, the NumPy version is at least 10 times faster, sometimes up to 100 times faster.
-Have a think about why this might be, what is NumPy doing to make this so much faster? There are two main parts to the answer.
- -Copies and views¶
A slicing operation (like reshaping before) creates a view on the original array, which is just a way of accessing array data. Thus the original array is not copied in memory. This means you can do this to large arrays without any great performance hit. You can use np.may_share_memory() to check if two arrays share the same memory block. Note however, that this uses heuristics and may give you false positives.
When modifying the view, the original array is modified as well:
- -a = np.arange(10)
-a
-b = a[3:7]
-
-np.may_share_memory(a, b)
-b[0] = 12
-b
-a # (!)
-a = np.arange(10)
-c = a[::2].copy() # force a copy
-c[0] = 12
-a
-np.may_share_memory(a, c) # we made a copy so there is no shared memory
-Whether you make a view or a copy can affect the speed of your code significantly. Be in the habit of checking whether your code is doing unnecessacy work. Also, be sure to benchmark your code as you work on it so that you notice any slowdowns and so that you know which parts are slow so you speed the right bits up.
- -Exercise 2¶
-
-
- Using
%timeit, time how long finding the square roots of a list of numbers would take under both standard Python and numpy.-
-
- Tip: Python's square root function is
math.sqrt. numpy's isnp.sqrt.
-
- - Tip: Python's square root function is
-Plotting data with matplotlib¶
Plotting of data is pandas is handled by an external Python module called matplotlib. Like pandas it is a large library and has a venerable history (first released in 2003) and so we couldn't hope to cover all its functionality in this course. To see the wide range of possibilities you have with matplotlib see its example gallery.
-Here we will cover the basic uses of it and how it integrates with pandas. While working through these examples you will likely find it very useful to refer to the matplotlib documentation.
- -First we import pandas and numpy in the same way as we did previously.
- -import numpy as np
-import pandas as pd
-from pandas import Series, DataFrame
-Some matplotlib functionality is provided directly through pandas (such as the plot() method as we will see) but for much of it you need to import the matplotlib interface itself.
The most common interface to matplotlib is its pyplot module which provides a way to affect the current state of matplotlib directly. By convention this is imported as plt.
We also set the figure format to be SVG so that the plots look a little nicer in our Jupyter notebook.
- -import matplotlib.pyplot as plt
-%config InlineBackend.figure_format = 'svg'
-Once we have imported matplotlib we can start calling its functions. Any functions called on the plt object will affect all of matplotlib from that point on in the script.
We first need to import some data to plot. Let's start with the data from the pandas section (available from cetml1659on.dat) and import it into a DataFrame:
df = pd.read_csv(
- 'data/cetml1659on.dat', # file name
- skiprows=6, # skip header
- sep='\s+', # whitespace separated
- na_values=['-99.9', '-99.99'], # NaNs
-)
-df.head()
-Pandas integrates matplotlib directly into itself so any dataframe can be plotted easily simply by calling the plot() method on one of the columns. This creates a plot object which you can then edit and alter, for example by setting the axis labels using the plt.ylabel() function before displaying it with plt.show().
Matplotlib operates on a single global state and calling any function on plt will alter that state. Calling df.plot() sets the currently operating plot. plt.ylabel() then alters that state and plt.show() displays it.
df['JAN'].plot()
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-Exercise 1¶
-
-
- Make sure you can reproduce the plot above. Try tweaking the labels or which column is plotted. -
- Try putting in two
plot()calls with different months (January and July for example) before callingshow().
-
-Making it prettier¶
-While it's useful to be able to quickly plot any data we have in front of us, matplotlib's power comes from its configurability. Let's experiment with a dataset and see how much we can change the plot.
-We'll start with a simple DataFrame contianing two columns, one with the values of a cosine, the other with the values of a sine.
X = np.linspace(-np.pi, np.pi, 256, endpoint=True)
-data = {'cos': np.cos(X), 'sin': np.sin(X)}
-trig = DataFrame(index=X, data=data)
-
-trig.plot()
-plt.show()
-You can see that it has plotted the sine and cosine curves between $\pi$ and $-\pi$. Now, let's go through and see how we can affect the display of this plot.
- -Changing colours and line widths¶
First step, we want to have the cosine in blue and the sine in red and a slighty thicker line for both of them.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.show()
-Exercise 2¶
-
-
- Using the temperature dataset, set the colours of the July and January lines to a warm colour and a cool colour. -
- Add in the yearly average column to the plot with a dashed line style. -
-Setting limits¶
Current limits of the figure are a bit too tight and we want to make some space in order to clearly see all data points.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-### New code
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-### End of new code
-
-plt.show()
-Setting ticks¶
Current ticks are not ideal because they do not show the interesting values ($\pm\pi$,$\pm\frac{\pi}{2}$) for sine and cosine. We’ll change them such that they show only these values.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-### New code
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-### End of new code
-
-plt.show()
-Setting tick labels¶
Ticks are now properly placed but their label is not very explicit. We could guess that 3.142 is $\pi$ but it would be better to make it explicit. When we set tick values, we can also provide a corresponding label in the second argument list. Note that we’ll use LaTeX to allow for nice rendering of the label.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-### New code
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-### End of new code
-
-plt.show()
-Moving spines¶
Spines are the lines connecting the axis tick marks and noting the boundaries of the data area. They can be placed at arbitrary positions and until now, they were on the border of the axis. We’ll change that since we want to have them in the middle. Since there are four of them (top/bottom/left/right), we’ll discard the top and right by setting their color to none and we’ll move the bottom and left ones to coordinate 0 in data space coordinates.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-### New code
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-### End of new code
-
-plt.show()
-Adding a legend¶
Let’s add a legend in the upper left corner. This only requires adding the keyword argument label (that will be used in the legend box) to the plot commands.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-
-### New code
-plt.legend(loc='upper left')
-### End of new code
-
-plt.show()
-Annotate some points¶
Let’s annotate some interesting points using the annotate command. We chose the $\frac{2}{3}\pi$ value and we want to annotate both the sine and the cosine. We’ll first draw a marker on the curve as well as a straight dotted line. Then, we’ll use the annotate command to display some text with an arrow.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-
-plt.legend(loc='upper left')
-
-### New code
-t = 2 * np.pi / 3
-plt.plot([t, t], [0, np.cos(t)], color='blue', linewidth=2.5, linestyle="--")
-plt.scatter([t, ], [np.cos(t), ], 50, color='blue')
-
-plt.annotate(r'$cos(\frac{2\pi}{3})=-\frac{1}{2}$',
- xy=(t, np.cos(t)), xycoords='data',
- xytext=(-90, -50), textcoords='offset points', fontsize=16,
- arrowprops=dict(arrowstyle="->", connectionstyle="arc3,rad=.2"))
-
-plt.plot([t, t],[0, np.sin(t)], color='red', linewidth=2.5, linestyle="--")
-plt.scatter([t, ],[np.sin(t), ], 50, color='red')
-
-plt.annotate(r'$sin(\frac{2\pi}{3})=\frac{\sqrt{3}}{2}$',
- xy=(t, np.sin(t)), xycoords='data',
- xytext=(+10, +30), textcoords='offset points', fontsize=16,
- arrowprops=dict(arrowstyle="->", connectionstyle="arc3,rad=.2"))
-### End of new code
-
-plt.show()
-Now you know how to make different modifications to your plots we can make some of these changes to our temerature data.
- -Saving plot to a file¶
You can take any plot you've created within Jupyter and save it to a file on disk using the plt.savefig() function. You give the function the name of the file to create and it will use whatever format is specified by the name.
trig.plot()
-
-plt.show()
-
-plt.savefig('my_fig.svg')
-Exercise 3¶
-
-
- Add in a legend for the data. -
- Add an annotation to one of the spikes in the data. Make sure the label is placed nicely.
-
-
- Tip: you can get the year and temperature for a spike using:-
warm_winter_year = df['JAN'].idxmax() -warm_winter_temp = df['JAN'].max() -
-
- - Tip: you can get the year and temperature for a spike using:
- Save the figure to a file and display it in your Jupyter notebook. -
-Bar charts¶
Of course, Matplotlib can plot more than just line graphs. One of the other most common plot types is a bar chart. Let's work towards plotting a bar chart of the average temperature per decade.
-Let's start by adding a new column to the data frame which represents the decade. We create it by taking the index (which is a list of years), converting each element to a string and then replacing the fourth character with a '0'.
years = Series(df.index, index=df.index).apply(str)
-decade = years.apply(lambda x: x[:3]+'0')
-
-df['decade'] = decade
-df.head()
-Once we have our decade column, we can use Pandas groupby() function to gather our data by decade and then aggregate it by taking the mean of each decade.
by_decade = df.groupby('decade')
-agg = by_decade.aggregate(np.mean)
-
-agg.head()
-At this point, agg is a standard Pandas DataFrame so we can plot it like any other, by putting .bar after the plot call:
agg.YEAR.plot.bar()
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-Exercise 4¶
-
-
Plot a bar chart of the average temperature per century.
--
-
- Set the limits of the y-axis to zoom in on the data. -
-Plot a histogram of the average annual temperature
--
-
- Make sure that the x-axis is labelled correctly. -
- Tip: Look in the documentation for the right command to run -
-Plot a scatter plot of each year's February temperature plotted against that year's January temperature. Is there an obvious correlation?
-
-
-Viewing Molecules¶
nglview is an extremely powerful and capable 3D molecule view that runs within a web browser. It supports complex visualisations of molecules from a range of file formats, and can even be used to view trajectories. It provides a full framework for building 3D molecular visualisation into your Jupyter notebooks or websites.
-While nglview is very powerful, that power and flexibility can be a little daunting for newcomers. BioSimSpace is a project that provides easy-to-use wrappers around common molecular simulation tasks. One such task is viewing molecules. BioSimSpace provides the function viewMolecules that uses nglview to do exactly that :-)
from BioSimSpace import viewMolecules
-v = viewMolecules("data/dioxin.pdb")
-The above code has use the molecule file parsers built into BioSimSpace to load the molecule contained in dioxin.pdb. This is then rendered using nglview. The above nglview interface allows you to rotate the molecule (left click and drag), zoom in and out (pinch or scroll up or down) and translate (right click and drag, or control+click on a Mac).
Try moving and rotating the molecule. If you lose the molecule, click the "Center" button in the General tab to recenter the molecule.
-
The BioSimSpace viewMolecules function has done two things:
-
-
- it first loaded the molecule(s) from the file, -
- and it then rendered them -
Loading molecules can take a long time and use a lot of memory. To prevent you from having to repeatedly load molecules, the viewMolecules function has returned a view object that can be re-used. To see how to use it, use python's help...
help(v)
-As you can see, we can use v.system() to view all of the loaded molecules again, without having to reload them.
v.system()
-You can change the representation of the molecule by clicking on the "Representation" tab. First click the "Remove" icon to remove the current representation. Then click the drop-down representation box to choose another representation (e.g. "spacefill"). Then click the "Add" icon to add that representation. Experiment with adding and removing different representations.
-
Loading lots of molecules¶
nglview isn't just limited to viewing small molecules. It also works really well as a viewer for large molecular systems. It (sometimes) is sufficiently clever to select appropriate representations for the molecules being loaded.
-For example, view the protein-ligand complex in data/complex.pdb
v = viewMolecules("data/complex.pdb")
-In this case, nglview has automatically selected a cartoon representation for the protein and a ball and stick representation for the ligand.
-You can achieve this yourself by using selections to set different representation for different molecules (or parts of molecules). First, delete the default representations by repeatedly clicking the "Remove" button in the representations tab. Once you have removed all of them, we will add a new representation. Select the type as surface, and then type "protein" into the selection box (which starts off with a "*" in it).
-
Click "Add". After some time thinking, nglview will show you a surface representation of the protein.
-Next, add a "spacefill" representation to the ligand. The ligand residue is called "LIG", so to do this, select "spacefill", type "LIG" into the selection box, and then click add. You should now see the ligand neatly bound into the protein.
-
The selection box can be used to select proteins ("protein"), water ("water"), everything ("*") or residues by name (e.g. "LIG") or number (e.g. "35"). Play around creating different selections and representations. For example, create a "point" representation for water, a "tube" representation of the protein and a "licorice" representation of all alanine residues. Note - you can control the opacity (level of transparency) of a representation by selecting the representation in the drop down box and changing the "opacity" slider in the "Parameters" tab - see below. You can also change things like the colour scheme of the representation in this "Parameters" tab
-
Viewing individual molecules¶
The view object returned by BioSimSpace can be used to view specific molecules from the file. To do this, use the molecules function. This takes a list of indicies of the molecules you want to view. For example, to view the first molecule (molecule at index 0) type;
v.molecules([0])
-while to view molecules 100-1000 use the below code (noting that you may need to add a "ball and stick" represntation in case nglview automatically hides the water molecules).
- -v.molecules( range(100,1000) )
-Loading more complex files¶
BioSimSpace provides reader and writers for a variety of molecular file formats. Some of these split the molecular data over multiple files, e.g. a topology and coordinate file. To view these, pass all of the necessary files to viewMolecules in a list, e.g.
v = viewMolecules(["data/ala.top","data/ala.crd"])
-This can be combined with molecule selection, e.g. to load and view only molecules 0-4 in the file pass the indicies of the molecules you want to view as a second argument to viewMolecule, e.g.
v = viewMolecules(["data/ala.top","data/ala.crd"], [0,1,2,3,4])
-(in reality, all molecules are loaded, but only molecules specified by the indicies are viewed. You can still use v.system() to view all molecules)
v.system()
-Regular Expressions¶
In the error handling session we tried to interpret strings as valid heights and weights. This involved looking for text such as "meter" or "kilogram" in the string, and then extracting the number. This process is called pattern matching, and is best undertaken using a regular expression.
-Regular expressions have a long history and are available in most programming languages. Python implements a standards-compliant regular expression module, which is called re.
import re
-Let's create a string that contains a height and see if we can use a regular expression to match that...
- -h = "2 meters"
-To search for string "meters" in a string, using re.search, e.g.
if re.search("meters", h):
- print("String contains 'meters'")
-else:
- print("No match")
-re.search returns a match object if there is a match, or None if there isn't.
m = re.search("meters", h)
-m
-This matches "meters", but what about "meter". "meter" is "meters" without an "s". You can specify that a letter is matched 0 or 1 times using "?"
- -h = "2 meter"
-m = re.search("meters?", h)
-m
-However, this has still not worked, as we match "meters" in the middle of the string. We need to match "meters" only at the end of the string. We do this using "$", which means match at end of string
- -m = re.search("meters?$", h)
-m
-We also want to be able to match "m" as well as "meters". To do this, we need to use the "or" operator, which is "|". It is a good idea to put this in round brackets to make both sides of the "or" statement clear.
- -h = "2 m"
-m = re.search("(m|meters?)$", h)
-m
-Next, we want to match the number, e.g. "X meters", where "X" is a number. You can use "\d" to represent any number. For example
- -h = "2 meters"
-m = re.search("\d (m|meters?)$", h)
-m
-A problem with the above example is that it only matches a number with a single digit, as "\d" only matches a single number. To match one or more digits, we need to put a "+" afterwards, as this means "match one or more", e.g.
- -h = "10 meters"
-m = re.search("\d+ (m|meters?)$", h)
-m
-This match breaks if the number is has decimal point, as it doesn't match the "\d". To match a decimal point, you need to use "\.", and also "?", which means "match 0 or 1 decimal points", and then "\d*", which means "match 0 or more digits"
- -h = "1.5 meters"
-m = re.search("\d+\.?\d* (m|meters?)$", h)
-m
-The number must match at the beginning of the string. We use "^" to mean match at start...
- -h = "some 1.8 meters"
-m = re.search("^\d+\.?\d* (m|meters?)$", h)
-m
-Finally, we want this match to be case insensitive, and would like the user to be free to use as many spaces as they want between the number and the unit, before the string or after the string... To do this we use "\s*" to represent any number of spaces, and match using re.IGNORECASE.
h = " 1.8 METers "
-m = re.search("^\s*\d+\.?\d*\s*(m|meters?)\s*$", h, re.IGNORECASE)
-m
-The round brackets do more than just groups parts of your search. They also allow you extract the parts that match.
- -m.groups()
-You can place round brackets around the parts of the match you want to capture. In this case, we want to get the number...
- -m = re.search("^\s*(\d+\.?\d*)\s*(m|meters?)\s*$", h, re.IGNORECASE)
-m.groups()
-As m.groups()[0] contains the match of the first set of round brackets (which is the number), then we can get the number using m.groups()[0]. This enables us to rewrite the string_to_height function from the last section as;
def string_to_height(height):
- """Parse the passed string as a height. Valid formats are 'X m', 'X meters' etc."""
- m = re.search("^\s*(\d+\.?\d*)\s*(m|meters?)\s*$", height, re.IGNORECASE)
-
- if m:
- return float(m.groups()[0])
- else:
- raise TypeError("Cannot extract a valid height from '%s'" % height)
-h = string_to_height(" 1.5 meters ")
-h
-
-Exercise 2¶
Update string_to_height so that it can also understand heights in both meters and centimeters (returning the height in meters), and update string_to_weight so that it can also understand weights in both grams and kilograms (returning the weight in kilograms). Note that you may find it easier to separate the number from the units. You can do this using the below function to divide the string into the number and units. This uses "\w" to match any word character.
- -def get_number_and_unit(s):
- """Interpret the passed string 's' as "X units", where "X" is a number and
- "unit" is the unit. Returns the number and (lowercased) unit
- """
- m = re.search("^\s*(\d+\.?\d*)\s*(\w+)\s*$", s, re.IGNORECASE)
-
- if m:
- number = float(m.groups()[0])
- unit = m.groups()[1].lower()
- return (number, unit)
- else:
- raise TypeError("Cannot extract a valid 'number unit' from '%s'" % s)
-
-What is Jupyter and how do you use it?¶
Project Jupyter exists to develop software and standards for interactive computing. Jupyter, which you are using now, provides an interactive notebook which runs on a cloud-server, and with which you interact using a web browser.
-As installed, Jupyter provides two things:
--
-
- An interactive Python notebook, which you are using now (this is an interactive notebook!) -
- An interactive bash terminal (we will use this later) -
What is an interactive Python notebook? It is a notebook that mixes documentation (like this!) with Python code (like below), with the output of that code.
-For example, below is the Python code to print "Hello World". Click on the below code and then press "SHIFT+Return". You should see that the code is run, and below "Hello World" will be printed.
- -print("Hello World")
-The above code is interactive because it runs when you pressed "SHIFT+Return". It is also interactive because you can change it. For example, click again on the code and change "Hello World" to "Hello Jupyter". Press "SHIFT+Return" again and you should see "Hello Jupyter" printed.
-A code cell can have as much or little Python in it as you want. The below cell defines a function and then runs it in a loop. What do you think will be printed out when you select it and press "SHIFT+Return"? Have a go. Are you right?
- -def countDown(start):
- for i in range(start, 0, -1):
- print("%d..." % i)
- print("Lift off!")
-
-countDown(10)
-Each cell can use the variables and functions defined in previous cells. For example, the next cell sets the value of the variable x to 5, while the following cell then uses that to call the countDown function.
x = 21
-countDown(x)
-You can go back up and edit the value of x. For example, set x to 7 and press SHIFT+Return. This has changed x. However, it doesn't re-run any code that depends on x. That is for you to do. You need to click on the next cell countDown(x) and press SHIFT+Return again. You should then see that the code is run and updated.
Because you can change any cell at any time, a notebook can get into a confused state. If we change x again to 21, then the cell countDown(x), is now out of date, as it depends on the old value of x. So, how can we know what state each cell is in? The state is based on the order in which cells are executed. This is shown by the little number next to each cell, e.g. see below.
The number "10" shows that the line x = 21 was the 10th cell executed by Python, while the line countDown(x) was the 9th cell executed. As countDown(x) was executed before x = 21, it was not affected by x = 21. If you want it to be affected, you need to execute that cell again (so it is now the 11th cell executed), e.g.
If you find yourself getting confused, then click "Kernel | Restart & Clear Output" from the menu above to clear all outputs, clear the state, and reset back to a clean notebook.
-
Interactive Graphics¶
In addition to mixing documentation with interactive Python, the notebook also allows Python modules to embed interactive graphics. For example, the BioSimSpace Project provides a easy wrapper around the Python molecular viewer nglview. This can be used to view molecules from within a notebook.
-Execute the cell below by selecting it and pressing SHIFT+Return. This will import the viewMolecules function from BioSimSpace, which will load the molecules contained in the file data/complex.pdb and will display it below.
from BioSimSpace import viewMolecules
-v = viewMolecules("data/complex.pdb")
-You can rotate and interact with molecules once they have loaded (the "[*]" next to a cell indicates that it is processing. It will turn into a number once processing is complete)
-You can change molecular representations and interact with the 3D molecule view. Later on we will go through some examples showing what you can do and how you can select parts of the molecule.
-In addition to viewing molecules, you can also draw graphs. This is achieved using a combination of the matplotlib, numpy and pandas modules. For example, the below code draws a sine and cosine curve.
- -# First, import pandas, numpy and matplotlib
-import pandas as pd
-from pandas import Series, DataFrame
-import numpy as np
-import matplotlib.pyplot as plt
-%config InlineBackend.figure_format = 'svg' #Â helps make things look better in Jupyter :-)
-
-# now define a sine and cosine curve
-X = np.linspace(-np.pi, np.pi, 256, endpoint=True)
-data = {'cos': np.cos(X), 'sin': np.sin(X)}
-trig = DataFrame(index=X, data=data)
-
-trig.plot()
-Later on we will go through pandas, matplotlib and numpy and you will learn how to draw lots of different and interesting graphs.
- -Mixing documentation and code¶
In addition to interactivity, the real benefit of a Jupyter notebook is that it encourages you to include lots of documentation with your code. In effect, a notebook is a combination of code, documentation, and (once it has been run) analysis and data.
-You can create documentation by changing the type of a cell. The type of a cell is indicated in the drop-down at the top, under the menu. If you click on the graph above, you will see that the type of a code cell is "Code".
-
If you now click on this documentation, you should see the cell type change to "Markdown"
-
Creating your own cells¶
You can create your own cells in two different ways;
--
-
- either by pressing SHIFT+Return on the last cell (which creates a new cell at the bottom of the notebook), -
- or by clicking "Insert | Cell Above" or "Insert | Cell Below" from the menu to insert a new cell above or below the currently selected cell. -
You can choose whether this should be a code or documentation (markdown) cell by selecting the appropriate value from the drop-down at the top.
-Code cells can contain any valid Python. Documentation (markdown) cells can contain any text. The text can be formatted, using the markdown standard.
-Have a go by creating some new documentation and code cells below. If you want some inspiration for documentation, take a look at the documentation cells above by double-clicking on them. You should see how I have inserted images, lists, headings etc.
-You can then render the documentation using SHIFT+Return (just like running a code cell)
- -a = 10
-b = 32
-print("The answer to life, the universe and everything is %d" % (a+b))
-Controlling the Python kernel¶
The Python process that is interpreting and executing your Python code is referred to as the "kernel" of the notebook (hence the "Kernel" menu above). When you execute a code cell it sends the code to the Python kernel, and then waits for the kernel to send a result back. The Python kernel can only execute one thing at a time. This means that, for slow functions, you could be executing cells too quickly and leave the Python kernel behind.
-For example, here is a function that fakes a slow function as it goes to sleep for the specified number of seconds.
- -def sleep(n):
- import os
- os.system("sleep %d" % n)
-Below we will have three code cells that each sleep for a long time. Execute all three quickly one after another...
- -sleep(30)
-sleep(30)
-sleep(30)
-You should see that there is a [*] next to each code cell. This is because they are all waiting for the Python kernel to finish executing and return.
-
If you wait you will see each sleep finishing and control then passing to the next.
-Sometimes you don't want to be patient, and you want to stop the Python kernel from running a function. To stop (interupt) the kernel, either click the "Stop" icon at the top of the screen, or select "Kernel | Interupt" from the menu.
-
Other useful buttons up there are "restart the kernel" (same as the menu Kernel | Restart & Clear Output) and the arrows that can move the selected cell up and down the notebook.
-Have a play. Explore the buttons and menu items. Take a look at the "Help" in the menu. Remember that, if everything goes wrong, you can always click "Stop" or select "Kernel | Restart & Clear Output" from the menu.
- -
-Running external programs in a code cell¶
You can do more than just execute Python in the code cells. You can run any command you want! Just put an exclamation mark first, and then type the command normally as you would in a terminal. For example, the below cell is running ls
! ls
-You can run any command that you want, exactly as you would typing in a terminal. For example, lets use cat to read the LICENSE file...
! cat LICENSE
-You can work with directories as well. For example, here we will copy the dioxin.pdb file from data into the current directory and will then view it using viewMolecules...
! cp data/dioxin.pdb ./
-from BioSimSpace import viewMolecules #Â importing again in case we restarted the kernel
-viewMolecules("dioxin.pdb")
-Saving the notebook¶
At the top of the screen is the name of the notebook, and its current save state.
-
The current save state tells you when the last checkpoint was saved, and whether or not the notebook contains any unsaved changes. Checkpoints provide points in the notebook that you can refer back to, using the menu item "File | Revert to Checkpoint". You are always able to revert to the last checkpoint, meaning that if you make a mistake, you can go back to a previous safe version.
-If you click "File | Save and Checkpoint", or click the "Save" icon (floppy disk icon on the far left), then the current state of the notebook will be saved. This includes not just the code and documentation you have written, but also all of the outputs and graphs that you have created. This is really useful, as it lets you save your work, and then come back later.
-For example, what don't you save now, and then close this window and then open the notebook again? You should see that everything you have done, and everything that was output is restored. Did it work?
-The notebook was saved to a file called 01_jupyter_howto.ipynb. The .ipynb signifies that the file is a Jupyter notebook (formerly called an interactive python notebook). You can change the name by clicking on the name of the notebook above, and then choosing a different name in the window that opens.
Alternatively, you can save a copy of this notebook under a different name by clicking "File | Make a Copy..." in the menu. You can then copy this file or send it to someone else. As a notebook contains the code, data, outputs and analysis, sharing notebooks is the closest thing we have to interactive papers. This is a really powerful concept and, I believe, will fundementally change how we report and share the results of computational and data analysis in science.
- -
-The Jupyter Hub¶
When you first logged into this website you launched this notebook by clicking on its name in the Jupyter Hub. This was the page that looked like a file browser and that (hopefully) is still open in one of the tabs in your browser. The Jupyter Hub is not just a file brower. It is the gateway to do a lot more than just interact with a single notebook.
-The first thing to note is that this Jupyter Hub is running on the cloud and consumes resources. It is important that you click "Logout" (top right) when you have finished, as this will free up the hub to be used by someone else. There is a timer that will automatically log you out 40 minutes after your web browser is closed. However, please remember to log out if you can.
-
The main view of the hub is the file browser that shows all of your files. You can navigate through the files and directories like any normal file browser. For example, you can see above the 01_jupyter_howto.ipynb file that contains this notebook. You should see that the file has a green icon next to it, as well as a Running indicator on the right. This indicates that the notebook is currently running.
Running notebooks consume cloud resources, which are limited. You can only have 1.8 GB of data in memory and have limited CPU. You should thus shutdown any running notebooks that you aren't using any more. To do this, click the checkbox to the left of the notebook filename. This will bring up some buttons at the top of the screen. Click the "Shutdown" button to shut down the notebook.
-
Alternatively, you can go to the "Running" tab to see everything of your's that is currently running in the hub. Click the "Shutdown" button on the right for anything that you aren't using and want to shut down.
-
Back in the "Files" tab, clicking the checkbox next to a file brings up buttons that let you download files. You can use the hub to download files by selecting them and clicking the "Download" button that will appear. Other buttons will appear depending on the type of file, e.g. "Duplicate", "View", "Edit" etc. The bin icon will delete the file.
-
You can create new notebooks or files by clicking on the "New" button on the right. This will open a drop-down menu for you to select the type of thing to create.
-
As well as letting you create a new jupyter notebook (by clicking "New | Python3"), you can also create new text files ("New | Text File") or new folders ("New | Folder"). Most interestingly, is that you can also create a new bash terminal. If you click "New | Terminal" it will open a new tab in which you will have a fully functional bash terminal shell. This is running on the same cloud server as your notebook. It is 100% completely a fully functional terminal within your web browser. You could even use it to run the normal command-line python ;-)
-
The cloud server on which this is running comes with lots of useful software that you can run from within the bash shell. For example you have;
--
-
update_workshops: This command will update all of the software and workshop material to the latest version. Useful if you delete anything by accident or the tutor needs to make a last-minute fix
-git: Git is installed. The workshop material is downloaded and updated from git repositories, e.g. https://github.com/ccpbiosim/python_and_data. You could also use git to push things back to the cloud, e.g. if you want to download lots of files.
-anaconda python3: A full anaconda python3 is installed, with pandas, matplotlib, numpy etc. etc.
-wget: This tool lets you quickly download (or should that be sideload?) files onto the cloud server
-top: Process monitor. You can use this to see which processes are consuming lots of resources. Any process that uses more than 1.8 GB of memory will automatically be killed by the cloud server. You can kill them yourself using thekillcommand.
-tarandbzip2: useful for packing/unpacking and compressing/uncompressing files
-AmberTools: The full AmberTools suite is installed in directory$AMBERHOME
-ProtoMS: The full ProtoMS package is installed in directory$PROTOMSHOME
-whamandwham-2d: Tools used for weighted histogram analysis
-SireandBioSimSpace: The Sire and BioSimSpace python simulation framework and workflow node packages.
-
The cloud server was build with docker. If you want to download it yourself to run on your own computer at home after the workshop then please install docker and type
- -docker run -it --rm -p 8888:8888 chryswoods/bss-workshop:latestThis will download and run the image from dockerhub and will make it available at the web address that will be printed to the screen (e.g. it will look something like http://localhost:8888/?token=641396480e6421eae8b18261d82a75f958fe166e1c8b20a8). Simply open that address in your browser :-). You can see and download the DockerFile used to generate this image from here.
Thanks to the Microsoft Azure Kubernetes Service on which this Jupyter image is running.
- -
-Lists¶
Lists provide a simple way to hold a collection of values. You create a list using square brackets. For example, here we can create a list that contains the values "cat", "dog" and "horse"
-a =[ "cat", "dog", "horse" ]
-a = ["cat", "dog", "horse"]
-print(a)
-You can access the items in the list by placing the index of the item in square brackets. The first item is at index 0
- -a[0]
-The second item is at index 1, and the third item at index 2.
- -a[2]
-What do you think will happen if we access the item at index 3?
- -a[3]
-You can also access the items in the list from the back. What do you think is at index -1?
- -a[-1]
-What about index -2, -3 or -4?
- -a[-3]
-You can add items onto a list by using the .append function. You can find this using tab completion and Python help...
help(a.append)
-a.append("fish")
-a
-a[3]
-You can put whatever you want into a list, including other lists!
- -b = [ 42, 15, a, [7,8,9] ]
-b[3]
-Putting lists inside lists allows for multidimensional lookup, e.g. can you work out why b[3][2] equals 9?
b[3][2]
-You can loop over the items in a list using a for loop, e.g.
- -for x in a:
- print(x)
-You can get the number of items in the list using the len function.
len(a)
-You can use this as an alternative way of looping over the elements of a list
- -for i in range(0,len(a)):
- print(a[i])
-A string behaves like a list of letters. For example, if we have the string s = "Hello World", then s[0] is "H" and s[-1] is d.
s = "Hello World"
-s[-1]
-You can loop over every letter in a string in the same way that you can loop over every item in a list.
- -for letter in s:
- print(letter)
-Exercises¶
Exercise 1¶
Create two Python lists called a and b. Put into these lists the values [2, 4, 6, 8], and [10, 20, 30, 40].
-Check that a[2] equals 6 and b[-1] equals 40. (note that you will need to use the menu "Insert | Insert Cell Below" to insert more cells below to create space for your code)
a = [2, 4, 6, 8]
-b = [10, 20, 30, 40]
-a[2] == 6 and b[-1] == 40
-Exercise 2¶
Now create a loop that loops over each item in a and b and that calculates and prints out the product a[i] * b[i].
for i in range(0,len(a)):
- print(a[i] * b[i])
-Exercise 3¶
Modify your code to create a list called c. Use the .append function to set c[i] = a[i] * b[i]. Check your code by making sure that c[-1] equals 320.
c = []
-for i in range(0,len(a)):
- c.append(a[i] * b[i])
-print(c)
-c[-1] == 320
-
-Dictionaries¶
Dictionaries are another type of Python container. Instead of storing values by index, they store them associated with a key.
-You create dictionaries using curly brackets, assiging values to their keys using a colon, e.g.
-a = { "cat" : "mieow", "dog" : "woof", "horse" : "neigh" }
-a = { "cat" : "mieow", "dog" : "woof", "horse" : "neigh"}
-a
-You can look up values in the dictionary by placing the key in square brackets. For example, we can look up the value associated with the key "cat" using a["cat"].
a["cat"]
-What happens if the key does not exist?
- -a['fish']
-You insert items into the dictionary by assigning values to keys, e.g.
- -a["fish"] = "bubble"
-a
-You can list all of the keys or values of a dictionary using the keys or values functions (which you can find using tab completion and Python help)
help(a.values)
-a.keys()
-a.values()
-You can loop over the dictionary by looping over the keys and looking up the values in a for loop, e.g.
- -for key in a.keys():
- print("A %s goes %s" % (key, a[key]))
-You can put anything as a value into a dictionary, including other dictionaries and even lists. The keys should be either numbers or strings.
- -b = { "a" : ["aardvark", "anteater", "antelope"], "b" : ["badger", "beetle"], 26.5: a}
-What do you think is at b["a"][-1]? What about b[26.5]["fish"]?
b[26.5]["fish"]
-Exercise¶
Below you have a dictionary that contains the full mapping of every letter to its Morse-code equivalent.
- -letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-Exercise 1¶
Use the morse code dictionary to look up the morse code for the letters "s" and "o". What is the morse code for "SOS" (the international emergency distress signal)?
- -print(letter_to_morse["s"], letter_to_morse["o"], letter_to_morse["s"])
-Exercise 2¶
Here is a string that contains a message that must be converted to morse code. Write a loop that converts each letter into the morse code equivalent, and stores it into a list. Print the list out to see the full morse code message that must be sent. Note that you will need to use the .lower() function to get the lower case of capital letters.
message = "SOS We have hit an iceberg and need help quickly"
-morse = []
-for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
-print(morse)
-Exercise 3¶
The inverted form of a dictionary is one where the keys are now looked up by value. For example, the below code inverts letter_to_morse such that the morse code is the key, and the letter is the value.
morse_to_letter = {}
-for letter in letter_to_morse.keys():
- morse_to_letter[ letter_to_morse[letter] ] = letter
-Check that this code works by verifying that morse_to_letter["..."] equals "s".
Next, loop through the morse code message you created in exercise 2 and see if you can convert it back to english. Note that you can join a list of letters together into a string using the code "".join(letters).
english = []
-for code in morse:
- english.append( morse_to_letter[code] )
-print("".join(english))
-
-Functions¶
Functions provide a way to package often-used code into reusable and easy to use components. For example, here is some code that multiplies together two lists
- -list1 = [2, 4, 6, 8]
-list2 = [10, 20, 30, 40]
-list3 = []
-for x, y in zip(list1,list2):
- list3.append(x * y)
-list3
-We don't want to keep typing out the above code every time we want to multiply the numbers in two lists. Instead, we can collect that code together into a function
- -def multiply(a, b):
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-We can now call this function directly on our lists, e.g.
- -list3 = multiply(list1, list2)
-list3
-The function is called using its name, and passing in the values as two arguments, e.g.
- -list4 = multiply( [1,2,3], [4,5,6] )
-list4
-The arguments are placed inside the round brackets. These are copied, in order, to the function. For example, [1,2,3] is copied into a, and [4,5,6] is copied as b. The code in the function is then executed. We can watch this code being run by adding in print statements, e.g.
def multiply(a, b):
- print("a = %s" % a)
- print("b = %s" % b)
- c = []
- for x,y in zip(a,b):
- print("%s times %s equals %s" % (x,y,x*y))
- c.append(x*y)
- print("c = %s" % c)
- return c
-You must pass the right number of arguments into a function. For example, this is what happens if you get the number of arguments wrong...
- -list5 = multiply(list1)
-You can write functions that take as many (or as few) arguments as you want. For example, here is a function that takes no arguments, and then a function that takes lots
- -def func0():
- return "no arguments to this function"
-def func1(a, b, c, d, e=5):
- return a+b+c+d+e
-func0()
-func1(1, 2, 3, 4, 5)
-func1(1, 2, 3, 4)
-Note that with the last function we have set a default value of the argument e. This is given the value of 5 if it is not specified. This allows us to pass 4 arguments instead of 5. Changing the default value by editing the definition of the function above will thus change the output of func1 when it is called with only four arguments.
Exercise¶
Here is the morse code dictionary from the last session, together with the code that converts a message from english into morse code.
- -letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-message = "SOS We have hit an iceberg and need help quickly"
-morse = []
-for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
-print(morse)
-Exercise 1¶
Create a function called encode that takes a message and returns the morse code equivalent. Test this function by encodig the message SOS We have hit an iceberg and need help quickly and check that you get the same result as in the last session. Now try using your function to encode other messages.
def encode(message):
- morse = []
- for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
- return morse
-encode(message) == morse
-encode("Hello World")
-Exercise 2¶
Using the answer from Exercise 2 in the dictionaries lesson, write a function called decode that converts a morse code message back to english. Check that you can decode the above morse code message back to english.
morse_to_letter = {}
-for letter in letter_to_morse.keys():
- morse_to_letter[ letter_to_morse[letter] ] = letter
-def decode(morse):
- english = []
- for code in morse:
- english.append( morse_to_letter[code] )
- return "".join(english)
-decode(morse)
-Exercise 3¶
Below is a list of messages. Loop over the messages and check that encode( decode(message) ) equals the original message. Do any of the messages fail to encode and decode correctly? If so, why? How can your check be modified to account for the limitations of your encode and decode functions?
- -messages = [ "hello world", "this is a long message", "Oh no this may break", "This message is difficult to encode." ]
-for message in messages:
- print("checking for message '%s'..." % message)
- print( message == decode( encode(message) ) )
-for message in messages:
- print("checking for message '%s'..." % message)
- print( message.lower() == decode(encode(message)) )
-
-Objects¶
In the last session you learned how to package up useful code into functions. This is a really useful idea, as it lets you re-use useful code in your own scripts, and to then share useful code with other people.
-However, it is normal for functions to rely on data. For example, consider the Morse code encode and decode functions in the last lesson. These only work because of the data contained in the letter_to_morse dictionary. The functions would break if anyone changes the data in this dictionary.
letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-def encode(message):
- morse = []
- for letter in message:
- morse.append( letter_to_morse[letter.lower()] )
- return morse
-encode("Hello")
-The above encode("Hello") has worked. However, if we change the data in letter_to_morse, e.g. swapping l from .-.. to -.--, then we get ['....', '.', '-.--', '-.--', '---'], which is wrong. We can make even larger changes, which would completely break the function...
While such changes are easy to spot in this example, they become more difficult to find in larger programs. In addition, as you share code, you will find that people using your code will do weird things to the data on which it depends, which can introduce weird bugs and problems.
-The solution is to package a function together with the data on which it depends into a single object. This idea is the foundation of object orientated programming. To explore this, let us start with a simple example that packages the encode function together with the letter_to_morse dictionary on which it depends.
class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-Above, we have packaged the data (letter_to_morse) together with the encode function into what we call a class. A Class describes how data and functions are combined together. An instance of a class is called an object, which we can create by calling Morse().
m = Morse()
-m is an object of the class Morse. It has its own copy of letter_to_morse within it, and its own copy of the encode function. We can call m's copy of the encode function by typing m.encode(...), e.g.
m.encode("Hello World")
-To create a new class, you use the class keyword, followed by the name of your class. In this case, class Morse defined a new class called Morse. You then add a colon, and write all of the functions that should be part of the class indented below. At a minimum, you must define one function, called the constructor. This function has the signature def __init__(self, arguments...). The first argument, self, is a special variable that allows an object of the class to access the data that belongs to itself. It is the job of the constructor to set up that data. For example, let's now create a new class that provides a simple guessing game.
class GuessGame:
- def __init__(self, secret):
- self._secret = secret
-
- def guess(self, value):
- if (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- print("Try again...")
-In this class, the constructor __init__(self, secret) takes an extra argument after self. This argument is saved as the _secret variable that is part of the self of the object. Note that we always name variables that are part of a class with a leading underscore. We can construct different object instances of GuessGame that have different secrets, e.g.
g1 = GuessGame("cat")
-g2 = GuessGame("dog")
-Here, the self._secret for g1 equals "cat". The self._secret for g2 equals "dog".
When we call the function g1.guess(value), it compares value against self._secret for g1.
g1.guess("dog")
-g1.guess("cat")
-When we call the function g2.guess(value) it compares value against self._secret for g2.
g2.guess("cat")
-g2.guess("dog")
-Exercise¶
Exercise 1¶
Edit the below GuessGame example so that it records how many unsuccessful guesses have been performed. Add a function called nGuesses() that returns the number of unsuccessful guesses. Once you have made the changes, check your class by creating an object of your class and using it to make some successful and unsuccessful guesses.
class GuessGame:
- def __init__(self, secret):
- self._secret = secret
- self._nguesses = 0
-
- def guess(self, value):
- if (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- self._nguesses += 1
- print("Try again...")
-
- def nGuesses(self):
- return self._nguesses
-g = GuessGame("cat")
-g.nGuesses() == 0
-g.guess("dog")
-g.nGuesses() == 1
-g.guess("horse")
-g.nGuesses() == 2
-g.guess("cat")
-g.nGuesses() == 2
-Exercise 2¶
Edit the constructor of your GuessGame class so that the user can optionally specify a maximum number of allowable guesses. If the maximum number of guesses is not supplied, then set the default value to 5.
Create a maxGuesses() function that returns the maximum number of allowable guesses.
Finally, edit the guess() function so that it will not let you make more than the maximum number of guesses (e.g. if the number of guesses exceeds the maximum number, then print out "Sorry, you have run out of guesses.").
Check that you code works by creating an object of GuessGame that only allows three guesses, and see what happens if you guess incorrectly more than three times.
- -class GuessGame:
- def __init__(self, secret, max_guesses=5):
- self._secret = secret
- self._nguesses = 0
- self._max_guesses = max_guesses
-
- def guess(self, value):
- if (self.nGuesses() >= self.maxGuesses()):
- print("Sorry, you have run out of guesses")
- elif (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- self._nguesses += 1
- print("Try again...")
-
- def nGuesses(self):
- return self._nguesses
-
- def maxGuesses(self):
- return self._max_guesses
-g = GuessGame("fish", 3)
-g.maxGuesses() == 3
-g.guess("cat")
-g.guess("dog")
-g.guess("horse")
-g.guess("gerbil")
-
-Classes¶
Classes allow you to define how to package data with functions to create objects. An object is an instance of a class, which contains its own data, and its own copy of functions that can operate on that data.
-You use classes to define objects that represent the concepts and things that your program will work with. For example, if your program managed exam results of students, then you may create one class that represents an Exam, and another that represents a Student.
- -class Exam:
- def __init__(self, max_score=100):
- self._max_score = max_score
- self._actual_score = 0
-
- def percent(self):
- return 100.0 * self._actual_score / self._max_score
-
- def setResult(self, score):
- if (score < 0):
- self._actual_score = 0
- elif (score > self._max_score):
- self._actual_score = self._max_score
- else:
- self._actual_score = score
-
- def grade(self):
- if (self._actual_score == 0):
- return "U"
- elif (self.percent() > 90.0):
- return "A"
- elif (self.percent() > 80.0):
- return "B"
- elif (self.percent() > 70.0):
- return "C"
- else:
- return "F"
-class Student:
- def __init__(self):
- self._exams = {}
-
- def addExam(self, name, exam):
- self._exams[name] = exam
-
- def addResult(self, name, score):
- self._exams[name].setResult(score)
-
- def result(self, exam):
- return self._exams[exam].percent()
-
- def grade(self, exam):
- return self._exams[exam].grade()
-
- def grades(self):
- g = {}
- for exam in self._exams.keys():
- g[exam] = self.grade(exam)
- return g
-We can now create a student, and give them a set of exams that they need to complete.
- -s = Student()
-s.addExam( "maths", Exam(20) )
-s.addExam( "chemistry", Exam(75) )
-At this point, the student has not completed any exams, so the grades are all 'U'
- -s.grades()
-However, we can now add the results...
- -s.addResult("maths", 15)
-s.addResult("chemistry", 62)
-s.grades()
-Programming with classes makes the code easier to read, as the code more closely represents the concepts that make up the program. For example, here we have a class that represents a full school of students.
- -class School:
- def __init__(self):
- self._students = {}
- self._exams = []
-
- def addStudent(self, name):
- self._students[name] = Student()
-
- def addExam(self, exam, max_score):
- self._exams.append(exam)
-
- for key in self._students.keys():
- self._students[key].addExam(exam, Exam(max_score))
-
- def addResult(self, name, exam, score):
- self._students[name].addResult(exam, score)
-
- def grades(self):
- g = {}
- for name in self._students.keys():
- g[name] = self._students[name].grades()
- return g
-We can now create a whole school of students and manage the exams and results for all of them with some reasonably readable code :-)
- -school = School()
-school.addStudent("Charlie")
-school.addStudent("Matt")
-school.addStudent("James")
-school.addExam( "maths", 20 )
-school.addExam( "physics", 50 )
-school.addExam( "english literature", 30 )
-school.grades()
-We can now add in the results of the exams, which have been returned to us by the exam markers...
- -englit_results = { "Charlie" : 10, "Matt" : 25, "James" : 3 }
-phys_results = { "Matt" : 48, "James" : 3 }
-maths_results = { "James" : 20, "Matt" : 18, "Charlie" : 4 }
-Indeed, we will do this by using a function...
- -def add_results(school, exam, results):
- for student in results.keys():
- school.addResult(student, exam, results[student])
-add_results(school, "english literature", englit_results)
-add_results(school, "physics", phys_results)
-add_results(school, "maths", maths_results)
-school.grades()
-class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- self._morse_to_letter = {}
- for letter in self._letter_to_morse.keys():
- self._morse_to_letter[ self._letter_to_morse[letter] ] = letter
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-
- def decode(self, morse):
- message = []
- for code in morse:
- message.append( self._morse_to_letter[code] )
- return "".join(message)
-m = Morse()
-message = "Hello World"
-m.decode( m.encode(message) ) == message.lower()
-Exercise 2¶
Below is a copy of the School class, together with a copy of the code needed to populate an object of that class with students and exam results. Edit the School class to add in the following functions:
-
-
.resits(): this should return the list of exams that each student should resit if they get a "F" or "U" grade.
-.prizeStudent(): this should return the name of the student who scored the highest average percent across all of the exams.
-.reviseCourse(threshold): this should return the name of the exam that gets the lowest average score across all students, if the average score is belowthreshold.
-
Use these functions to find out which students need to resit which exams, which student should be awarded the annual school prize, and which courses should be revised as the average mark is less than 50%.
- -class School:
- def __init__(self):
- self._students = {}
- self._exams = []
-
- def addStudent(self, name):
- self._students[name] = Student()
-
- def addExam(self, exam, max_score):
- self._exams.append(exam)
-
- for key in self._students.keys():
- self._students[key].addExam(exam, Exam(max_score))
-
- def addResult(self, name, exam, score):
- self._students[name].addResult(exam, score)
-
- def grades(self):
- g = {}
- for name in self._students.keys():
- g[name] = self._students[name].grades()
- return g
-
- def resits(self):
- r = {}
- for name in self._students.keys():
- student_resits = []
-
- for exam in self._exams:
- grade = self._students[name].grade(exam)
-
- if (grade == "F" or grade == "U"):
- student_resits.append(exam)
-
- if len(student_resits) > 0:
- r[name] = student_resits
-
- return r
-
- def prizeStudent(self):
- prize_score = 0
- prize_student = None
-
- for name in self._students.keys():
- avg_score = 0
- for exam in self._exams:
- avg_score += self._students[name].result(exam)
-
- avg_score /= len(self._exams)
-
- if avg_score > prize_score:
- prize_score = avg_score
- prize_student = name
-
- return prize_student
-
- def reviseCourse(self, threshold=50):
- revise_course = {}
-
- for exam in self._exams:
- avg_score = 0
- for name in self._students.keys():
- avg_score += self._students[name].result(exam)
- avg_score /= len(self._students)
-
- if avg_score < threshold:
- revise_course[exam] = avg_score
-
- return revise_course
-students = ["Charlie", "James", "Matt"]
-exams = { "maths" : 20, "physics" : 50, "english literature" : 30 }
-results = { "maths" : { "James" : 20, "Matt" : 18, "Charlie" : 4 },
- "physics" : { "Matt" : 48, "James" : 3 },
- "english literature" : { "Charlie" : 10, "Matt" : 25, "James" : 3 } }
-school = School()
-for student in students:
- school.addStudent(student)
-for exam in exams.keys():
- school.addExam(exam, exams[exam])
-for exam in results:
- add_results(school, exam, results[exam])
-school.grades()
-school.resits()
-school.prizeStudent()
-school.reviseCourse(50)
-
-Documentation¶
Python has great in-built documentation that is available via the help function. For example
l = ["cat", "dog", "fish"]
-help(l)
-You can add similar documentation to the functions that you write. You do this by adding in a documentation string as the first string after defining the function e.g.
- -def multiply(a, b):
- """This function returns the element-wise multiplication of the passed lists 'a' and 'b'"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-multiply( [1,2,3], [4,5,6] )
-help(multiply)
-The documentation string should be placed between two sets of triple quotes ("""). This is a convention that makes it easier to expand the documentation later, and that ensures that nothing you write in the documentation will be expanded or interpreted as Python.
Documentation should provide an easy to understand, and brief description of what the function does. It should not give information that is obvious by reading the function signature. For example, this is a bad piece of documentation.
- -def multiply(a, b):
- """function multiply(a,b) -> list"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-help(multiply)
-It is much better to say what the function does, and then what it returns (as this can't be seen from the signature). Good documentation would be
- -def multiply(a, b):
- """Calculates the element-wise multiplication of a and b, returning a list of the results"""
- c = []
- for x,y in zip(a,b):
- c.append(x*y)
- return c
-help(multiply)
-Your documentation can span over multiple lines. If you are describing the arguments, then you should use one line per argument, for example
- -def make_complex(real, imag=0):
- """Create and return a complex number
-
- Keyword arguments:
-
- real -- the real part of the number
- imag -- the imaginary part of the number
- """
- return (real,imag)
-help(make_complex)
-By convention, you will notice above that the last """ is placed on its own line if the documentation spans multiple lines. It is on the same line if the documentation is short.
In general, keep your documentation short, to the point, and avoid repeating obvious information. However, be precise, as this may be the only part of your code that somebody else reads before they use your function in their program.
-A good suggestion is to look at documentation you like and try to copy that style. Also, look for code that you think is poorly documented, and try to avoid their mistakes.
- -Exercise¶
Below is a series of undocumented functions. Take a look through the functions and try to work out what they do. Once you understand the functions, write some documentation for each function. Get your neighbour to read your documentation. Do they understand what the function does based on what you have written? Do the function names -combined with your documentation accurately convey the result of calling the function?
-Note that you may have to use help(...) yourself if you don't recognise some of the code in the functions. Also try to play with the function to see how it behaves.
def add(a, b):
- """Calculates the element-wise sum of a and b, returning a list of the results"""
- c = []
- for x,y in zip(a,b):
- c.append(x+y)
- return c
-help(add)
-def subtract(a, b):
- """Calculates the element-wise ratio of a and b, returning a list of results. This function
- is badly named, as it DOES NOT return the element-wise difference of a and b!"""
- c = []
- for x,y in zip(a,b):
- c.append(x / y)
- return c
-help(subtract)
-def capitalise(message):
- """Capitalises every word in message, returning the result"""
- words = message.split(" ")
- for i in range(0,len(words)):
- words[i] = "%s%s" % (words[i][0].upper(), words[i][1:])
- return " ".join(words)
-help(capitalise)
-def surprise(x):
- """Prints 'Surprise!' if x is less than a random number between 0 and 1. Returns nothing"""
- import random
- if x < random.random():
- print("Surprise!")
-help(surprise)
-For this last function, try calling it via list_interface("ipynb").
def list_interface(x):
- """Returns a list of all files in the current directory that start with '0' and end with x"""
- import glob
- f = glob.glob("*.%s" % x)
- l = []
- for x in f:
- if x.startswith("0"):
- l.append(x)
- return l
-help(list_interface)
-
-Documenting Classes¶
It is almost as easy to document a class as it is to document a function. Simply add docstrings to all of the classes functions, and also below the class name itself. For example, here is a simple documented class
- -class Demo:
- """This class demonstrates how to document a class.
-
- This class is just a demonstration, and does nothing.
-
- However the principles of documentation are still valid!
- """
-
- def __init__(self, name):
- """You should document the constructor, saying what it expects to
- create a valid class. In this case
-
- name -- the name of an object of this class
- """
- self._name = name
-
- def getName(self):
- """You should then document all of the member functions, just as
- you do for normal functions. In this case, returns
- the name of the object
- """
- return self._name
-d = Demo("cat")
-help(d)
-Often, when you write a class, you want to hide member data or member functions so that they are only visible within an object of the class. For example, above, the self._name member data should be hidden, as it should only be used by the object.
You control the visibility of member functions or member data using an underscore. If the member function or member data name starts with an underscore, then it is hidden. Otherwise, the member data or function is visible.
-For example, we can hide the getName function by renaming it to _getName
class Demo:
- """This class demonstrates how to document a class.
-
- This class is just a demonstration, and does nothing.
-
- However the principles of documentation are still valid!
- """
-
- def __init__(self, name):
- """You should document the constructor, saying what it expects to
- create a valid class. In this case
-
- name -- the name of an object of this class
- """
- self._name = name
-
- def _getName(self):
- """You should then document all of the member functions, just as
- you do for normal functions. In this case, returns
- the name of the object
- """
- return self._name
-d = Demo("cat")
-help(d)
-Member functions or data that are hidden are called "private". Member functions or data that are visible are called "public". You should document all public member functions of a class, as these are visible and designed to be used by other people. It is helpful, although not required, to document all of the private member functions of a class, as these will only really be called by you. However, in years to come, you will thank yourself if you still documented them... ;-)
-While it is possible to make member data public, it is not advised. It is much better to get and set values of member data using public member functions. This makes it easier for you to add checks to ensure that the data is consistent and being used in the right way. For example, compare these two classes that represent a person, and hold their height.
- -class Person1:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self.height = 0
-class Person2:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self._height = 0
-
- def setHeight(self, height):
- """Set the person's height to 'height', returning whether or
- not the height was set successfully
- """
- if height < 0 or height > 300:
- print("This is an invalid height! %s" % height)
- return False
- else:
- self._height = height
- return True
-
- def getHeight(self):
- """Return the person's height"""
- return self._height
-The first example is quicker to write, but it does little to protect itself against a user who attempts to use the class badly.
- -p = Person1()
-p.height = -50
-p.height
-p.height = "cat"
-p.height
-The second example takes more lines of code, but these lines are valuable as they check that the user is using the class correctly. These checks, when combined with good documentation, ensure that your classes can be safely used by others, and that incorrect use will not create difficult-to-find bugs.
- -p = Person2()
-p.setHeight(-50)
-p.getHeight()
-p.setHeight("cat")
-p.getHeight()
-class GuessGame:
- """
- This class provides a simple guessing game. You create an object
- of the class with its own secret, with the aim that a user
- then needs to try to guess what the secret is.
- """
- def __init__(self, secret, max_guesses=5):
- """Create a new guess game
-
- secret -- the secret that must be guessed
- max_guesses -- the maximum number of guesses allowed by the user
- """
- self._secret = secret
- self._nguesses = 0
- self._max_guesses = max_guesses
-
- def guess(self, value):
- """Try to guess the secret. This will print out to the screen whether
- or not the secret has been guessed.
-
- value -- the user-supplied guess
- """
- if (self.nGuesses() >= self.maxGuesses()):
- print("Sorry, you have run out of guesses")
- elif (value == self._secret):
- print("Well done - you have guessed my secret")
- else:
- self._nguesses += 1
- print("Try again...")
-
- def nGuesses(self):
- """Return the number of incorrect guesses made so far"""
- return self._nguesses
-
- def maxGuesses(self):
- """Return the maximum number of incorrect guesses allowed"""
- return self._max_guesses
-help(GuessGame)
-Exercise 2¶
Below is a poorly-written class that uses public member data to store the name and age of a Person. Edit this class so that the member data is made private. Add get and set functions that allow you to safely get and set the name and age.
class Person:
- """Class the represents a Person, holding their name and age"""
- def __init__(self, name="unknown", age=0):
- """Construct a person with unknown name and an age of 0"""
- self.setName(name)
- self.setAge(age)
-
- def setName(self, name):
- """Set the person's name to 'name'"""
- self._name = str(name) #Â str ensures the name is a string
-
- def getName(self):
- """Return the person's name"""
- return self._name
-
- def setAge(self, age):
- """Set the person's age. This must be a number between 0 and 130"""
- if (age < 0 or age > 130):
- print("Cannot set the age to an invalid value: %s" % age)
-
- self._age = age
-
- def getAge(self):
- """Return the person's age"""
- return self._age
-p = Person(name="Peter Parker", age=21)
-p.getName()
-p.getAge()
-Exercise 3¶
Add a private member function called _splitName to your Person class that breaks the name into a surname and first name. Add new functions called getFirstName and getSurname that use this function to return the first name and surname of the person.
class Person:
- """Class the represents a Person, holding their name and age"""
- def __init__(self, name="unknown", age=0):
- """Construct a person with unknown name and an age of 0"""
- self.setName(name)
- self.setAge(age)
-
- def setName(self, name):
- """Set the person's name to 'name'"""
- self._name = str(name) #Â str ensures the name is a string
-
- def getName(self):
- """Return the person's name"""
- return self._name
-
- def setAge(self, age):
- """Set the person's age. This must be a number between 0 and 130"""
- if (age < 0 or age > 130):
- print("Cannot set the age to an invalid value: %s" % age)
-
- self._age = age
-
- def getAge(self):
- """Return the person's age"""
- return self._age
-
- def _splitName(self):
- """Private function that splits the name into parts"""
- return self._name.split(" ")
-
- def getFirstName(self):
- """Return the first name of the person"""
- return self._splitName()[0]
-
- def getSurname(self):
- """Return the surname of the person"""
- return self._splitName()[-1]
-p = Person(name="Peter Parker", age=21)
-p.getFirstName()
-p.getSurname()
-
-Exceptions¶
Mistakes and errors happen in computer programs as much as in real life. Like life, how you handle an error in your program shows your level of professionalism, and gives others evidence that they can trust that you have written a program that will work well.
-In the last section we indicated errors in the Person.setHeight function by printing a message to the screen and returning False to indicate that the call to setHeight had failed.
class Person:
- """Class that holds a person's height"""
- def __init__(self):
- """Construct a person who has zero height"""
- self._height = 0
-
- def setHeight(self, height):
- """Set the person's height to 'height', returning whether or
- not the height was set successfully
- """
- if height < 0 or height > 300:
- print("This is an invalid height! %s" % height)
- return False
- else:
- self._height = height
- return True
-
- def getHeight(self):
- """Return the person's height"""
- return self._height
-p = Person()
-p.setHeight(-20)
-This is not a good way of indicating an error. The issues with this are;
--
-
- How does the person calling
getHeightknow to check whether the call returnsTrueorFalse
- - What if we wanted to return something else? Should we return the error state and the value we want together? -
- If the error state is not checked, and nobody reads the error message printed to the screen, then the program is broken, as the person has been created with a height of 0. -
The solution is to send something to the programmer that they cannot ignore, which indicates that there is an error. That something is called an "exception".
-Take a look at this simple code that sets the height...
- -def setHeight(height):
- if height < 0 or height > 300:
- raise ValueError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(-5)
-When we try to use an invalid value for the height, we raise (or throw) a ValueError exception. This stops the function from continuing, and gives us a very helpful print out of what went wrong, and where.
ValueError is just a class. The name of the class provides us with useful information (there was an error with a value in the program). You choose what error you want to raise. Python provides a set of usefully named error classes that you can use:
-
-
IOError: Error raised when you have a problem with IO, e.g. opening or closing files
-ZeroDivisionError: Error raised when you divide by zero
-TypeError: Error raised when you are using the wrong type, e.g. maybe setting the height to a string
-IndexError: Error raised when you are using an invalid index to access a list or other similar container
-KeyError: Error raised when you are using an invalid key to access a dictionary or other similar container
-
A full list of standard Python exceptions is available here.
-You are free to raise any exception class you want. It is your job as a programmer to choose the one that is most sensible, e.g.
- -def setHeight(height):
- if height < 0 or height > 300:
- raise ZeroDivisionError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(400)
-Using a ZeroDivisionError is a bad choice, as the error has nothing to do with division by zero. A ValueError is the right choice as the error relates to an invalid value passed to the function.
You are free to create your own exception classes.
- -class InvalidHeightError(Exception):
- pass
-def setHeight(height):
- if height < 0 or height > 300:
- raise InvalidHeightError("Invalid height: %s. This should be between 0 and 300" % height)
-
- print("Height is set to %s" % height)
-setHeight(-10)
-Your own exception classes must be declared as derived from type Exception, hence why you have to write class InvalidHeightError(Exception):. As the class doesn't need to do anything else, you can use pass to say that nothing else needs to be added. Note that you can call your error class anything you want. By convention, it is good to end the class name with Error so that other programmers know what it is for.
Exercise¶
Here is an extended copy of the Person code from above.
class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This shoud be between 0 and 2.5 meters" % height)
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- if weight < 0 or weight > 500:
- raise ValueError("Invalid weight: %s. This should be between 0 and 500 kilograms" % weight)
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- return self.getWeight() / self.getHeight()**2
-Exercise 1¶
Edit the above copy of Person to ensure that the .setWeight function only accepts valid weights. A valid weight is any number that is between 0 and 500 kilograms. You should raise a ValueError if the weight is outside this range. For the moment, do not worry about the user supplying a non-numeric weight.
Also edit the above copy of Person to ensure that the .setHeight function only accepts valid heights. A valid height is any number that is between 0 and 2.5 meters. You should raise a ValueError if the height is outside this range. For the moment, do not worry about the user supplying a non-numeric height.
Check that a ValueError exception is correctly raised if invalid heights or weights are supplied. Also check that the ValueError exception is not raised if a valid height and weight are supplied.
p = Person(height=2.8, weight=500)
-p = Person(height=1.8, weight=501)
-p = Person(height=1.8, weight=150)
-Exercise 2¶
If you run the following code;
-p = Person()
-p.bmi()
-it will raise a DivideByZero exception. This is because the calculation involves dividing by the height squared, which is zero in a default-constructed Person. While an exception has been raised, it is not very intuitive for another programmer to debug. A solution is to create your own named exception that provides more information.
Create a new exception called NullPersonError, and edit the .bmi() function so that this exception is raised if it is called on a Person whose height or weight is zero.
Check that the NullPersonError exception is raised if .bmi() is called on a default-constructed Person. Check that this exception is not raised if .bmi() is called on a properly constructed Person.
class NullPersonError(Exception):
- pass
-class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This shoud be between 0 and 2.5 meters" % height)
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- if weight < 0 or weight > 500:
- raise ValueError("Invalid weight: %s. This should be between 0 and 500 kilograms" % weight)
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- if (self.getHeight() == 0 or self.getWeight() == 0):
- raise NullPersonError("Cannot calculate the BMI of a person with zero "
- "height or weight (%s,%s)" % (self.getHeight(),self.getWeight()))
-
- return self.getWeight() / self.getHeight()**2
-p = Person()
-p.bmi()
-p = Person(height=1.8, weight=77.5)
-p.bmi()
-
-Error Handling¶
Exceptions are useful for more than just signalling errors. They can also be used to help you handle the error, and potentially even fix the problem (true self-healing program!).
-Consider this cut down version of the .setHeight function from the last session...
def setHeight(height):
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-The code currently correctly detects if the user supplies a height that is below 0 or above 2.5. However, what about when the user tries to set the height to something that is not a number?
- -setHeight("cat")
-We get a weird error message that says we have a TypeError, as you cannot order a string and an integer.
One way to address this is to ask that height is converted to a float, using height = float(height)
def setHeight(height):
- height = float(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-However, this hasn't made the error any easier to understand, as we now get a ValueError raised...
setHeight("cat")
-The solution is for us to handle the exception, using a try...except block
def setHeight(height):
- try:
- height = float(height)
- except:
- raise TypeError("Invalid height: '%s'. You can only set the height to a numeric value" % height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-setHeight("cat")
-What's happened here? The try: line starts a try-block. The code that is in the try-block is run. If any of this code raises an exception, then execution stops in the try-block, and switches instead to the code in the except-block (everything within the except: block). In our case, float(height) raised an exception, so execution jumped to the except-block, in which we ran the raise TypeError(...) code.
Now the error is much more informative, allowing the user to better understand what has gone wrong. However, exception handling can do more than this. It can allow you to fix the problem. Consider this example...
- -setHeight("1.8 m")
-We as humans can see that this could be an acceptable input. However, the computer needs help to understand. We can add code to the except-block that can try to resolve the problem. For example, imagine we had a function that could interpret heights from strings...
- -def string_to_height(height):
- """This function tries to interpret the passed argument as a height
- in meters. The format should be 'X m', 'X meter' or 'X meters',
- where 'X' is a number
- """
- # convert height to a string - this always works
- height = str(height)
-
- words = height.split(" ")
-
- if len(words) == 2:
- if words[1] == "m" or words[1] == "meter" or words[1] == "meters":
- try:
- return float(words[0])
- except:
- pass
-
- # Getting here means that we haven't been able to extract a valid height
- raise TypeError("Cannot extract a valid height from '%s'" % height)
-We can now call this function from within the except-block of setHeight
def setHeight(height):
- try:
- height = float(height)
- except:
- height = string_to_height(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This should be between 0 and 2.5 m" % height)
- print("setting the height to %s" % height)
-setHeight("1.8 m")
-Exercise¶
Exercise 1¶
Here is a copy of the Person class from the last session. Edit the setHeight function so that it uses exception handling and the string_to_height function to correctly interpret heights such as "1.8 m", and so that it gives a useful error message if it is given something weird. Check that the function correctly responds to a range of valid and invalid inputs.
class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- try:
- height = float(height)
- except:
- height = string_to_height(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This shoud be between 0 and 2.5 meters" % height)
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- if weight < 0 or weight > 500:
- raise ValueError("Invalid weight: %s. This should be between 0 and 500 kilograms" % weight)
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- if (self.getHeight() == 0 or self.getWeight() == 0):
- raise NullPersonError("Cannot calculate the BMI of a person with zero "
- "height or weight (%s,%s)" % (self.getHeight(),self.getWeight()))
-
- return self.getWeight() / self.getHeight()**2
-p = Person(height="cat", weight=20)
-Exercise 2¶
Create a string_to_weight function that interprets weights in kilograms (e.g. "5 kg", "5 kilos" or "5 kilograms"). Now edit the Person.setWeight function so that it uses exception handling and string_to_weight to to correctly interpret weights such as 35.5 kg and gives a useful error message if it is given something weird. Check that your function responds correctly to a range of valid and invalid inputs.
def string_to_weight(weight):
- """This function tries to interpret the passed argument as a weight
- in kilograms. The format should be 'X kg' 'X kilogram' or 'X kilograms',
- where 'X' is a number
- """
- # convert weight to a string - this always works
- weight = str(weight)
-
- words = weight.split(" ")
-
- if len(words) == 2:
- if words[1] == "kg" or words[1] == "kilogram" or words[1] == "kilograms" \
- or words[1] == "kilo" or words[1] == "kilos":
- try:
- return float(words[0])
- except:
- pass
-
- # Getting here means that we haven't been able to extract a valid weight
- raise TypeError("Cannot extract a valid weight from '%s'" % weight)
-class Person:
- """Class that holds a person's height"""
- def __init__(self, height=0, weight=0):
- """Construct a person with the specified name, height and weight"""
- self.setHeight(height)
- self.setWeight(weight)
-
- def setHeight(self, height):
- """Set the person's height in meters"""
- try:
- height = float(height)
- except:
- height = string_to_height(height)
-
- if height < 0 or height > 2.5:
- raise ValueError("Invalid height: %s. This shoud be between 0 and 2.5 meters" % height)
- self._height = height
-
- def setWeight(self, weight):
- """Set the person's weight in kilograms"""
- try:
- weight = float(weight)
- except:
- weight = string_to_weight(weight)
- if weight < 0 or weight > 500:
- raise ValueError("Invalid weight: %s. This should be between 0 and 500 kilograms" % weight)
- self._weight = weight
-
- def getHeight(self):
- """Return the person's height in meters"""
- return self._height
-
- def getWeight(self):
- """Return the person's weight in kilograms"""
- return self._weight
-
- def bmi(self):
- """Return the person's body mass index (bmi)"""
- if (self.getHeight() == 0 or self.getWeight() == 0):
- raise NullPersonError("Cannot calculate the BMI of a person with zero "
- "height or weight (%s,%s)" % (self.getHeight(),self.getWeight()))
-
- return self.getWeight() / self.getHeight()**2
-p = Person(weight="55.6 kilos", height="1.5 meters")
-
-Modules¶
You can turn any Python script that you write into a module that other people can import and use in their own code.
-For example;
- -import superhero
-What has happened here???
-There is a file in your current directory called superhero.py. The line import superhero will look in the current directory, to find a file called superhero.py. It then runs this file, just as if you had typed it into the screen.
This is just a simple Python script, which we can print out using
- -! cat superhero.py
-We can get help on the module using help
help(superhero)
-This documentation comes from the class and function documentation put into the file.
-You can also use the data, classes and functions in the file, e.g.
- -ironman = superhero.Superhero(name="Iron Man", weakness="rust")
-superhero.battle(ironman, superhero.lex)
-superhero.lex.steal("rust")
-superhero.battle(ironman, superhero.lex)
-One thing to note is that all of the classes, functions and data in the script has been imported into its own namespace, named after the script (e.g. superhero.). We can import the file and put all names into the current namespace using
from superhero import *
-battle(ironman, lex)
-While any python script can be imported as a module, there are a few conventions you should follow that will make your module easier for others to use.
--
-
- Add documentation to the module. As you can see, there is a docstring at the top of
superhero.py, which is the first thing written out by help(). This should provide an overview of the module.
- - Avoid actually running any code or creating any variables. The current
superhero.pyis bad as it does this, which is why you see "Is it a bird..." printed when you import it!
-
The way to avoid creating any variables or running code is to let the script detect when it is being imported, and to not create any variables if that is the case.
-You can detect if your Python script is not being imported using
-if __name__ == "__main__":
- print("I am not being imported.")
-if __name__ == "__main__":
- print("I am not being imported")
-To show how this works, there is a superhero2.py script, which is identical to superhero.py, except all code that should not be run on import is hidden inside the if __name__ == "__main__": block.
! cat superhero2.py
-import superhero2
-By using if __name__ == "__main__": we have prevented superhero2.py from printing anything out when it is imported, and have also prevented it from creating the variables lex and superman.
You can see this by running the superhero2.py script directory, e.g. using
! python superhero2.py! python superhero2.py
-Exercise¶
Exercise 1¶
Use the "New Text File" option in the Jupyter Home to create a new python text file called morse.py. Copy the below class into this file.
class Morse:
- def __init__(self):
- self._letter_to_morse = {'a':'.-', 'b':'-...', 'c':'-.-.', 'd':'-..', 'e':'.', 'f':'..-.',
- 'g':'--.', 'h':'....', 'i':'..', 'j':'.---', 'k':'-.-', 'l':'.-..', 'm':'--',
- 'n':'-.', 'o':'---', 'p':'.--.', 'q':'--.-', 'r':'.-.', 's':'...', 't':'-',
- 'u':'..-', 'v':'...-', 'w':'.--', 'x':'-..-', 'y':'-.--', 'z':'--..',
- '0':'-----', '1':'.----', '2':'..---', '3':'...--', '4':'....-',
- '5':'.....', '6':'-....', '7':'--...', '8':'---..', '9':'----.',
- ' ':'/' }
-
- self._morse_to_letter = {}
- for letter in self._letter_to_morse.keys():
- self._morse_to_letter[ self._letter_to_morse[letter] ] = letter
-
- def encode(self, message):
- morse = []
- for letter in message:
- morse.append( self._letter_to_morse[letter.lower()] )
- return morse
-
- def decode(self, morse):
- message = []
- for code in morse:
- message.append( self._morse_to_letter[code] )
- return "".join(message)
-Add documentation to this class, and to the module. Next, import the module and get help using the commands
-import morse
-help(morse)
-Does your documentation make sense?
- -import morse
-help(morse)
-Exercise 2¶
Create some checks of your module that should not be run when the module is imported (i.e. only run directly). The checks should be, e.g.
-morse = Morse()
-
- for message in ["Hello world", "something to encode", "test message"]:
- test = morse.decode( morse.encode(message) )
-
- if message.lower() == test:
- print("Success: %s" % message)
- else:
- print("Failed: %s" % message)
-Validate that the check doesn't run on import using
-import morse
-Validate that the check runs from the command line using
- -! python morse.py! python morse.py
-
-Pandas¶
Pandas is a library providing high-performance, easy-to-use data structures and data analysis tools. The core of pandas is its dataframe which is essentially a table of data. Pandas provides easy and powerful ways to import data from a variety of sources and export it to just as many. It is also explicitly designed to handle missing data elegantly which is a very common problem in data from the real world.
-The offical pandas documentation is very comprehensive and you will be answer a lot of questions in there, however, it can sometimes be hard to find the right page. Don't be afraid to use Google to find help.
- -Pandas has a standard convention for importing it which you will see used in a lot of documentation so we will follow that in this course:
- -import pandas as pd
-from pandas import Series, DataFrame
-Series¶
The simplest of pandas' data structures is the Series. It is a one-dimensional list-like structure.
-Let's create one from a list:
Series([14, 7, 3, -7, 8])
-There are three main components to this output.
-The first column (0, 2, etc.) is the index, by default this is numbers each row starting from zero.
-The second column is our data, stored i the same order we entered it in our list.
-Finally at the bottom there is the dtype which stands for 'data type' which is telling us that all our data is being stored as a 64-bit integer.
-Usually you can ignore the dtype until you start doing more advanced things.
In the first example above we allowed pandas to automatically create an index for our Series (this is the 0, 1, 2, etc. in the left column) but often you will want to specify one yourself
s = Series([14, 7, 3, -7, 8], index=['a', 'b', 'c', 'd', 'e'])
-print(s)
-We can use this index to retrieve individual rows
- -s['a']
-to replace values in the series
- -s['c'] = -1
-or to get a set of rows
- -s[['a', 'c', 'd']]
-Exercise 1¶
-
-
- Create a Pandas
Serieswith 10 or so elements where the indices are years and the values are numbers.
- - Experiment with retrieving elements from the
Series.
- - Try making another
Serieswith duplicate values in the index, what happens when you access those elements?
- - How does a Pandas
Seriesdiffer from a Pythonlistordict?
-
# Answer
-
-ex1_1 = Series([4, 7.1, 7.3, 7.8, 8.1], index=[2000, 2001, 2002, 2003, 2004])
-
-ex1_1
-# Answer
-
-ex1_1[2002]
-# Answer
-
-ex1_1[[2001, 2004]]
-# Answer
-
-ex1_2 = Series([4, 7.1, 7.3, 7.8, 8.1], index=[2000, 2001, 2004, 2001, 2004])
-
-ex1_2
-# Answer
-
-ex1_2[2001]
-Series operations¶
A Series is list-like in the sense that it is an ordered set of values. It is also dict-like since its entries can be accessed via key lookup. One very important way in which is differs is how it allows operations to be done over the whole Series in one go, a technique often referred to as 'broadcasting'.
A simple example is wanting to double the value of every entry in a set of data. In standard Python, you might have a list like
- -my_list = [3, 6, 8, 4, 10]
-If you wanted to double every entry you might try simply multiplying the list by 2:
my_list * 2
-but as you can see, that simply duplicated the elements. Instead you would have to use a for loop or a list comprehension:
[i * 2 for i in my_list]
-With a pandas Series, however, you can perform bulk mathematical operations to the whole series in one go:
my_series = Series(my_list)
-print(my_series)
-my_series * 2
-As well as bulk modifications, you can perform bulk selections by putting more complex statements in the square brackets:
- -s[s < 0] # All negative entries
-s[(s * 2) > 4] # All entries which, when doubled are greater than 4
-These operations work because the Series index selection can be passed a series of True and False values which it then uses to filter the result:
(s * 2) > 4
-Here you can see that the rows a, b and e are True while the others are False. Passing this to s[...] will only show rows that are True.
Multi-Series operations¶
It is also possible to perform operations between two Series objects:
s2 = Series([23,5,34,7,5])
-s3 = Series([7, 6, 5,4,3])
-s2 - s3
-Exercise 2¶
-
-
- Create two
Seriesobjects of equal length with no specified index and containing any values you like. Perform some mathematical operations on them and experiment to make sure it works how you think.
- - What happens then you perform an operation on two series which have different lengths? How does this change when you give the series some indices? -
- Using the
Seriesfrom the first exercise with the years for the index, Select all entries with even-numbered years. Also, select all those with odd-numbered years.
-
# Answer
-
-ex2_1a = Series([1,3,5,7,4,3])
-ex2_1b = Series([3,7,9,4,2,4])
-
-ex2_1a + ex2_1b
-# Answer
-
-ex2_2a = Series([1,3,5,7,4,3])
-ex2_2b = Series([3,7,9,4])
-
-ex2_2a + ex2_2b
-# Answer
-
-ex2_2a = Series([1,3,5,7,4,3], index=[1,2,3,4,5,6])
-ex2_2b = Series([3,7,9,4], index=[1,3,5,7])
-
-ex2_2a + ex2_2b
-ex1_1[ex1_1.index % 2 == 0]
-DataFrame¶
While you can think of the Series as a one-dimensional list of data, pandas' DataFrame is a two (or possibly more) dimensional table of data. You can think of each column in the table as being a Series.
data = {'city': ['Paris', 'Paris', 'Paris', 'Paris',
- 'London', 'London', 'London', 'London',
- 'Rome', 'Rome', 'Rome', 'Rome'],
- 'year': [2001, 2008, 2009, 2010,
- 2001, 2006, 2011, 2015,
- 2001, 2006, 2009, 2012],
- 'pop': [2.148, 2.211, 2.234, 2.244,
- 7.322, 7.657, 8.174, 8.615,
- 2.547, 2.627, 2.734, 2.627]}
-df = DataFrame(data)
-This has created a DataFrame from the dictionary data. The keys will become the column headers and the values will be the values in each column. As with the Series, an index will be created automatically.
df
-Or, if you just want a peek at the data, you can just grab the first few rows with:
- -df.head(3)
-Since we passed in a dictionary to the DataFrame constructor, the order of the columns will not necessarilly match the order in which you defined them. To enforce a certain order, you can pass a columns argument to the constructor giving a list of the columns in the order you want them:
DataFrame(data, columns=['year', 'city', 'pop'])
-When we accessed elements from a Series object, it would select an element by row. However, by default DataFrames index primarily by column. You can access any column directly by using square brackets or by named attributes:
df['year']
-df.city
-Accessing a column like this returns a Series which will act in the same way as those we were using earlier.
Note that there is one additional part to this output, Name: city. Pandas has remembered that this Series was created from the 'city' column in the DataFrame.
type(df.city)
-df.city == 'Paris'
-This has created a new Series which has True set where the city is Paris and False elsewhere.
We can use filtered Series like this to filter the DataFrame as a whole. df.city == 'Paris' has returned a Series containing booleans. Passing it back into df as an indexing operation will use it to filter based on the 'city' column.
df[df.city == 'Paris']
-You can then carry on and grab another column after that filter:
- -df[df.city == 'Paris'].year
-If you want to select a row from a DataFrame then you can use the .loc attribute which allows you to pass index values like:
df.loc[2]
-df.loc[2]['city']
-Adding new columns¶
New columns can be added to a DataFrame simply by assigning them by index (as you would for a Python dict) and can be deleted with the del keyword in the same way:
df['continental'] = df.city != 'London'
-df
-del df['continental']
-Exercise 3¶
-
-
- Create the
DataFramecontaining the census data for the three cities.
- - Select the data for the year 2001. Which city had the smallest population that year? -
- Find all the cities which had a population smaller than 2.6 million. -
# Answer
-
-print(df[df.year == 2001])
-# Answer
-
-print(df[df['pop'] < 2.6])
-Reading from file¶
One of the msot common situations is that you have some data file containing the data you want to read. Perhaps this is data you've produced yourself or maybe it's from a collegue. In an ideal world the file will be perfectly formatted and will be trivial to import into pandas but since this is so often not the case, it provides a number of features to make your ife easier.
-Full information on reading and writing is available in the pandas manual on IO tools but first it's worth noting the common formats that pandas can work with:
--
-
- Comma separated tables (or tab-separated or space-separated etc.) -
- Excel spreadsheets -
- HDF5 files -
- SQL databases -
For this course we will focus on plain-text CSV files as they are perhaps the most common format. Imagine we have a CSV file like (you can download this file from city_pop.csv):
- -! cat data/city_pop.csv # Uses the IPython 'magic' !cat to print the file
-We can use the pandas function read_csv() to read the file and convert it to a DataFrame. Full documentation for this function can be found in the manual or, as with any Python object, directly in the notebook by putting a ? after the name:
help(pd.read_csv)
-pd.read_csv('data/city_pop.csv')
-We can see that by default it's done a fairly bad job of parsing the file (this is mostly because I;ve construsted the city_pop.csv file to be as obtuse as possible). It's making a lot of assumptions about the structure of the file but in general it's taking quite a naïve approach.
The first this we notice is that it's treating the text at the top of the file as though it's data. Checking the documentation we see that the simplest way to solve this is to use the skiprows argument to the function to which we give an integer giving the number of rows to skip:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
-)
-The next most obvious problem is that it is not separating the columns at all. This is controlled by the sep argument which is set to ',' by default (hence comma separated values). We can simply set it to the appropriate semi-colon:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';'
-)
-Reading the descriptive header of our data file we see that a value of -1 signifies a missing reading so we should mark those too. This can be done after the fact but it is simplest to do it at import-time using the na_values argument:
pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';',
- na_values='-1'
-)
-The last this we want to do is use the year column as the index for the DataFrame. This can be done by passing the name of the column to the index_col argument:
df3 = pd.read_csv(
- 'data/city_pop.csv',
- skiprows=5,
- sep=';',
- na_values='-1',
- index_col='year'
-)
-df3
-Exercise 4¶
-
-
- Alongside
data/city_pop.csvthere is another file calleddata/cetml1659on.dat. This contains some historical weather data for a location in the UK. Import that file as a PandasDataFrameusingread_csv(), making sure that you cover all the NaN values.
- - How many years had a negative average temperature in January? -
- What was the average temperature in June over the years in the data set? Tip: look in the documentation for which method to call. -
We will come back to this data set in a later stage.
- -# Answer
-
-weather = pd.read_csv(
- 'data/cetml1659on.dat', # file name
- skiprows=6, # skip header
- sep='\s+', # whitespace separated
- na_values=['-99.9', '-99.99'], # NaNs
-)
-weather.head()
-# Answer
-
-len(weather[weather.JAN < 0])
-# Answer
-
-weather.JUN.mean()
-
-Basic NumPy¶
NumPy ('Numerical Python') is the defacto standard module for doing numerical work in Python. Its main feature is its array data type which allows very compact and efficient storage of homogenous (of the same type) data.
-A lot of the material in this section is based on SciPy Lecture Notes (CC-by 4.0).
-As you go through this material, you'll likely find it useful to refer to the NumPy documentation, particularly the array objects section.
-As with pandas there is a standard convention for importing numpy, and that is as np:
import numpy as np
-Now that we have access to the numpy package we can start using its features.
Creating arrays¶
In many ways a NumPy array can be treated like a standard Python list and much of the way you interact with it is identical. Given a list, you can create an array as follows:
python_list = [1, 2, 3, 4, 5, 6, 7, 8]
-numpy_array = np.array(python_list)
-print(numpy_array)
-# ndim give the number of dimensions
-numpy_array.ndim
-# the shape of an array is a tuple of its length in each dimension. In this case it is only 1-dimensional
-numpy_array.shape
-# as in standard Python, len() gives a sensible answer
-len(numpy_array)
-nested_list = [[1, 2, 3], [4, 5, 6]]
-two_dim_array = np.array(nested_list)
-print(two_dim_array)
-two_dim_array.ndim
-two_dim_array.shape
-It's very common when working with data to not have it already in a Python list but rather to want to create some data from scratch. numpy comes with a whole suite of functions for creating arrays. We will now run through some of the most commonly used.
The first is np.arange (meaning "array range") which works in a vary similar fashion the the standard Python range() function, including how it defaults to starting from zero, doesn't include the number at the top of the range and how it allows you to specify a 'step:
np.arange(10) #0 .. n-1 (!)
-np.arange(1, 9, 2) # start, end (exclusive), step
-Next up is the np.linspace (meaning "linear space") which generates a given floating point numbers starting from the first argument up to the second argument. The third argument defines how many numbers to create:
np.linspace(0, 1, 6) # start, end, num-points
-Note how it included the end point unlike arange(). You can change this feature by using the endpoint argument:
np.linspace(0, 1, 5, endpoint=False)
-np.ones creates an n-dimensional array filled with the value 1.0. The argument you give to the function defines the shape of the array:
np.ones((3, 3)) # reminder: (3, 3) is a tuple
-Likewise, you can create an array of any size filled with zeros:
- -np.zeros((2, 2))
-The np.eye (referring to the matematical identity matrix, commonly labelled as I) creates a square matrix of a given size with 1.0 on the diagonal and 0.0 elsewhere:
np.eye(3)
-The np.diag creates a square matrix with the given values on the diagonal and 0.0 elsewhere:
np.diag([1, 2, 3, 4])
-Finally, you can fill an array with random numbers:
- -np.random.rand(4) # uniform in [0, 1]
-np.random.randn(4) # Gaussian
-Exercise 5¶
-
-
- Experiment with
arange,linspace,ones,zeros,eyeanddiag.
- - Create different kinds of arrays with random numbers. -
- Look at the function
np.empty. What does it do? When might this be useful?
-
Reshaping arrays¶
Behind the scenes, a multi-dimensional NumPy array is just stored as a linear segment of memory. The fact that it is presented as having more than one dimension is simply a layer on top of that (sometimes called a view). This means that we can simply change that interpretive layer and change the shape of an array very quickly (i.e without NumPy having to copy any data around).
This is mostly done with the reshape() method on the array object:
my_array = np.arange(16)
-my_array
-my_array.shape
-my_array.reshape((2, 8))
-my_array.reshape((4, 4))
-Note that if you check, my_array.shape will still return (16,) as reshaped is simply a view on the original data, it hasn't actually changed it. If you want to edit the original object in-place then you can use the resize() method.
You can also transpose an array using the transpose() method which mirrors the array along its diagonal:
my_array.reshape((2, 8)).transpose()
-my_array.reshape((4,4)).transpose()
-Exercise 6¶
Using the NumPy documentation at https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html, to create, in one line a NumPy array which looks like:
-[10, 60, 20, 70, 30, 80, 40, 90, 50, 100]
-Hint: you will need to use transpose(), reshape() and arange() as well as one new function from the "Shape manipulation" section of the documentation. Can you find a method which uses less than 4 function calls?
# Answer
-
-np.arange(10, 110, 10).reshape(2, 5).transpose().ravel()
-More Numpy¶
Carrying on from the last lesson we will continue learning how to manipulate data in numpy before using matplotlib to plot our data.
import numpy as np
-Basic data types¶
You may have noticed that, in some instances, array elements are displayed with a trailing dot (e.g. 2. vs 2). This is due to a difference in the data-type used:
a = np.array([1, 2, 3])
-a.dtype
-b = np.array([1., 2., 3.])
-b.dtype
-Different data-types allow us to store data more compactly in memory, but most of the time we simply work with floating point numbers. Note that, in the example above, NumPy auto-detects the data-type from the input but you can specify it explicitly:
- -c = np.array([1, 2, 3], dtype=float)
-c.dtype
-The default data type is floating point.
- -d = np.ones((3, 3))
-d.dtype
-There are other data types as well:
- -e = np.array([1+2j, 3+4j, 5+6*1j])
-type(1j)
-#e.dtype
-f = np.array([True, False, False, True])
-f.dtype
-g = np.array(['Bonjour', 'Hello', 'Hallo',])
-g.dtype # <--- strings containing max. 7 letters
-We previously came across dtypes when learing about pandas. This is because pandas uses NumPy as its underlying library. A pandas.Series is essentially a np.array with some extra features wrapped around it.
Exercise 1¶
Recreate some of the arrays we created in yesterday's session and look at what dtype they have.
- -Why NumPy¶
To show some of the advantages of NumPy over a standard Python list, let's do some benchmarking. It's an important habit in programming that whenever you think one method may be faster than another, you check to see whether your assumption is true.
-Python provides some tools to make this easier, particularly via the timeit module. Using this functionality, IPython provides a %timeit magic function to make our life easier. To use the %timeit magic, simply put it at the beginning of a line and it will give you information about how ling it took to run. It doesn't always work as you would expect so to make your life easier, put whatever code you want to benchmark inside a function and time that function call.
We start by making a list and an array of 10000 items each of values counting from 0 to 9999:
- -python_list = list(range(100000))
-numpy_array = np.arange(100000)
-We are going to go through each item in the list and double its value in-place, such that the list is changed after the operation. To do this with a Python list we need a for loop:
def python_double(a):
- for i, val in enumerate(a):
- a[i] = val * 2
-
-%timeit python_double(python_list)
-To do the same operation in NumPy we can use the fact that multiplying a NumPy array by a value will apply that operation to each of its elements:
def numpy_double(a):
- a *= 2
-
-%timeit numpy_double(numpy_array)
-As you can see, the NumPy version is at least 10 times faster, sometimes up to 100 times faster.
-Have a think about why this might be, what is NumPy doing to make this so much faster? There are two main parts to the answer.
- -Copies and views¶
A slicing operation (like reshaping before) creates a view on the original array, which is just a way of accessing array data. Thus the original array is not copied in memory. This means you can do this to large arrays without any great performance hit. You can use np.may_share_memory() to check if two arrays share the same memory block. Note however, that this uses heuristics and may give you false positives.
When modifying the view, the original array is modified as well:
- -a = np.arange(10)
-a
-b = a[3:7]
-
-np.may_share_memory(a, b)
-b[0] = 12
-b
-a # (!)
-a = np.arange(10)
-c = a[::2].copy() # force a copy
-c[0] = 12
-a
-np.may_share_memory(a, c) # we made a copy so there is no shared memory
-Whether you make a view or a copy can affect the speed of your code significantly. Be in the habit of checking whether your code is doing unnecessacy work. Also, be sure to benchmark your code as you work on it so that you notice any slowdowns and so that you know which parts are slow so you speed the right bits up.
- -Exercise 2¶
-
-
- Using
%timeit, time how long finding the square roots of a list of numbers would take under both standard Python and numpy.-
-
- Tip: Python's square root function is
math.sqrt. numpy's isnp.sqrt.
-
- - Tip: Python's square root function is
# Answer
-
-import math
-
-python_list_2 = list(range(100000))
-
-def python_sqrt(a):
- for i, val in enumerate(a):
- a[i] = math.sqrt(val)
-
-%timeit python_sqrt(python_list)
-# Answer
-
-numpy_array_2 = np.arange(100000)
-
-def numpy_sqrt(a):
- np.sqrt(a)
-
-%timeit numpy_sqrt(numpy_array)
-
-Plotting data with matplotlib¶
Plotting of data is pandas is handled by an external Python module called matplotlib. Like pandas it is a large library and has a venerable history (first released in 2003) and so we couldn't hope to cover all its functionality in this course. To see the wide range of possibilities you have with matplotlib see its example gallery.
-Here we will cover the basic uses of it and how it integrates with pandas. While working through these examples you will likely find it very useful to refer to the matplotlib documentation.
- -First we import pandas and numpy in the same way as we did previously.
- -import numpy as np
-import pandas as pd
-from pandas import Series, DataFrame
-Some matplotlib functionality is provided directly through pandas (such as the plot() method as we will see) but for much of it you need to import the matplotlib interface itself.
The most common interface to matplotlib is its pyplot module which provides a way to affect the current state of matplotlib directly. By convention this is imported as plt.
We also set the figure format to be SVG so that the plots look a little nicer in our Jupyter notebook.
- -import matplotlib.pyplot as plt
-%config InlineBackend.figure_format = 'svg'
-Once we have imported matplotlib we can start calling its functions. Any functions called on the plt object will affect all of matplotlib from that point on in the script.
We first need to import some data to plot. Let's start with the data from the pandas section (available from cetml1659on.dat) and import it into a DataFrame:
df = pd.read_csv(
- 'data/cetml1659on.dat', # file name
- skiprows=6, # skip header
- sep='\s+', # whitespace separated
- na_values=['-99.9', '-99.99'], # NaNs
-)
-df.head()
-Pandas integrates matplotlib directly into itself so any dataframe can be plotted easily simply by calling the plot() method on one of the columns. This creates a plot object which you can then edit and alter, for example by setting the axis labels using the plt.ylabel() function before displaying it with plt.show().
Matplotlib operates on a single global state and calling any function on plt will alter that state. Calling df.plot() sets the currently operating plot. plt.ylabel() then alters that state and plt.show() displays it.
df['JAN'].plot()
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-Exercise 1¶
-
-
- Make sure you can reproduce the plot above. Try tweaking the labels or which column is plotted. -
- Try putting in two
plot()calls with different months (January and July for example) before callingshow().
-
# Answer
-
-df['JUL'].plot()
-df['JAN'].plot()
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-Making it prettier¶
-While it's useful to be able to quickly plot any data we have in front of us, matplotlib's power comes from its configurability. Let's experiment with a dataset and see how much we can change the plot.
-We'll start with a simple DataFrame contianing two columns, one with the values of a cosine, the other with the values of a sine.
X = np.linspace(-np.pi, np.pi, 256, endpoint=True)
-data = {'cos': np.cos(X), 'sin': np.sin(X)}
-trig = DataFrame(index=X, data=data)
-
-trig.plot()
-plt.show()
-You can see that it has plotted the sine and cosine curves between $\pi$ and $-\pi$. Now, let's go through and see how we can affect the display of this plot.
- -Changing colours and line widths¶
First step, we want to have the cosine in blue and the sine in red and a slighty thicker line for both of them.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.show()
-Exercise 2¶
-
-
- Using the temperature dataset, set the colours of the July and January lines to a warm colour and a cool colour. -
- Add in the yearly average column to the plot with a dashed line style. -
# Answer
-
-df['JUL'].plot(color='orange')
-df['JAN'].plot(color='blue')
-df['YEAR'].plot(linestyle=':')
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.legend(loc='upper left')
-
-plt.show()
-Setting limits¶
Current limits of the figure are a bit too tight and we want to make some space in order to clearly see all data points.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-### New code
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-### End of new code
-
-plt.show()
-Setting ticks¶
Current ticks are not ideal because they do not show the interesting values ($\pm\pi$,$\pm\frac{\pi}{2}$) for sine and cosine. We’ll change them such that they show only these values.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-### New code
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-### End of new code
-
-plt.show()
-Setting tick labels¶
Ticks are now properly placed but their label is not very explicit. We could guess that 3.142 is $\pi$ but it would be better to make it explicit. When we set tick values, we can also provide a corresponding label in the second argument list. Note that we’ll use LaTeX to allow for nice rendering of the label.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-### New code
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-### End of new code
-
-plt.show()
-Moving spines¶
Spines are the lines connecting the axis tick marks and noting the boundaries of the data area. They can be placed at arbitrary positions and until now, they were on the border of the axis. We’ll change that since we want to have them in the middle. Since there are four of them (top/bottom/left/right), we’ll discard the top and right by setting their color to none and we’ll move the bottom and left ones to coordinate 0 in data space coordinates.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-### New code
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-### End of new code
-
-plt.show()
-Adding a legend¶
Let’s add a legend in the upper left corner. This only requires adding the keyword argument label (that will be used in the legend box) to the plot commands.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-
-### New code
-plt.legend(loc='upper left')
-### End of new code
-
-plt.show()
-Annotate some points¶
Let’s annotate some interesting points using the annotate command. We chose the $\frac{2}{3}\pi$ value and we want to annotate both the sine and the cosine. We’ll first draw a marker on the curve as well as a straight dotted line. Then, we’ll use the annotate command to display some text with an arrow.
- -trig.cos.plot(color="blue", linewidth=2.5, linestyle="-")
-trig.sin.plot(color="red", linewidth=2.5, linestyle="-")
-
-plt.xlim(trig.index.min() * 1.1, trig.index.max() * 1.1)
-plt.ylim(trig.cos.min() * 1.1, trig.cos.max() * 1.1)
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi])
-plt.yticks([-1, 0, +1])
-
-plt.xticks([-np.pi, -np.pi/2, 0, np.pi/2, np.pi],
- [r'$-\pi$', r'$-\pi/2$', r'$0$', r'$+\pi/2$', r'$+\pi$'])
-
-plt.yticks([-1, 0, +1],
- [r'$-1$', r'$0$', r'$+1$'])
-
-ax = plt.gca() # gca stands for 'get current axis'
-ax.spines['right'].set_color('none')
-ax.spines['top'].set_color('none')
-ax.xaxis.set_ticks_position('bottom')
-ax.spines['bottom'].set_position(('data',0))
-ax.yaxis.set_ticks_position('left')
-ax.spines['left'].set_position(('data',0))
-
-plt.legend(loc='upper left')
-
-### New code
-t = 2 * np.pi / 3
-plt.plot([t, t], [0, np.cos(t)], color='blue', linewidth=2.5, linestyle="--")
-plt.scatter([t, ], [np.cos(t), ], 50, color='blue')
-
-plt.annotate(r'$cos(\frac{2\pi}{3})=-\frac{1}{2}$',
- xy=(t, np.cos(t)), xycoords='data',
- xytext=(-90, -50), textcoords='offset points', fontsize=16,
- arrowprops=dict(arrowstyle="->", connectionstyle="arc3,rad=.2"))
-
-plt.plot([t, t],[0, np.sin(t)], color='red', linewidth=2.5, linestyle="--")
-plt.scatter([t, ],[np.sin(t), ], 50, color='red')
-
-plt.annotate(r'$sin(\frac{2\pi}{3})=\frac{\sqrt{3}}{2}$',
- xy=(t, np.sin(t)), xycoords='data',
- xytext=(+10, +30), textcoords='offset points', fontsize=16,
- arrowprops=dict(arrowstyle="->", connectionstyle="arc3,rad=.2"))
-### End of new code
-
-plt.show()
-Now you know how to make different modifications to your plots we can make some of these changes to our temerature data.
- -Saving plot to a file¶
You can take any plot you've created within Jupyter and save it to a file on disk using the plt.savefig() function. You give the function the name of the file to create and it will use whatever format is specified by the name.
trig.plot()
-
-plt.show()
-
-plt.savefig('my_fig.svg')
-Exercise 3¶
-
-
- Add in a legend for the data. -
- Add an annotation to one of the spikes in the data. Make sure the label is placed nicely.
-
-
- Tip: you can get the year and temperature for a spike using:-
warm_winter_year = df['JAN'].idxmax() -warm_winter_temp = df['JAN'].max() -
-
- - Tip: you can get the year and temperature for a spike using:
- Save the figure to a file and display it in your Jupyter notebook. -
# Answer
-
-df['JUL'].plot(color='orange')
-df['JAN'].plot(color='blue')
-df['YEAR'].plot(linestyle=':')
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.legend(loc='upper left')
-
-warm_winter_year = df['JAN'].idxmax()
-warm_winter_temp = df['JAN'].max()
-
-plt.annotate(r'A warm winter',
- xy=(warm_winter_year, warm_winter_temp),
- xytext=(-30, +30), textcoords='offset points', fontsize=14,
- arrowprops=dict(arrowstyle="->", connectionstyle="arc3,rad=.2"))
-
-plt.show()
-Bar charts¶
Of course, Matplotlib can plot more than just line graphs. One of the other most common plot types is a bar chart. Let's work towards plotting a bar chart of the average temperature per decade.
-Let's start by adding a new column to the data frame which represents the decade. We create it by taking the index (which is a list of years), converting each element to a string and then replacing the fourth character with a '0'.
years = Series(df.index, index=df.index).apply(str)
-decade = years.apply(lambda x: x[:3]+'0')
-
-df['decade'] = decade
-df.head()
-Once we have our decade column, we can use Pandas groupby() function to gather our data by decade and then aggregate it by taking the mean of each decade.
by_decade = df.groupby('decade')
-agg = by_decade.aggregate(np.mean)
-
-agg.head()
-At this point, agg is a standard Pandas DataFrame so we can plot it like any other, by putting .bar after the plot call:
agg.YEAR.plot.bar()
-
-plt.ylabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-Exercise 4¶
-
-
Plot a bar chart of the average temperature per century.
--
-
- Set the limits of the y-axis to zoom in on the data. -
-Plot a histogram of the average annual temperature
--
-
- Make sure that the x-axis is labelled correctly. -
- Tip: Look in the documentation for the right command to run -
-Plot a scatter plot of each year's February temperature plotted against that year's January temperature. Is there an obvious correlation?
-
-
# Answer
-
-years = Series(df.index, index=df.index).apply(str)
-century = years.apply(lambda x: x[:2]+'00')
-
-df['century'] = century
-
-by_century = df.groupby('century')
-century_avg = by_century.agg(np.mean)
-
-
-century_avg.YEAR.plot.bar()
-
-plt.xlabel(r'Century')
-plt.ylabel(r'Average yearly temperature ($^\circ$C)')
-plt.ylim(8, 10.5)
-
-plt.show()
-# Answer
-
-df.YEAR.plot.hist()
-
-plt.xlabel(r'Temperature ($^\circ$C)')
-
-plt.show()
-# Answer
-
-df.plot.scatter(x='JAN', y='FEB')
-
-plt.xlabel(r'Temperature in January($^\circ$C)')
-plt.ylabel(r'Temperature in February($^\circ$C)')
-
-plt.show()
-
-Viewing Molecules¶
nglview is an extremely powerful and capable 3D molecule view that runs within a web browser. It supports complex visualisations of molecules from a range of file formats, and can even be used to view trajectories. It provides a full framework for building 3D molecular visualisation into your Jupyter notebooks or websites.
-While nglview is very powerful, that power and flexibility can be a little daunting for newcomers. BioSimSpace is a project that provides easy-to-use wrappers around common molecular simulation tasks. One such task is viewing molecules. BioSimSpace provides the function viewMolecules that uses nglview to do exactly that :-)
from BioSimSpace import viewMolecules
-v = viewMolecules("data/dioxin.pdb")
-The above code has use the molecule file parsers built into BioSimSpace to load the molecule contained in dioxin.pdb. This is then rendered using nglview. The above nglview interface allows you to rotate the molecule (left click and drag), zoom in and out (pinch or scroll up or down) and translate (right click and drag, or control+click on a Mac).
Try moving and rotating the molecule. If you lose the molecule, click the "Center" button in the General tab to recenter the molecule.
-
The BioSimSpace viewMolecules function has done two things:
-
-
- it first loaded the molecule(s) from the file, -
- and it then rendered them -
Loading molecules can take a long time and use a lot of memory. To prevent you from having to repeatedly load molecules, the viewMolecules function has returned a view object that can be re-used. To see how to use it, use python's help...
help(v)
-As you can see, we can use v.system() to view all of the loaded molecules again, without having to reload them.
v.system()
-You can change the representation of the molecule by clicking on the "Representation" tab. First click the "Remove" icon to remove the current representation. Then click the drop-down representation box to choose another representation (e.g. "spacefill"). Then click the "Add" icon to add that representation. Experiment with adding and removing different representations.
-
Loading lots of molecules¶
nglview isn't just limited to viewing small molecules. It also works really well as a viewer for large molecular systems. It (sometimes) is sufficiently clever to select appropriate representations for the molecules being loaded.
-For example, view the protein-ligand complex in data/complex.pdb
v = viewMolecules("data/complex.pdb")
-In this case, nglview has automatically selected a cartoon representation for the protein and a ball and stick representation for the ligand.
-You can achieve this yourself by using selections to set different representation for different molecules (or parts of molecules). First, delete the default representations by repeatedly clicking the "Remove" button in the representations tab. Once you have removed all of them, we will add a new representation. Select the type as surface, and then type "protein" into the selection box (which starts off with a "*" in it).
-
Click "Add". After some time thinking, nglview will show you a surface representation of the protein.
-Next, add a "spacefill" representation to the ligand. The ligand residue is called "LIG", so to do this, select "spacefill", type "LIG" into the selection box, and then click add. You should now see the ligand neatly bound into the protein.
-
The selection box can be used to select proteins ("protein"), water ("water"), everything ("*") or residues by name (e.g. "LIG") or number (e.g. "35"). Play around creating different selections and representations. For example, create a "point" representation for water, a "tube" representation of the protein and a "licorice" representation of all alanine residues. Note - you can control the opacity (level of transparency) of a representation by selecting the representation in the drop down box and changing the "opacity" slider in the "Parameters" tab - see below. You can also change things like the colour scheme of the representation in this "Parameters" tab
-
Viewing individual molecules¶
The view object returned by BioSimSpace can be used to view specific molecules from the file. To do this, use the molecules function. This takes a list of indicies of the molecules you want to view. For example, to view the first molecule (molecule at index 0) type;
v.molecules([0])
-while to view molecules 100-1000 use the below code (noting that you may need to add a "ball and stick" represntation in case nglview automatically hides the water molecules).
- -v.molecules( range(100,1000) )
-Loading more complex files¶
BioSimSpace provides reader and writers for a variety of molecular file formats. Some of these split the molecular data over multiple files, e.g. a topology and coordinate file. To view these, pass all of the necessary files to viewMolecules in a list, e.g.
v = viewMolecules(["data/ala.top","data/ala.crd"])
-This can be combined with molecule selection, e.g. to load and view only molecules 0-4 in the file pass the indicies of the molecules you want to view as a second argument to viewMolecule, e.g.
v = viewMolecules(["data/ala.top","data/ala.crd"], [0,1,2,3,4])
-(in reality, all molecules are loaded, but only molecules specified by the indicies are viewed. You can still use v.system() to view all molecules)
v.system()
-
-Regular Expressions¶
In the error handling session we tried to interpret strings as valid heights and weights. This involved looking for text such as "meter" or "kilogram" in the string, and then extracting the number. This process is called pattern matching, and is best undertaken using a regular expression.
-Regular expressions have a long history and are available in most programming languages. Python implements a standards-compliant regular expression module, which is called re.
import re
-Let's create a string that contains a height and see if we can use a regular expression to match that...
- -h = "2 meters"
-To search for string "meters" in a string, using re.search, e.g.
if re.search("meters", h):
- print("String contains 'meters'")
-else:
- print("No match")
-re.search returns a match object if there is a match, or None if there isn't.
m = re.search("meters", h)
-m
-This matches "meters", but what about "meter". "meter" is "meters" without an "s". You can specify that a letter is matched 0 or 1 times using "?"
- -h = "2 meter"
-m = re.search("meters?", h)
-m
-However, this has still not worked, as we match "meters" in the middle of the string. We need to match "meters" only at the end of the string. We do this using "$", which means match at end of string
- -m = re.search("meters?$", h)
-m
-We also want to be able to match "m" as well as "meters". To do this, we need to use the "or" operator, which is "|". It is a good idea to put this in round brackets to make both sides of the "or" statement clear.
- -h = "2 m"
-m = re.search("(m|meters?)$", h)
-m
-Next, we want to match the number, e.g. "X meters", where "X" is a number. You can use "\d" to represent any number. For example
- -h = "2 meters"
-m = re.search("\d (m|meters?)$", h)
-m
-A problem with the above example is that it only matches a number with a single digit, as "\d" only matches a single number. To match one or more digits, we need to put a "+" afterwards, as this means "match one or more", e.g.
- -h = "10 meters"
-m = re.search("\d+ (m|meters?)$", h)
-m
-This match breaks if the number is has decimal point, as it doesn't match the "\d". To match a decimal point, you need to use "\.", and also "?", which means "match 0 or 1 decimal points", and then "\d*", which means "match 0 or more digits"
- -h = "1.5 meters"
-m = re.search("\d+\.?\d* (m|meters?)$", h)
-m
-The number must match at the beginning of the string. We use "^" to mean match at start...
- -h = "some 1.8 meters"
-m = re.search("^\d+\.?\d* (m|meters?)$", h)
-m
-Finally, we want this match to be case insensitive, and would like the user to be free to use as many spaces as they want between the number and the unit, before the string or after the string... To do this we use "\s*" to represent any number of spaces, and match using re.IGNORECASE.
h = " 1.8 METers "
-m = re.search("^\s*\d+\.?\d*\s*(m|meters?)\s*$", h, re.IGNORECASE)
-m
-The round brackets do more than just groups parts of your search. They also allow you extract the parts that match.
- -m.groups()
-You can place round brackets around the parts of the match you want to capture. In this case, we want to get the number...
- -m = re.search("^\s*(\d+\.?\d*)\s*(m|meters?)\s*$", h, re.IGNORECASE)
-m.groups()
-As m.groups()[0] contains the match of the first set of round brackets (which is the number), then we can get the number using m.groups()[0]. This enables us to rewrite the string_to_height function from the last section as;
def string_to_height(height):
- """Parse the passed string as a height. Valid formats are 'X m', 'X meters' etc."""
- m = re.search("^\s*(\d+\.?\d*)\s*(m|meters?)\s*$", height, re.IGNORECASE)
-
- if m:
- return float(m.groups()[0])
- else:
- raise TypeError("Cannot extract a valid height from '%s'" % height)
-h = string_to_height(" 1.5 meters ")
-h
-def string_to_weight(weight):
- """Parse the passed string as a weight. Valid formats are 'X kg', 'X kilos', 'X kilograms' etc."""
- m = re.search("^\s*(\d+\.?\d*)\s*(kgs?|kilos?|kilograms?)\s*$", weight, re.IGNORECASE)
-
- if m:
- return float(m.groups()[0])
- else:
- raise TypeError("Cannot extract a valid weight from '%s'" % weight)
-string_to_weight("23.5 kilos"), string_to_weight("5kg"), string_to_weight("10 kilogram")
-Exercise 2¶
Update string_to_height so that it can also understand heights in both meters and centimeters (returning the height in meters), and update string_to_weight so that it can also understand weights in both grams and kilograms (returning the weight in kilograms). Note that you may find it easier to separate the number from the units. You can do this using the below function to divide the string into the number and units. This uses "\w" to match any word character.
- -def get_number_and_unit(s):
- """Interpret the passed string 's' as "X units", where "X" is a number and
- "unit" is the unit. Returns the number and (lowercased) unit
- """
- m = re.search("^\s*(\d+\.?\d*)\s*(\w+)\s*$", s, re.IGNORECASE)
-
- if m:
- number = float(m.groups()[0])
- unit = m.groups()[1].lower()
- return (number, unit)
- else:
- raise TypeError("Cannot extract a valid 'number unit' from '%s'" % s)
-def string_to_height(height):
- """Parse the passed string as a height. Valid formats are 'X m', 'X centimeters' etc."""
- (number, unit) = get_number_and_unit(height)
-
- if re.search("cm|centimeters?", unit):
- return number / 100.0
-
- elif re.search("m|meters?", unit):
- return number
- else:
- raise TypeError("Cannot convert a number with units '%s' to a valid height" % unit)
-def string_to_weight(weight):
- """Parse the passed string as a weight. Valid formats are 'X kg', 'X grams' etc."""
- (number, unit) = get_number_and_unit(weight)
-
- if re.search("kgs?|kilos?|kilograms?", unit):
- return number
-
- elif re.search("g|grams?", unit):
- return number / 1000.0
- else:
- raise TypeError("Cannot convert a number with units '%s' to a valid weight" % unit)
-string_to_height("55 cm"), string_to_height("2m"), string_to_height("15meters")
-string_to_weight("15g"), string_to_weight("5 kilograms"), string_to_weight("5gram")
-
-