DATA2001: Data Science, Big Data and Data Diversity Data

The University of Sydney Page 1
DATA2001: Data Science,
Big Data and Data Diversity
Data Cleaning and Exploration
with Python
Presented by Alan Fekete
Material prepared by Uwe Roehm
School of Computer Science

The University of Sydney Page 2
Jupyter Notebooks:
The Python Environment in DATA2001

The University of Sydney Page 3
Jupyter Notebooks support interactive Data Science
with Python
– IPython interactive command shell offers:
– Introspection
– Tab completion
– Command history
– Jupyter runs in a browser and supports:
– Sharing and documenting of live code
– Data cleaning, visualisation, machine learning, ...
– Jupyter’s gallery of interesting notebooks:
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks
– We provide Jupyter servers which run Python 3
https://ucpu0.ug.it.usyd.edu.au/ (remember you need to be using VPN, if off campus)

The University of Sydney Page 4
1. Click here for file
open dialogue
2. Click upload
next to file name

The University of Sydney Page 5
Installing Python and Jupyter using Anaconda
– You can use our Jupyter
server
– but it can be slow
– If you wish, you
can install Python and
Jupyter privately, eg
using
Anaconda Distribution,
which includes Python,
the Jupyter Notebook,
and other commonly used
packages for scientific
computing and data
science.

The University of Sydney Page 6
Python and Data Science Libraries

The University of Sydney Page 7
Python background
– Students who did data1002: this should mostly be revision
– If you didn’t really master pandas, matplotlib before, do so now!
– Also note the following key differences
• More sophisticated ways to consider the kinds of data (not just
numerical/castegorical)
– Students who learned Python elsewhere (eg infi1110): you
need to learn how to use particular libraries (Pandas,
matplotlib etc) from the examples here, and online resources

The University of Sydney Page 8
Python general concepts
– general program syntax
– variables and types
– integer and float numbers, string types, type conversion
– list, dictionary, tuple and set
– condition statements (if/elif/else)
– for loops, ranges
– functions
– print(), len(), lower(), upper(), ...
– nesting of functions; example: print( len( str.upper() ) )