Numpy (Python3) tutorial

Basic idea of numpy

Manipulate array of numbers very efficiently

The bad example, whithout numpy:

In [1]:
import time
import math
xs = range(1000000)
t11 = time.time()
ys = []
for x in xs:
  ys.append(math.sqrt(x))

t12 = time.time()
print(t12-t11)

0.33169031143188477

The good example, with numpy:

In [2]:
import numpy as np
xs = np.arange(1000000)
t21 = time.time()
ys = np.sqrt(xs)
t22 = time.time()
print(t22-t21)

0.018557310104370117

The ratio:

In [3]:
print ((t12-t11)/(t22-t21))

17.873835678036873

Numpy objects

Array objets are sets of numbers of same type.

Some definitions:

name definition
size total number of elements
shape integer tuple giving the number of elements in each dimension
rank size of "shape" corresponds to the number of dimensions
dtype type of elements in the array

Some types:

type name type code
bolean Bool
unsigned UInt8, UInt16, UInt32, UInt64
integer Int8, Int16, Int32 (Int), Int64
float Float32, Float64 (Float)
complex Complex32, Complex64 (Complex)

Importing Numpy

In [6]:
import numpy as np
In [5]:
from numpy import *
sqrt(3)
np.sqrt()
Out[5]:
1.7320508075688772

Creating a Numpy array

creating it explicitely:

In [6]:
x = np.array([1,2,3])
x
Out[6]:
array([1, 2, 3])
In [7]:
y = np.array([[1,2],[3,4],[5,6]],dtype=float)
y
Out[7]:
array([[1., 2.],
       [3., 4.],
       [5., 6.]])
In [10]:
x.size
Out[10]:
3
In [8]:
y.shape
Out[8]:
(3, 2)
In [12]:
x.dtype
Out[12]:
dtype('int64')
In [13]:
y.size
Out[13]:
6
In [14]:
y.shape
Out[14]:
(3, 2)
In [15]:
y.dtype
Out[15]:
dtype('float64')

creating from a python list:

In [16]:
np.asarray((1,2,3))
Out[16]:
array([1, 2, 3])

creating from building function:

In [17]:
np.zeros(3)
Out[17]:
array([ 0.,  0.,  0.])
In [18]:
np.zeros((5,5))
Out[18]:
array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
In [19]:
np.ones((2,3),float)
Out[19]:
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]])
In [22]:
np.arange(10)
Out[22]:
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
In [23]:
np.identity(10)
Out[23]:
array([[ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.]])
In [25]:
np.linspace(0,np.pi,10)
Out[25]:
array([ 0.        ,  0.34906585,  0.6981317 ,  1.04719755,  1.3962634 ,
        1.74532925,  2.0943951 ,  2.44346095,  2.7925268 ,  3.14159265])

creating from a function:

In [26]:
def x2(x):
  return x*x

def xy(x,y):
  return x*y
In [27]:
np.fromfunction(x2,(5,))
Out[27]:
array([  0.,   1.,   4.,   9.,  16.])
In [28]:
np.fromfunction(xy,(5,5))
Out[28]:
array([[  0.,   0.,   0.,   0.,   0.],
       [  0.,   1.,   2.,   3.,   4.],
       [  0.,   2.,   4.,   6.,   8.],
       [  0.,   3.,   6.,   9.,  12.],
       [  0.,   4.,   8.,  12.,  16.]])

change dimension:

In [29]:
x = np.array([1,2,3,4,5,6,7,8,9,10,11,12])
x
Out[29]:
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])
In [30]:
np.reshape(x,(2,6))
Out[30]:
array([[ 1,  2,  3,  4,  5,  6],
       [ 7,  8,  9, 10, 11, 12]])
In [31]:
x.shape = (6,2)
x
Out[31]:
array([[ 1,  2],
       [ 3,  4],
       [ 5,  6],
       [ 7,  8],
       [ 9, 10],
       [11, 12]])
In [32]:
y = np.reshape(x,(3,4))
y
Out[32]:
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])
In [33]:
np.reshape(y,(12,))
y
Out[33]:
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])

casting:

In [34]:
x = np.array([1,2,3],float)
x
Out[34]:
array([ 1.,  2.,  3.])
In [35]:
y = np.array([1,2,3],int)
y
Out[35]:
array([1, 2, 3])
In [36]:
(x+y).dtype
Out[36]:
dtype('float64')

Indexing arrays

Retreieve an element or a group of elements:

In [10]:
a = np.arange(12)
a
Out[10]:
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
In [11]:
a[0]
Out[11]:
0
In [12]:
a[1]=-a[1]
a
Out[12]:
array([ 0, -1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
In [13]:
a[:]
Out[13]:
array([ 0, -1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
In [42]:
a[2:4] = [-2,-3]
a
Out[42]:
array([ 0, -1, -2, -3,  4,  5,  6,  7,  8,  9, 10, 11])
In [43]:
a[5:]
Out[43]:
array([ 5,  6,  7,  8,  9, 10, 11])

Multidimentional arrays

In [47]:
a.shape = (3,4)
a
Out[47]:
array([[ 0, -1, -2, -3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])
In [48]:
a[2,0]
Out[48]:
8
In [49]:
a[0]
Out[49]:
array([ 0, -1, -2, -3])
In [50]:
a[0,:]
Out[50]:
array([ 0, -1, -2, -3])
In [51]:
a[:,0]
Out[51]:
array([0, 4, 8])
In [52]:
a[1:,1:]
Out[52]:
array([[ 5,  6,  7],
       [ 9, 10, 11]])
In [53]:
a[1:,1:] = np.ones((2,3))
a
Out[53]:
array([[ 0, -1, -2, -3],
       [ 4,  1,  1,  1],
       [ 8,  1,  1,  1]])

Using a list of indexes:

In [54]:
a = np.arange(12)**2
a
Out[54]:
array([  0,   1,   4,   9,  16,  25,  36,  49,  64,  81, 100, 121])
In [55]:
index = [0,4,3]
a[index]
Out[55]:
array([ 0, 16,  9])
In [56]:
a[np.arange(3)]
Out[56]:
array([0, 1, 4])

another example:

In [57]:
a = np.reshape(a,(3,4))
a
Out[57]:
array([[  0,   1,   4,   9],
       [ 16,  25,  36,  49],
       [ 64,  81, 100, 121]])
In [59]:
ind1 = [1,2]
ind2 = [0,3]
a[ind1,ind2]
Out[59]:
array([ 16, 121])

and another one:

In [63]:
ind1 = np.array([[1,2],[0,2]])
ind1
Out[63]:
array([[1, 2],
       [0, 2]])
In [64]:
ind2 = np.array([[0,3],[1,2]])
ind2
Out[64]:
array([[0, 3],
       [1, 2]])
In [65]:
a[ind1,ind2]
Out[65]:
array([[ 16, 121],
       [  1, 100]])

Special values

nan, inf:

In [66]:
x = np.array([0,1],float)
y = x/0.
y

/usr/lib/python3.6/site-packages/ipykernel_launcher.py:2: RuntimeWarning: divide by zero encountered in true_divide
  
/usr/lib/python3.6/site-packages/ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in true_divide
Out[66]:
array([ nan,  inf])

track special values:

In [67]:
np.isnan(y)
Out[67]:
array([ True, False], dtype=bool)
In [68]:
np.isinf(y)
Out[68]:
array([False,  True], dtype=bool)
In [69]:
np.isfinite(y)
Out[69]:
array([False, False], dtype=bool)
In [70]:
np.isfinite(x)
Out[70]:
array([ True,  True], dtype=bool)

replace bad values:

In [71]:
y[np.isnan(y)]=0
y[np.isinf(y)]=1e10
y
Out[71]:
array([  0.00000000e+00,   1.00000000e+10])

Array methods

Some important methods:

In [14]:
x = np.array([0,4,3,2])
x.min()
Out[14]:
0
In [15]:
x.max()
Out[15]:
4
In [16]:
x.mean()
Out[16]:
2.25
In [17]:
x.std()
Out[17]:
1.479019945774904
In [18]:
x.argmax()
Out[18]:
1
In [19]:
x.argmin()
Out[19]:
0
In [20]:
x.argsort()
Out[20]:
array([0, 3, 2, 1])
In [21]:
x.sum()
Out[21]:
9
In [80]:
x.conjugate()
Out[80]:
array([0, 4, 3, 2])

Operations on arrays (ufunc)

Operations elements by elements return an array of same shape.

add (+) substract (-) multiply (*) divide ()
remainder (%) power (**)
abs fabs floor ceil
fmod conjugate
maximum minimum
cos sin arccos arcsin
cosh sinh arccosh arcsinh
tan arctan tanh arctanh
log log10 exp
greater (>) greater_equal(>=) equal(==)
less (<) less_equal(<=) not_equal(!=)
logical_or logical_xor logical_not logical_and
bitwise_or () bitwise_xor (^) bitwise_not (~) bitwise_and (&)
rshift(>>) lshift(<<)

Examples:

In [22]:
x = np.array([1,2,3],float)
y = np.array([4,5,6],float)
In [23]:
x/y
Out[23]:
array([0.25, 0.4 , 0.5 ])
In [24]:
x**2
Out[24]:
array([1., 4., 9.])
In [25]:
x = np.array([1,2,3])
x
Out[25]:
array([1, 2, 3])
In [26]:
y = np.reshape(np.arange(12),(4,3))
y
Out[26]:
array([[ 0,  1,  2],
       [ 3,  4,  5],
       [ 6,  7,  8],
       [ 9, 10, 11]])

sum a vector with a matrix

In [27]:
x+y
Out[27]:
array([[ 1,  3,  5],
       [ 4,  6,  8],
       [ 7,  9, 11],
       [10, 12, 14]])
In [90]:
y*= 10
y
Out[90]:
array([[  0,  10,  20],
       [ 30,  40,  50],
       [ 60,  70,  80],
       [ 90, 100, 110]])
In [91]:
y+= y
y
Out[91]:
array([[  0,  20,  40],
       [ 60,  80, 100],
       [120, 140, 160],
       [180, 200, 220]])

compare arrays

In [94]:
x = np.array([1,2,3])
y = np.array([0,2,3],float)
(x>y)
Out[94]:
array([ True, False, False], dtype=bool)
In [95]:
(x==2)
Out[95]:
array([False,  True, False], dtype=bool)
In [96]:
(x>y) | (x==2)
Out[96]:
array([ True,  True, False], dtype=bool)

Array functions

General functions on arrays:

Example: replace specific values in a vectors:

In [97]:
x = np.arange(16)
x = np.reshape(x,(4,4))
x
Out[97]:
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15]])
In [98]:
y = np.fmod(x,3)
y
Out[98]:
array([[0, 1, 2, 0],
       [1, 2, 0, 1],
       [2, 0, 1, 2],
       [0, 1, 2, 0]])
In [100]:
np.where((y==0),-1,y)
Out[100]:
array([[-1,  1,  2, -1],
       [ 1,  2, -1,  1],
       [ 2, -1,  1,  2],
       [-1,  1,  2, -1]])

Replace values using a mask:

In [106]:
x = np.arange(16)
x = np.reshape(x,(4,4))
x
Out[106]:
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15]])
In [107]:
mask = np.where((y==0),1,0)
mask
Out[107]:
array([[1, 0, 0, 1],
       [0, 0, 1, 0],
       [0, 1, 0, 0],
       [1, 0, 0, 1]])
In [108]:
np.putmask(x,mask,y)
x
Out[108]:
array([[ 0,  1,  2,  0],
       [ 4,  5,  0,  7],
       [ 8,  0, 10, 11],
       [ 0, 13, 14,  0]])

Example: remove bad values in a vector:

In [110]:
x = np.array([0,1,2,3,4,5],float)
y = np.array([1,0,3,0,5,2],float)
ly = np.log10(y)
ly

/usr/lib/python3.6/site-packages/ipykernel_launcher.py:3: RuntimeWarning: divide by zero encountered in log10
  This is separate from the ipykernel package so we can avoid doing imports until
Out[110]:
array([ 0.        ,        -inf,  0.47712125,        -inf,  0.69897   ,
        0.30103   ])
In [111]:
c = (y!=0)
c
Out[111]:
array([ True, False,  True, False,  True,  True], dtype=bool)
In [112]:
x = np.compress(c,x)
x
Out[112]:
array([ 0.,  2.,  4.,  5.])
In [113]:
y = np.compress(c,y)
y
Out[113]:
array([ 1.,  3.,  5.,  2.])
In [114]:
ly = np.log10(y)
ly
Out[114]:
array([ 0.        ,  0.47712125,  0.69897   ,  0.30103   ])

Other very usefull functions:

ravel:

In [115]:
x = np.identity(3)
x
Out[115]:
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
In [116]:
np.ravel(x)
Out[116]:
array([ 1.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  1.])
In [119]:
x = np.array([1,2,3])
y = np.array([4,5,6])

concatenate:

In [120]:
z = np.concatenate((x,y))
z
Out[120]:
array([1, 2, 3, 4, 5, 6])

clip:

In [121]:
np.clip(z,2,5)
Out[121]:
array([2, 2, 3, 4, 5, 5])

sum:

In [124]:
np.sum(z)
Out[124]:
21

Matrix operations: transpose, diagonal, trace, dot product:

In [125]:
a = np.array([1,2])
b = np.array([4,5])
In [126]:
np.dot(a,b)
Out[126]:
14

Polynomials:

In [4]:
from numpy import poly1d
p1 = poly1d([3,4,5])
p2 = poly1d([1,1,1])
print(p1)

   2
3 x + 4 x + 5
In [128]:
print(p1+p2)

   2
4 x + 5 x + 6
In [129]:
print(p1.deriv())

 
6 x + 4
In [130]:
print(p1.integ(k=6))

   3     2
1 x + 2 x + 5 x + 6
In [ ]:
In [ ]: