5.3. Tutorial: Basic Array Manipulation¶

Arrays are sequences of numerical data, with each element having the same underlying data type – integers, real (floating point) numbers, or complex numbers. Of course, arrays are very important for scientific data, and the majority of data you will be manipulating with Larch will be in the form of arrays. Larch depends on the numpy library for providing basic array data types and the methods for fast manipulation of array data. These arrays can be multi-dimensional, have their dimensionality change dynamically, an can be sliced (subsets of elements taken). Many built-in functions will act on the whole array, generally element-by-element, in a highly efficient way, greatly reducing the need to “loop over” elements.

This section introduces the creation, basic manipulation, and key properties of arrays. The discussion here is not exhaustive, but is intended to get you far enough along to be able to manipulate arrays sufficiently for most needs. The numpy documentation is quite extensive and well-written, and should be consulted for more details.

5.3.1. Functions for creating arrays¶

There are a handful of builtin functions for creating arrays from scratch. These are listed in Table of Array Creation Functions In addition, when dealing with data stored in files, reading the files (see 6) with Larch will create arrys for you. Like lists, arrays are mutable. That is, elements of arrays can be changed without changing the rest of the array.

Table of Array Creation Functions. These functions can all be used to create arrays in Larch.

Function Name description example

array array from list arr = array([1,2,3])

arange indices 0, N-1 arr = arange(10)

zeros fill with N zeros arr = zeros(10)

ones fill with N ones arr = ones(10)

linspace fill with bounds and N arr = linspace(0, 1, 11)

eye 2-d identity matrix arr = eye(3)

meshgrid 2-d mesh arrays mx, my = meshgrid(x, y)

Some examples of using these functions are needed:

larch> i = arange(10)
larch> i
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
larch> f = arange(10, dtype=float)
larch> f
array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
larch> c = arange(10, dtype=complex)
larch> c
array([ 0.+0.j,  1.+0.j,  2.+0.j,  3.+0.j,  4.+0.j,  5.+0.j,  6.+0.j,
        7.+0.j,  8.+0.j,  9.+0.j])
larch> i3 = eye(3)
larch> i3
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

Here, the dtype argument sets the data type for the array members. For example, float means a double precision floating point representation of a real number and complex means double precision complex floating point. The Table of Array Data types below lists the available data types that can be passed as a dtype argument.

The linspace() function is particularly useful for creating arrays, as it takes a starting value, ending value, and number of points between these:

larch> s = linspace(0, 10, 21)
larch> s
array([  0. ,   0.5,   1. ,   1.5,   2. ,   2.5,   3. ,   3.5,   4. ,
         4.5,   5. ,   5.5,   6. ,   6.5,   7. ,   7.5,   8. ,   8.5,
         9. ,   9.5,  10. ])

Several variants are possible. For more information, consult the numpy tutorials, or use the online help system within Larch (which will print out the documentation string from the underlying numpy function):

larch> help(linspace)

    Return evenly spaced numbers over a specified interval.

    Returns `num` evenly spaced samples, calculated over the
    interval [`start`, `stop` ].

    The endpoint of the interval can optionally be excluded.


    Parameters
    ----------
    start : scalar
        The starting value of the sequence.
    stop : scalar
        The end value of the sequence, unless `endpoint` is set to False.
        In that case, the sequence consists of all but the last of ``num + 1``
        evenly spaced samples, so that `stop` is excluded.  Note that the step
        size changes when `endpoint` is False.
    num : int, optional
        Number of samples to generate. Default is 50.
    endpoint : bool, optional
        If True, `stop` is the last sample. Otherwise, it is not included.
        Default is True.
    retstep : bool, optional
        If True, return (`samples`, `step`), where `step` is the spacing
        between samples.

    ....

The eye() function creates a 2 dimensional square array (or, possibly matrix, though that term can connote additional meaning that a square array does not necessarily have) with values of 1 for the diagonal elements, and 0 elsewhere.

The function meshgrid() can be used to build a mesh of values from two one dimensional arrays. That is:

larch> x = array([0, 1, 2, 3, 4])
larch> y = array([-1, 0.5, 0, 0.5, 1])
larch> mx, my = meshgrid(x, y)
larch> print mx
array([[0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4]])
larch> print my
array([[-1. , -1. , -1. , -1. , -1. ],
       [-0.5, -0.5, -0.5, -0.5, -0.5],
       [ 0. ,  0. ,  0. ,  0. ,  0. ],
       [ 0.5,  0.5,  0.5,  0.5,  0.5],
       [ 1. ,  1. ,  1. ,  1. ,  1. ]])

That is, the values of each mx and my contain the coordinates of a two-dimensional mesh or map of the input x and y values.

5.3.2. The array dtype – datatype¶

Arrays are sequences of numbers stored in memory to make access to the elements of the arrays fast and memory use efficient. As mentioned above, each array will use one of several storage conventions depending on what type of data is needed for the elements. Basically, this dictates how many bytes to use and whether these values are meant to hold integers, real floating points, or complex floating point numbers. This information is encapsulated in the arrays dtype. It can be one of several values, listed in the

Table of Array Data types. Each array has exactly one of these create arrays in Larch. The dtype can be used in any of the array creation functions using the dtype keyword argument (e.g, dtype=float32).

dtype description

bool boolean (True or False)

int8 signed 8-bit integer (-128 to 127)

int16 signed 16-bit integer (-32768 to 32767)

int32 signed 32-bit integer (-2**31 to 2**31-1)

int64 signed 64-bit integer (-2**63 to 2**63-1)

uint8 unsigned 8-bit integer (0 to 255)

uint16 unsigned 16-bit integer (0 to 65535)

uint32 unsigned 32-bit integer (0 to 2**32)

uint64 unsigned 64-integer (0 to 2**64)

float32 single precision float

float64 or float double precision float

complex64 single precision complex two float32s

complex128 or complex double precision complex, two float64s.

5.3.3. Basic array manipulation¶

Arrays can either be used as a single object, or individual or ranges of elements can be extracted from them. Usually, mathematical operations and functions done to arrays are applied element-by-element. For example:

larch>  x = arange(5)
larch> print x
[0 1 2 3 4]
larch> print 2*x+1
[1 3 5 7 9]

and so on for all of the basic mathematical operators. To add to arrays of equal lengths together is also easy:

larch>  y = array([10, 12, 14, 16, 18])
larch> print y - 3*x
[10  9  8  7  6]

If the arrays are not of equal length, an exception is raised. In that case, you can take a sub-set of the larger array to match the size of the smaller one.

Boolean operators also apply to each element, so that:

larch> print y > 13
[False False  True  True  True]

The any() and all() functions (in the tables below) can be used to determine if any or all of the Boolean values are True.

You can extract single elements from arrays with brackets, just as for lists:

larch> print y[2]
14
larch> print (y-3*x)[2]
7

which leads us to the next section.

5.3.4. Slicing and extracting sub-arrays¶

An important aspect of arrays is that they can be treated as a single entity. That is, sin(x) operates on each element of x. But sometimes it is necessary to get a particular element from an array or work on only a selected part of an array. To do these, one takes a sub-set of the array – a slice. For extracting contiguous portions of 1-dimension arrays, this is pretty straightforward, using the range of indices needed for the slice between square brackets [ and ]. For example:

larch> arr = linspace(0, 2, 21)
larch> print arr
[ 0.   0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.   1.1  1.2  1.3  1.4
  1.5  1.6  1.7  1.8  1.9  2. ]
larch> print arr[20]
2.0
larch> print arr[15]
1.5
larch> print arr[10:15]
[ 1.   1.1  1.2  1.3  1.4]

The general syntax for a slice is pretty complicated, but the simplest cases are straightforward. As with lists, arr[i] selects value at index i (counting from 0). Similarly, arr[i:j] selects elements starting at i and ending at (but not including – see the example above) j. If i is omitted, it is taken as 0 (the first element), and if j is omitted, it defaults to the last element of the array. In addition, if i and/or j are negative, they count from the end of the array:

larch> print arr[:-8]
[ 0.   0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.   1.1  1.2]
larch> print arr[-2:]
[ 1.9  2.]

A slice can take a third argument k – the stride – which allows selection of every k elements. That is:

larch> print arr[1:6:2]
[ 0.1  0.3  0.5]
larch> print arr[::2]
[ 0.   0.2  0.4  0.6  0.8  1.   1.2  1.4  1.6  1.8  2. ]

If k is negative, it starts from the end of the array:

larch> print arr[::-3]
[2.   1.7  1.4  1.1  0.8  0.5  0.2]

For mult-dimensional arrays, slices can be made for each dimension, with slices separated by commas. If no slice is given, the whole array along that dimension is used. Thus:

larch> x = arange(30).reshape((6, 5)
larch> print x
[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]
 [20 21 22 23 24]
 [25 26 27 28 29]]
larch> print x[2]  # third row
[10 11 12 13 14]
larch> print x[:,1]  # second column
[ 1  6 11 16 21 26]
larch> print x[:3,2:5]
[[ 2  3  4]
 [ 7  8  9]
 [12 13 14]]

Note that multi-dimensional arrays use a layout like C and unlike Fortran by default. In addition to the comma-based syntax shown above to extract different dimensions or use brackets around each dimension:

larch> print x[1][:4]  # second row, first 4 columns
[5 6 7 8]
larch> print x[1,:4]   # same
[5 6 7 8]

In general, the syntax for a slice is then arr[i1:j1,:k1, i2:j2:k2, ...] with default values for i of 0, for j of the length of the array, and for k of 1.

5.3.5. Array attributes and methods¶

Arrays have several useful attributes and methods. As mentioned above, each array has a dtype attribute describing how its data is mapped in memory. In addition, each has a size attribute giving the number of elements, a ndim giving the number of dimensions, and shape giving the a tuple with the length along each dimension. The dimensionality and shape of a multi-dimensional array can be specified by setting the value of shape to the desired value:

larch> x = arange(12, dtype=float)
larch> x
array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 11., 12.])
larch> x.shape = (2, 6)
larch> x
array([[ 0.,  1.,  2.,  3.,  4.,  5.],
       [ 6.,  7.,  8.,  9., 10., 11.]])

Table of Array Attribute and Methods. Those ending with parentheses (()) are methods, that act on the array. Most of the attributes and the vast majority of methods return a value based on the array contents, leaving the array unchanged. The attributes and methods marked read/write operate in place, changing the array.

attribute description notes

dtype data type read only

size number of elements (int) read only

ndim number of dimensions (int) read only

shape length along each dimension (tuple) read/write

real real part of array read/write

imag imaginary part of array read/write

resize() grow/shrink array to specified size read/write

conjugate() conjugate of array

all() boolean: if all values are True (non-zero)

any() boolean: if any value is True (non-zero)

min() minimum value of array elements

max() maximum value of array elements

mean() mean value of array elements

std() standard deviation of array elements

prod() product of array elements

sum() sum of array elements

argmin() index of first minimum value

argmax() index of first maximum value

argsort() array of indices for sorted array (min to max)

cumprod() array of cumulative product of elements

cumsum() array of cumulative sum of elements

astype() array re-cast as a different dtype

round() array of rounded elements

diagonal() array of diagonal elements

trace() sum of diagonal elements()

transpose() transpose of array

flatten() array “flattened” to 1-dimension

tolist() list containing array elements

reshape() array reshaped to specified shape tuple

5.3.6. Mathematical functions for arrays¶

Many of the basic mathemetical functions in larch automatically work on arrays element-by-element. For example, sqrt() returns the square-root of a single value or for each element in an array:

larch> print sqrt(3)
1.73205080757
larch> x = arange(10)
larch> print sqrt(x)
[ 0.          1.          1.41421356  1.73205081  2.          2.23606798
  2.44948974  2.64575131  2.82842712  3.        ]

The numpy library provides the underlying functions, and they are much faster than looping over elements of the array:

larch> for el in x:  # this is much slower than sqrt(x)!!
.....>     print el, sqrt(el)
.....> endfor
0.0
1.0
1.41421356237
1.73205080757
2.0
2.2360679775
2.44948974278
2.64575131106
2.82842712475
3.0

There are a large number of general purpose mathematical functions available in larch – the _math group contains over 500 items on startup. A partial list is given in the The Table of Array-aware Mathematical functions below. What’s more, many more are available by importing them from the scipy library.

Table of Array-aware Mathematical functions. More info on each of these can be found with the builtin help() function. The table is broken up by categories to make printing easier.

General Purpose Functions

function description

all all values are True

allclose all values of 2 arrays are close

info print information about array storage

fabs absolute value of values

sqrt square root of values

exp exponential of values

expm1 exp(x) - 1 for values x

exp2 2**x for values x

ln / log natural logarithm of values

log1p log(x) + 1 for values x

log10 base-10 logarithm of values

log2 base-2 logarithm of values

mod modulus of values

ldexp x * 2**y for values x and y

fmin element-wise minima of two arrays

fmax element-wise maxima of two arrays

fmod element-wise modulus of two arrays

frexp split value into fractional and exponent

Trigonometry Functions

function description

angle phase angle for complex values

acos / arccos inverse of cosine

asin / arcsin inverse of sine

atan / arctan inverse of tangent

atan2 / arctan2 inverse of tangent of ratio of two values

acosh / arccosh inverse of hyperbolic cosine

asinh / arcsinh inverse of hyperbolic sine

atanh / arctanh inverse of hyperbolic tangent

cos cosine

cosh hyperbolic cosine

sin sine

sinh hyperbolic sine

tan tangent

tanh hyperbolic tangent

deg2rad convert degrees to radians

rad2deg convert radians to degrees

hypot hypotenuse (distance) of two values

Array Manipulation and Re-shaping

function description

append append a value to an array

insert insert a value into a specified location of an array

concatenate Join a sequence of arrays together

tile build array by repeating an array a number of times

repeat repeat elements of an array a number of times

split Split array into sub-arrays vertically (row)

hsplit Split array into sub-arrays horizontally (column)

dsplit Split array into sub-arrays along the 3rd axixpth (depth)

hstack Stack arrays in sequence horizontally (column)

vstack Stack arrays in sequence vertically (row )

dstack Stack arrays in sequence along third dimension (depth)

take take values at specified indices

choose construct array from index array and a set of arrays

where select array elements depending on an input condition

Statistical Functions

function description

average average value of an array, with optional weights

max maximum value of an array

mean mean value of an array

median median value of an array

min minimum value of an array

var variance of an array

std standard deviation of an array

trapz integrate using composite trapezoidal rule

remainder remainder of division (x1 - floor(x1 / x2) * x2)

percentile returns the given percentile of array elements

ceil ceiling values (round “up”) of input

floor floor values (round “down”) of input

round round values (away from 0) of input

clip set upper/lower bounds on array values

digitize indices of bins for binned values

bincount number of occurrences of each value in array

histogram build a histogram from an array

histogram2d build a 2-d histogram from two arrays

convolve discrete convolution of two 1-d arrays

correlate cross-correlation of two 1-d arrays

cumprod cumulative product

cumsum cumulative sum

Multi-dimensional and Matrix Functions

function description

tril upper triagonal of an array

triu lower triagonal of an array

diagonal diagonal elements of an array

trace sum of diagonal elements

dot dot product of two arrays

inner inner product of two arrays

outer outer product of two arrays

kron Kronecker product of two arrays

swapaxes rotate axes of an array

transpose transpose array

fliplr Flip an array horizontally

flipud Flip an array vertically

rot90 rotate an array 90 degrees counter clockwise

Array and Signal Processing

function description

diff finite difference of array elements

gradient gradient of an array

interp linear interpolation of 1-d arrays

poly Evaluate a polynomial at a point

root the roots of a polynomial

polyfit least squares polynomial fit

Random number generation and other

function description

random.random randomly distributed reals, on [0, 1).

random.randint array of random integers, over specified range

random.normal normally distributed random numbers

Fourier transforms

function description

fft.fft Fourier transform of a 1-d array

fft.ifft inverse Fourier transform of a 1-d array

fft.fft2 Fourier transform of a 2-d array

fft.ifft2 inverse Fourier transform of a 2-d array

Many other functions that work on arrays are available from numpy subpackages and from scipy.

Table Of Contents

Previous topic

Next topic

This Page