Wednesday, June 19, 2013

matplotlib followup for Python 3 on OS X 10.8

In my last post, I reported success in installing matplotlib on OS X. I realize now that this is not anything new to report, since the currently recommended install process (which I read belatedly today) in the README is to use Homebrew (or MacPorts), which is what I did. I'm just happy to know that it works.

I thought I'd say a word about Python 3 here.

I did

$ brew install python3 --framework

which first installed readline and sqlite (linked into /usr/local/opt), as well as gdbm and xz, and also Distribute and pip.

So now we have:

$ ls /usr/local/lib/python3.3/site-packages/
__pycache__    setuptools-0.6c11-py3.3.egg-info
distribute-0.6.45-py3.3.egg  setuptools.pth
easy-install.pth   site.py
pip-1.3.1-py3.3.egg   sitecustomize.py


$ python3
Python 3.3.2 (default, Jun 19 2013, 15:41:32) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from test import autotest
== CPython 3.3.2 (default, Jun 19 2013, 15:41:32) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]
==   Darwin-12.4.0-x86_64-i386-64bit little-endian
==   /usr/local/bin
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1)
[  1/372] test_grammar
[  2/372] test_opcodes
[  3/372] test_dict
[  4/372] test_builtin

The tests hang at this point, and I haven't figured out why yet. They do fine with my system Python. Then I did:

$ pip3 install numpy
$ pip3 install nose

since we need numpy for matplotlib (and don't get it for free as with System Python).

Following this advice, I did:

$ python3
..
>>> import numpy
>>> numpy.test('full')

Ran 4808 tests in 64.611s

OK (KNOWNFAIL=6, SKIP=6)
<nose.result.TextTestResult run=4808 errors=0 failures=0>

Now to matplotlib

$ git clone git://github.com/matplotlib/matplotlib.git
$ cd matplotlib
$ python3 setup.py build
$ sudo python3 setup.py install

(Grabs pyparsing, dateutil, tornado).

$ cd..
$ python3
..
>>> import matplotlib.pyplot as plt
>>> Y = [1,4,9,16]
>>> plt.scatter(range(len(Y)),Y,s=250,color='r')
<matplotlib.collections.PathCollection object at 0x1094e4650>
>>> plt.savefig('example.png')

works!

$ pip3 install cython

had a permissions problem, so I did

$ sudo chmod -R 755 /usr/local/lib/python3.3/site-packages/
$ pip3 install cython

Now, for scipy

$ git clone git://github.com/scipy/scipy.git scipy
$ cd scipy
$ python3 setup.py build
$ sudo python3 setup.py install

$ cd ..
$ python3
..
>>> from scipy.stats import norm
>>> norm.cdf(2)
0.97724986805182079

>>> import scipy
>>> scipy.test('full')
..

Ran 7486 tests in 725.812s

FAILED (KNOWNFAIL=42, SKIP=296, failures=82)


Some failures, but all in all it looks good, and a very easy install.

Tuesday, June 18, 2013

Matplotlib (and SciPy) on OS X Mountain Lion

Yesterday I installed matplotlib on my MacBook---for about the 10th time. This can be a complex undertaking, but a basic installation is relatively easy, so I thought I would outline it here.

Some things I did that made it easier:

• I used a clean install of OS X. I did this to make sure cruft (links, alternative versions of libraries from old installs) does not interfere. Also, this way I can be sure that things work for the reasons I have written about here.

To do the install I just made a second partition on my hard drive and installed OS X on it from a USB installer. If you've never done this before, good instructions for the USB part are here.

It's quite straightforward. Use the App store to update when the install is complete. I now have OS X 10.8.4.

Although it's not necessarily good practice, I used the same username and password for my accounts on OS X on both partitions, this allows me to read and write files in both accounts easily using the Finder.

• I used the System Python. This doesn't appear to be a very popular choice, but it avoids the headache of having multiple Pythons and various $PATH issues, etc. If you do install a second Python, make sure it's a framework version, since even saving a plot as a png using matplotlib will fail otherwise. I may do some more experiments later, but I think this is the right choice

My Python is

$ which python
/usr/bin/python

(well, it's not really the whole of Python on OS X, but that is a whole 'nother story).

$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)

A bonus is that we already have numpy

>>> import numpy
>>> numpy.__version__
'1.6.1'

• I used Homebrew to get the most important matplotlib prerequisites, libpng and freetype. zlib is also a prerequisite but comes with OS X.

The instructions for Homebrew are here. Weirdly, I got a prompt for an admin password, which happened because the directory where Homebrew puts stuff, /usr/local, did not exist yet. So I bailed from the process and first did:

sudo mkdir /usr/local

Now, look for issues revealed by brew doctor and fix them if needed. Finally:

$ brew doctor
Your system is ready to brew.
$ brew update
Already up-to-date.
$ brew install pkgconfig libpng freetype
..
$ brew list
freetype libpng  pkg-config

the "Bottles" mentioned in the output I cut means these are pre-built.

I got pkgconfig because it helped with building/linking problems previously, but should not really be necessary when the libraries are in /usr/local. Not sure why it is some times pkg-config and other times pkgconfig, but OK:

$ ls /usr/local/bin/pkg-config 
/usr/local/bin/pkg-config
$ ls /usr/local/lib/pkgconfig/
freetype2.pc libpng.pc libpng15.pc

(If you use e.g. the XQuartz versions of these two libraries, you can set the PKG_CONFIG_PATH environment variable to point to the correct directory, and pkg-config will do the rest).

Now we're ready for matplotlib.

$ git clone git://github.com/matplotlib/matplotlib.git
..
$ cd matplotlib
$ python setup.py build
$ sudo python setup.py install

And test it:

cd ..
python 
>>> import matplotlib.pyplot as plt
>>> Y = [1,4,9,16]
>>> plt.scatter(range(len(Y)),Y,s=250,color='r')
<matplotlib.collections.PathCollection object at 0x103963110>
>>> plt.savefig('example.png')

Works!

>>> import matplotlib
>>> matplotlib.test()
..KK.KK.....KK.KK..KK.KK.KK.KK.KK  etc.

I didn't test extensively but there don't seem to be massive failures. And matplotlib has lots of optional prerequisites I didn't install that would explain some failures.

It's worth pointing out that the matplotlib install process grabs several additional packages:

$ ls /Library/Python/2.7/site-packages
README
distribute-0.6.28-py2.7.egg
easy-install.pth
matplotlib-1.4.x-py2.7-macosx-10.8-intel.egg
nose-1.3.0-py2.7.egg
pyparsing-1.5.7-py2.7.egg
setuptools.pth
tornado-3.1-py2.7.egg
$

Let's get a few more things while we're at it:

$ sudo easy_install pip
Password: ****
Searching for pip
Reading http://pypi.python.org/simple/pip/
Best match: pip 1.3.1
..
Installing pip script to /usr/local/bin
Installing pip-2.7 script to /usr/local/bin
..

Come to think of it, SciPy would be a great addition. For that we need gfortran (Fortran compiler) and Cython:

$ brew install gfortran
(takes a while, several dependencies)..

Homebrew doesn't have Cython so:

$ sudo pip install cython

And now for SciPy:

$ git clone git://github.com/scipy/scipy.git scipy

$ cd scipy
$ python setup.py build
$ sudo python setup.py install

$ cd ..
$ python
..
>>> from scipy.stats import norm
>>> norm.cdf(2)
0.97724986805182079
If you check for the libraries loaded (I used export DYLD_PRINT_LIBRARIES=1), you'll see we're using Accelerate.

Finally, get PyCogent too..

$ git clone git://github.com/pycogent/pycogent.git
$ cd pycogent
$ python setup.py build
$ sudo python setup.py install

That was pretty easy!