Monday, January 31, 2011

Jake Shimabukuro


I guess I'm probably the last person in the world to learn about this guy:

my favorite:

Incredible feeling...

one more
and the last
but one.
(Can't help myself)

Unix spoken here (4)

There are a just few more Unix commands I should mention before we exhaust everyone's patience. (Three posts here, here, and here).

I'm sure you're familiar with sudo, which stands for superuser, do once. If not, you can see wikipedia. Normally you will use this to write (install) or even delete files in /usr/bin and the like.

> mv /usr/bin/znew ~/Desktop/znew
mv: rename /usr/bin/znew to /Users/telliott_admin/Desktop/znew: Permission denied

Don't forget to read the man page on this one. There is always something interesting in the manual.

Many Unix commands appear simple but have numerous options and arguments that make them more like a Swiss Army Knife than a pocketknife. Output on the command line using man can be broken up into chunks or streamed using the down arrow, but maybe you want to study it or keep it as a reference.

For some reason you need to format the output from man in a special way before directing it to a file, for example:

man ls | col -bx > results.txt
head -n 8 results.txt
ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]

Nearly the whole alphabet! But doing this without the col leads to selective stuttering:

LS(1)                     BSD General Commands Manual                    LS(1)

l ls s -- list directory contents

l ls s [- -A AB BC CF FG GH HL LO OP PR RS ST TU UW W@ @a ab bc cd de ef fg gh hi ik kl lm mn no op pq qr rs st tu uw wx x1 1] [_ f_ i_ l_ e _ ._ ._ .]

For each operand that names a _ f_ i_ l_ e of a type other than directory, l ls s

head and tail are useful for looking at the first few lines or last few lines of a file. less is like what you get when you just do man some_command ---it gives chunks or streams.

Try man cat to see what else that can do.

There is also split but I couldn't figure out how to make it take my favorite separator ('\n\n').

diff does what you might expect. The use case might be that you have some good edits but also some bad edits to a file, so you want to revert to the old one, and then try the edits one by one.

> diff old.txt new.txt > results.txt
> cat results.txt
> my edit
> deleted a line here

Or drop results.txt onto TextMate for a little color (graphic).

There's a command to find out information about different processes with finer grained control than Activity Viewer. For example:

ps -u username
ps -U userid
ps -p processid


> ps -p 48068 -O %mem
48068 1.1 ?? S 0:00.91 /Applications/ -psn_0_2404939

The man page on ps is extensive.

find is tricky. I need a lot more practice to understand it. Here's a simple one, and then a fun one though:

> find . -name "old.txt" -print

> find -f ~/* | wc
118791 235608 11536780

I "find" all the files under my home directory and feed the results to wc (word count), discovering that there are 118,791 lines in the output---that's the number of files! This link seems to have good info on find.

Sometimes I just want to use Python as a calculator or get an ascii value, but without calling up the interpreter (and having to dismiss it):

> python -c 'print 3.0/17'
> python -c 'print ord("a")'
> python -c 'print chr(101)'
> python -c 'import math; print math.e'

> hexdump -C post.txt
00000000 4d 61 6e 79 20 55 6e 69 78 20 63 6f 6d 6d 61 6e |Many Unix comman|
00000010 64 73 20 61 70 70 65 61 72 20 74 6f 20 62 65 20 |ds appear to be |
00000020 73 69 6d 70 6c 65 20 62 75 74 20 68 61 76 65 20 |simple but have |

Finally, here are a few more that either I've found useful or I'm sure they will be, but I don't feel expert enough to try to explain them. Have fun exploring:

openssl sha1

If you get bored here; or try to check out how Greg's email address gets generated!

And there's always ssh. But that really does deserve a post of its own.

Unix spoken here (3)

Let's do some more with Unix commands. Previous posts here and here.

You might have noticed that I have a very short prompt in Terminal, just the character `>` followed by a space. That's because I have an (invisible) file in my home directory named .bash_profile, containing this line:

> cat ~/.bash_profile
PS1="> "; export PS1

We can change back to the original behavior by changing the name of the file or moving it, and then re-starting Terminal:

mv ~/.bash_profile ~/Desktop/x.txt

c-98-236-78-154:Desktop telliott$

We could comment out that line of the file (now that it's visible with no leading `.` in the filename, we can easily open it in TextEdit).

#PS1="> ";  export PS1

But I like it, so I'll revert to the original situation.

I think there is a way to get TextEdit to show hidden files in the Open dialog, but I forget how at the moment. In any event, it's interesting that having opened and closed the file while it was visible on the Desktop, Open Recent now shows .bash_profile as an option.

Another way to edit the file is to use pico (a command line editor):

> pico ~/.bash_profile

I'm sure you can figure out how to use pico.

One more useful thing in .bash_profile is modification to the $PATH variable. Let's look at that from the command line:

> echo $PATH

When I call ls, for example, these directories are searched in turn until the program is discovered finally. It is in:

> whereis ls

If I wanted to modify the $PATH variable I could do:

> PATH=~/Desktop:$PATH;  export PATH
> echo $PATH

My $PATH is pretty complicated for several reasons: use of MacPorts, some things related to ncbi and X11, etc. It would take too long to explain.

The shell has command history. If you want to repeat a command, just press the up arrow key. To repeat a command further back, keep pushing until you see the one you want. Then press Return. Suppose you have a series of commands (say 3 in a row), and the first of these is 6 commands ago. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Because the command history is updated each time, 6 arrow presses gets you back exactly where you need to be for each of the three commands in the sequence.

Alternatively you could do history:

> history
501 cd Desktop
502 pico ~/bash_profile
503 echo $PATH
504 file ls
505 whereis ls
506 PATH=~/Desktop:$PATH; export $PATH
507 PATH=~/Desktop:$PATH; export PATH
508 echo $PATH
509 history

which lists each command I've entered since the last time I booted my machine (even saves between Terminal sessions). One can select a command from the history by number or by partial name match:

> !505
whereis ls
> !wh
whereis ls

First it prints the command, and then the result. You can combine commands on a single line with `;`

> !504; !512
whereis ls; echo abc
> history | tail -2
515 whereis ls; echo abc
516 history
> !515
whereis ls; echo abc

We're getting pretty good with this stuff! We don't really want to list all 500+ commands in the history, just the last two. So we feed the results of history to the Unix tool called tail through a `|`, a Unix "pipe." tail takes an option for how many lines at the end of the "file" to display.

The Unix command grep searches lines of text for those matching a pattern. It's particularly useful in combination with `|`.

> history | grep pico
40 pico ~/.hgrc
298 pico
502 pico ~/bash_profile
515 history | grep pico

Here, I edited some file with pico but I can't remember which one it was. So the entire output of the history is fed through the pipe to grep, searching for lines containing the pattern. Now I can just do:

> !40
pico ~/.hgrc

I won't show you the file.

The last command for this post is ln with the option -s which makes a "soft link" to a file. I can never remember, but you want to put the existing file first (that's the opposite of the way I would design it):

> cat temp/y.txt
echo "abc"
> ln -s foo temp/y.txt
ln: temp/y.txt: File exists
> ln -s temp/y.txt foo
> ls
foo temp
> cat foo
echo "abc"
> ls -l
total 8
lrwxr-xr-x 1 telliott staff 10 Jan 31 07:53 foo -> temp/y.txt
drwxr-xr-x 5 telliott staff 170 Jan 31 07:53 temp

Now the listing for foo shows an `l` as the first character in the file mode, and it also shows -> temp/y.txt, where it links to.

Sunday, January 30, 2011

Unix spoken here (2)

Let's continue with an introduction to Unix commands. First post here. And here are two more good links (one, two).

Let's start by putting some data into a file. We can use the command echo and the `>` redirection operator. Take a look at what's there with cat:

> cd
> cd Desktop
> echo "abc" > x.txt
> cat x.txt

Suppose we wish to move this file to a new sub-directory. First we construct the new directory and then we move the file with mv:

> mkdir temp
> mv x.txt temp/x.txt
> ls temp
> ls

Let's make the file `x.txt` actually do something, e.g.:

> echo "echo xyz" > temp/x.txt
> cat temp/x.txt
echo xyz
> sh temp/x.txt

Note that we wrote over the previous contents of temp/x.txt and the file system did not warn us first. There is no way to recover from this if it was a mistake!

The sh command executes the text in the file you give it; in this case the output is familiar. Normally, we'd probably use the file extension .sh for such a script, but it's not required. (Even with Python scripts, the .py extension is only required if you want to import a module).

We can also use the she-bang thing (#!) to make the code in a file executable. Use TextEdit to save the following code in a file y.txt on the Desktop:

#! sh
echo abc

Then we do the following. An executable can be executed just by entering its path, but.. there is a restriction that you can't do that for a program in the current directory. You must specify it as the relative path from the directory where we are currently./executable where the `.` means this directory (the restriction has never made a lot of sense to me either, but it's apparently a security hole):

> cat y.txt
#! sh
echo abc
> ./y.txt
-bash: ./y.txt: Permission denied

What has happened? We need to make the file "executable" by setting the "permission bits." Without getting into too much detail, these are output using a new option with the ls command:

> ls -l y.txt
-rw-r--r-- 1 telliott staff 19 Jan 30 20:08 y.txt
> chmod 711 y.txt
> ls -l y.txt
-rwx--x--x 1 telliott staff 19 Jan 30 20:08 y.txt
> ./y.txt

The three permissions are r (read), w (write) and x (execute), listed in order for the user, her group, and the world. 7 is all of them. `x` is 1, `w` is 2, `r` is 4, and the combinations result from addition.

There are various shorthands for this. First reverting the previous change, then we could do this:

> chmod 644 y.txt
> chmod u+x y.txt
> ls -l y.txt
-rwxr--r-- 1 telliott staff 19 Jan 30 20:08 y.txt

We have given the user (only) permission to execute the file.

> ls -l ~/Desktop
total 1
drwxr-xr-x 2 telliott staff 102 Jan 30 20:08 temp
-rw-r--r-- 1 telliott staff 15 Jan 30 20:08 y.txt

When we do ls on the Desktop directory (above), we get a listing for both the temp sub-directory and the file. Note that the first character in the "file mode" is d for the directory. staff is my group name, and after that comes the number of bytes in each "file", including directories.

From man ls

The Long Format
If the -l option is given, the following informa-
tion is displayed for each file: file mode, num-
ber of links, owner name, group name, number of
bytes in the file, abbreviated month, day-of-
month file was last modified, hour file last mod-
ified, minute file last modified, and the path-
name. In addition, for each directory whose con-
tents are displayed, the total number of 512-byte
blocks used by the files in the directory is dis-
played on a line by itself, immediately before
the information for the files in the directory.
If the file or directory has extended attributes,
the permissions field printed by the -l option is
followed by a '@' character. Otherwise, if the
file or directory has extended security informa-
tion (such as an access control list), the per-
missions field printed by the -l option is fol-
lowed by a '+' character.

And finally, ls also has a recursive option, which can be grouped into -lR:

> ls -lR
total 8
drwxr-xr-x 3 telliott_admin staff 102 Jan 31 06:59 temp
-rw-r--r-- 1 telliott_admin staff 15 Jan 31 07:00 y.txt

total 8
-rw-r--r-- 1 telliott_admin staff 4 Jan 31 06:59 x.txt

Let's continue by copying this file into our temp directory:

> cp y.txt temp
> ls
temp y.txt
> ls temp
x.txt y.txt

On the other hand, mv actually "moves" the file:

> mv y.txt temp
> ls
> ls temp
x.txt y.txt

Once again, note the lack of warnings. The old copies of y.txt on the Desktop (and in temp) are gone forever.

Eventually, we tire of this. First we try to delete a single file:

> rm temp/y.txt
> ls temp

But maybe temp has lots of files, so we try:

> rmdir temp
rmdir: temp: Directory not empty
> rm -r temp
> ls

The `-r` option is really dangerous. The classic Unix "joke" is to tell someone to do some combination of rm and -r and *. Don't fall for it.

You might think about looking at the man page for rm and maybe using the -i option.

Unix spoken here (1)

There are probably hundreds of pages on the web with good info about basic Unix commands (e.g. here, here). I thought I'd give some of the commands I think are most useful, but I have to start at the beginning. So, here goes with Unix in 5 minutes, Unleashed, For Dummies..

> cd
> pwd
> cd Desktop
> pwd
> cd ~
> pwd
> cd Desktop
> ls
> cat x.txt
some data

The first command, cd, says to change directory; with no other argument given, we go to our home directory. To see where we are in the file system we can do pwd: print working directory. Also, the ~ symbol is a shorthand for the user's home directory.

The "shell" is smart and has "command-completion." For example, from my home directory I enter cd De and then press tab. The shell fills out "Desktop."

Another very common command is ls: list the contents of the current directory (since there is no argument). And to see the contents of a file you might do cat with the results shown.

> ls -a
. .DS_Store .temp
.. .localized x.txt
> ls /usr/bin/pyth*
/usr/bin/python /usr/bin/python2.6-config
/usr/bin/python-config /usr/bin/pythonw
/usr/bin/python2.5 /usr/bin/pythonw2.5
/usr/bin/python2.5-config /usr/bin/pythonw2.6

Any directory may have "hidden" files whose names are prefaced by a dot `.`. These can be shown with ls -a. There are many (many) other options with results cataloged under man some_command.

The shell understands paths to files in both an absolute and relative sense. A path starting with / is absolute, otherwise it's relative. Hence /usr/bin is a directory starting from the root of the file system, while Desktop is only understood as a valid destination if I'm in my home directory (or some other directory with Desktop as a sub-directory).

The character * is a wild card and matches anything; ls /usr/bin/pyth* lists all the entries matching pyth- + something in the given directory.

> pushd /usr/bin
/usr/bin ~/Desktop
> pwd
> popd
> pwd
> cat -n ~/Desktop/x.txt
1 some data
> cal
January 2011
Su Mo Tu We Th Fr Sa
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31
> cd ..
> pwd

Sometimes we want to make a jump in the file system but intend to come right back. pushd and popd are useful here, with results as shown.

As one more example of using options or arguments to a command, cat takes the option -n, which numbers the output lines. cal is probably faster than finding the, since I don't usually have it in the Dock.

Finally, to move toward the tips of the file system tree, you specify the directory at the next level so it has to be explicit, but there is only one parent, which we can reach implicitly with cd .., meaning "go up."

Saturday, January 29, 2011

Distributed Objects on OS X (2)

I struggled for a while to get Distributed Objects working between different machines (first post here for same-machine example). According to the docs and this, you should just use

NSSocketPortNameServer *name_server;
name_server = [NSSocketPortNameServer sharedInstance];
NSLog(@"%@", [name_server description]);
BOOL result;
result = [conn registerName:@"my_server"

But it didn't work for me. So I asked a question on SO (here), and asked the Google. The latter approach finally worked. I got a working example with code from here. The discussion in the docs for the NSConnection class helped too (see initWithReceivePort:sendPort:. If you set it up with nil for the sendPort, it uses the same port for both sending and receiving, which is what you want.)

[UPDATE: On reflection, it does seem a bit weird, because a Mach port is supposed to be a one-way channel, yet these are clearly two way. I guess they are probably complex objects built from multiple NSMacPorts. ]


// gcc talk3.m -o test -framework Foundation
#import <Foundation/Foundation.h>

@interface VendedObject:NSObject {}
-(NSString *) speak;

@implementation VendedObject

-(NSString *) speak {
return @"woof";

@interface NSConnection (NetworkServiceAdditions)

+ (id) networkServiceConnectionWithName:(NSString *) inName
rootObject:(id) inRootObject;


@implementation NSConnection (NetworkServiceAdditions)

+ (id) networkServiceConnectionWithName:(NSString *) inName
rootObject:(id) inRootObject;
NSSocketPort *port = [[[NSSocketPort alloc] init] autorelease];
NSConnection *connection = [NSConnection connectionWithReceivePort: port
sendPort: nil];

[[NSSocketPortNameServer sharedInstance] registerPort:port
[connection setRootObject: inRootObject];
return connection;


int main(){
NSAutoreleasePool *pool;
pool = [[NSAutoreleasePool alloc] init];
VendedObject *obj;
obj = [[[VendedObject alloc ] init] autorelease];
NSLog(@"%@", [obj description]);
NSConnection *conn;
conn = [NSConnection networkServiceConnectionWithName:@"my_server"
rootObject:obj ];
NSSocketPortNameServer *name_server;
name_server = [NSSocketPortNameServer sharedInstance];
NSLog(@"%@", [name_server description]);
NSLog(@"%@", [name_server portForName:@"my_server"]);
[[NSRunLoop mainRunLoop] run];
[pool drain];
return 0;

Note: if you're not just killing the Server (as I did) you should do invalidate on the connection (and I guess the ports too?).

Client is on my laptop:

from Foundation import *

proxy_obj = NSConnection.rootProxyForConnectionWithRegisteredName_host_usingNameServer_(
"my_server", "osxserver.local", NSSocketPortNameServer.sharedInstance())

if not proxy_obj:
print 'Did not get an object from the server.'
print proxy_obj.description()
print type(proxy_obj)
print proxy_obj.speak()


> python 
<VendedObject: 0x10010ced0>
<objective-c class NSDistantObject at 0x7fff70a64868>

It works!

NSTask again

I have a bit more to explore about NSTask (previous post here). There is some discussion on SO about the difference between an NSTask and an NSThread (here).

The script uses the #! thing and is set to be executable (by the user only). The code looks for an int in argv and a second arg that tells whether to examine the environment variable dictionary.

#! /usr/bin/python
import sys, os

n = int(sys.argv[1])
n = 2
print 'woof ' * n
if len(sys.argv) == 3:
D = os.environ
k = 'MYKEY'
if k in D: print k, D[k]
else: print k, 'not present'

The first attempt just launches the task without setting it up first.

The second example does the setup. The path p2 (to Python directly) isn't used in these versions but I left it in. For a while, I was getting errors when I tried to setup the task and then launch it unless I fed it to Python first and gave the script path as an argument. Then, the errors just stopped. I can't recreate the problem. This kind of bug that "just disappeared" happened with the environment dictionary as well. No clue as to why.

from Foundation import *

p1 = NSHomeDirectory() + '/Desktop/'
p2 = '/usr/bin/python'

def f1():
t = NSTask.launchedTaskWithLaunchPath_arguments_(
p1, [str(1)])

def doit(t):

def f2():
t = NSTask.alloc().init()


Of note here is that the arguments must be in a list or array, and that the int argment must be converted to a string. The error if not is weird (but at least we're pointed to the correct line):

2011-01-29 03:18:00.109 Python[19298:60f] -[OC_PythonNumber fileSystemRepresentation]: unrecognized selector sent to instance 0x101a6d950
Traceback (most recent call last):
File "", line 48, in <module>
File "", line 15, in f1
p1, [1])
ValueError: NSInvalidArgumentException - -[OC_PythonNumber fileSystemRepresentation]: unrecognized selector sent to instance 0x101a6d950

Otherwise we get this:

> python 
woof woof

Next, we instantiate the task and then set its arguments and environment, having modified the dict to have an extra key/value pair:

D = NSProcessInfo.processInfo().environment()

def f3():
t = NSTask.alloc().init()
n = str(3)
print 'MYKEY', t.environment()['MYKEY']


> python 
woof woof woof

The ultimate line of output is from the task and shows that we did get the modified environment.

And in the last example, we use an NSFileHandle and redirect standard output. This could be useful in the case where you don't have full control over the task that is being executed. I found that NSFileHandle.fileHandleForWritingAtPath_ apparently requires the file to already exist.

p3 = NSHomeDirectory() + '/Desktop/results.txt'

def f4():
t = NSTask.alloc().init()
n = NSNumber.numberWithInt_(4).stringValue()
# file must exist or this fails!
f = NSFileHandle.fileHandleForWritingAtPath_(p3)
data = f.readToEndOfFileInBackgroundAndNotify()
print data


> python 
> cat results.txt
woof woof woof woof

The redirect is good, but I couldn't get the file read to work. I tried before and after launch, and I tried re-grabbing the file handle, and I also tried readDataToEndOfFile. Not sure what's up with that yet.

[UPDATE: Perhaps the answer has something to do with NSPipe (here). ]

Thursday, January 27, 2011

Decorators in Python

There is a nice simple post about Python decorators from Bruce Eckels (here; discussion on Stack Overflow here, docs here, more here). I copied some of his example code and show it below, but first, the output of the run:

> python 
Entering func1
inside func1()
Exited func1
Entering func2
inside func2()
Exited func2

The main point is that a decorator is a kind of macro for functions. It changes them, and can do so in almost unlimited ways. The output above shows that the decorator code can be executed before or after the function itself is called. (And he notes that you don't even have to do that!) They're used quite a bit in PyObjC (e.g. @objc.signature('v@0:@8')). I should probably explore all of that in another post.

As Bruce says:

Decorators allow you to inject or modify code in functions or classes

The only constraint upon the object returned by the decorator is that it can be used as a function -- which basically means it must be callable. Thus, any classes we use as decorators must implement __call__.

What should the decorator do? Well, it can do anything but usually you expect the original function code to be used at some point. This is not required, however.

class entryExit(object):

def __init__(self, f):
print 'init'
self.f = f

def __call__(self):
print "Entering", self.f.__name__
print "Exited", self.f.__name__

def func1():
print "inside func1()"

def func2():
print "inside func2()"

print 'start'

A peek into the Objective-C "runtime"

I'm playing with Cocoa today. I ran across a post (here), reminding me that all our fancy "messages" get changed by the Objective-C "runtime." The real call is objc_msgSend and you can do that yourself if you want:

// gcc test.m -o test -framework Foundation
#import <Foundation/Foundation.h>

@interface MyClass:NSObject {}
-(NSString *) speak:(int) n;

@implementation MyClass
-(NSString *) speak:(int) n {
int i;
NSMutableArray *mA;
mA = [NSMutableArray arrayWithCapacity:n];
for (i=0; i<n; i++) {
[mA addObject:@"woof"];
return [mA componentsJoinedByString:@"-"];

int main(){
NSAutoreleasePool *pool ;
pool = [[NSAutoreleasePool alloc] init];
MyClass *obj;
obj = [[[MyClass alloc ] init] autorelease];
NSLog(@"%@", [obj description]);
NSLog(@"%@", [obj speak:2]);

NSString *s;
int count = 3;
SEL sel = @selector(speak:);
s = objc_msgSend(obj,sel,count);
NSLog(@"%@", s);
[pool drain];
return 0;

Note: you have to use an int variable, you can't just pass an int directly to objc_msgSend.

> ./test
2011-01-27 13:21:37.890 test[38774:903] <MyClass: 0x10010cd20>
2011-01-27 13:21:37.892 test[38774:903] woof-woof
2011-01-27 13:21:37.893 test[38774:903] woof-woof-woof

Distributed Objects on OS X

I got a simple example with Distributed Objects working. This is supposed to be the standard way to communicate between processes. I tried it first with both the Server and the Client in PyObjC. Eventually I rewrote the Server in Objective-C and then it worked.

[UPDATE: Found a great working example (old, but from an Apple engineer here). ]

The output is:

> python 
<VendedObject: 0x10010cce0>
<objective-c class NSDistantObject at 0x7fff70a64868>

I'm not sure what the problem was but the error I got was

[OC_PythonString initWithBytes:length:encoding:]: unrecognized selector sent to instance 0x3635130


// gcc talk.m -o test -framework Foundation
#import <Foundation/Foundation.h>

@interface VendedObject:NSObject {}
-(NSString *) speak;

@implementation VendedObject

-(NSString *) speak {
return @"woof";

int main(){
NSAutoreleasePool *pool;
pool = [[NSAutoreleasePool alloc] init];
VendedObject *obj;
obj = [[[VendedObject alloc ] init] autorelease];
NSLog(@"%@", [obj description]);

NSConnection *conn;
conn = [[[NSConnection alloc] init] autorelease];
[conn setRootObject:obj];
BOOL result;
result = [conn registerName:@"my_server"];
if (!result) {
NSLog(@"Failed to register Name");
else {
NSLog(@"%@", [conn description]);
[[NSRunLoop mainRunLoop] run];
[pool drain];
return 0;

Run with:

> ./test
2011-01-27 12:09:39.001 test[36995:903] <VendedObject: 0x10010cce0>
2011-01-27 12:09:39.005 test[36995:903] (** NSConnection 0x100112050 receivePort <NSMachPort: 0x100112530> sendPort <NSMachPort: 0x100112530> refCount 1 **)

You'll have to kill the process when you're done. If you have tried once already in Terminal, you will need to Quit Terminal and Re-launch. Otherwise registerName: will fail.


from Foundation import *

proxy_obj = NSConnection.rootProxyForConnectionWithRegisteredName_host_(
#"my_server", "osxserver.local")
"my_server", None)
if not proxy_obj:
print 'Did not get an object from the server.'
print proxy_obj.description()
print type(proxy_obj)
print proxy_obj.speak()

I also tried to connect to another machine on my local network, but I couldn't get that to work either. The proxy_obj was nil/None.

Wednesday, January 26, 2011


I want to send information between two running applications and I mistakenly thought I could use an NSNotification to do this. [See the UPDATE].

So this post is about the first step, getting an NSNotification to work within a running app. The following is from an Xcode Cocoa-Python app, and in we have:

from Foundation import *
from AppKit import *
import objc

class SpeakAppDelegate(NSObject):
def applicationDidFinishLaunching_(self, sender):
NSLog("Application really did finish launching.")
nc = NSNotificationCenter.defaultCenter()
self, "mycallback:", 'love_note', None)

def mycallback_(self,note):
print 'note'
print note.description()

def button_(self,sender):
print sender, 'button'
nc = NSNotificationCenter.defaultCenter()
'love_note', None, {'path':'xyz'})

There's a button in the window hooked up to the IBAction. I push the button and get:

<NSButton: 0x61c7b0> button
NSConcreteNotification 0x6337a0 {name = love_note; userInfo = {
path = xyz;

But I couldn't get it to work between apps, for example with this one compiled in Terminal:

// gcc tell.m -o test -framework Foundation
#import <Foundation/Foundation.h>

int main() {
NSNotificationCenter *nc;
nc = [NSNotificationCenter defaultCenter];
[nc postNotificationName:@"love_note"
userInfo:nil ];
return 0;

I asked on Stack Overflow (here), and they say I should be using Distributed Objects (here). So I have to look into that.

[ UPDATE: Just substitute NSDistributedNotificationCenter in the code above. It works! ]

Saturday, January 22, 2011

Nullspace of a vector

This post is a note to myself about linear algebra.

Let's consider two vectors in R2:

a = ( 3, 4) = [  3 ]
[ 4 ]

b = (-3, 1) = [ -3 ]
[ 1 ]

We can see from a plot that these vectors a and b are obviously not pointing in the same direction.

Also we notice that a is not a multiple (or a linear combination) of b (or vice versa). There is no constant such that when the elements of a are multiplied by the constant we get b.

Now consider a point P in R2, say

P = (2,-4)

We can construct a linear combination of a and b that reaches this P or any other point.

How? Simply place one of the vectors at the origin and move along it (perhaps in a negative direction), and place the second vector at P and move along it, and find where the two lines meet.

Call the coordinates of the point where the lines cross x and y. There are actually two possibilities depending on which vector we choose for each role. Suppose we move in the reverse direction from the origin along a and from P along b. From the point-slope equation we know that:

(0 - y) / ( 0 - x) =  4/3  (from a)

(Py - y) / (Px - x) = -1/3 (from b)
(-4 - y) / ( 2 - x) = -1/3

We can solve these two equations pretty easily. From the first

y = 4/3 x

and from the second:

(-12 - 3y) = -(2 - x)
x = -10 - 3y
= -10 - 3(4/3) x

5x = -10
x = -2
y = -8/3

The other solution is symmetrical (we're dealing with a parallelogram). We would need to go +2 across and +8/3 up from P to reach x, y = (4, -4/3).

Suppose we want to know the actual multipliers for a and b, i.e. the fraction of the length of a and b that we travel along each vector.

From the figure, we can estimate that the values will be about -0.7 and -1.25.

Call these multipliers u and v. In matrix language

M = [ 3 -3 ]
[ 4 1 ]

[ 3 -3 ] [ u ] = [ 2 ]
[ 4 1 ] [ v ] [ -4 ]

One way we can solve the system (call it the Algebra 2 way) is as follows:

3u - 3v = 2
4u + v = -4

Solve the second equation for v:

v = -4u - 4

Plug into the first equation:

3u - 3(-4u - 4) = 2
15u = -10
u = -2/3
v = 8/3 - 4 = -4/3

Finally, consider P not just as a point but as a vector c:

a = ( 3, 4)
b = (-3, 1)
c = ( 2,-4)

Since we could have chosen P to be any point, c could be any vector in the x,y-plane and it would be constructed as a linear combination of a and b:

u a + v b

We say that c is in the column space of the matrix whose columns are the vectors a and b:

M = [ 3 -3 ]
[ 4 1 ]

because we can get to c by taking a linear combination of the columns of M.

And of course, having a + b = c, we can also add -c to get:

u a + v b - c = 0

If we think of another matrix as the column vectors a, b and c lined up:

[ 3 -3  2 ] [ u ] = [ 0 ]
[ 4 1 -4 ] [ v ] = [ 0 ]
[ w ] = [ 0 ]

The combination u = -2/3, v = -4/3, w = -1 solves this equation and brings us back to zero:

-2/3 [ 3 ] -4/3 [ -3 ] - [  2 ] = [ 0 ]
[ 4 ] [ 1 ] [ -4 ] [ 0 ]

[ -2 ] + [ 4 ] + [ -2 ] = [ 0 ]
[ -8/3 ] [ -4/3 ] [ 4 ] [ 0 ]

That solution (a vector in R3):

x = [ u ] = [ -2/3 ]
[ v ] [ -4/3 ]
[ w ] [ -1 ]

is said to be in the nullspace of

A = [ 3 -3  2 ]
[ 4 1 -4 ]

because Ax = 0.

Thursday, January 20, 2011

Sal Khan rocks

I'm always the last person to learn about cool new stuff, so when I say I was scouting around on the web today and ran into Sal Khan and his Khanacademy, I expect you will say, I know about him already. I mean, it's 10,000,000 downloads and counting, or something.

If you don't know, the site consists of a very large number of videos on math, science, economics and a few other topics (here). Since Sal's been interviewed a lot, if I still watched PBS reliably, I would've known about him already. But I decided a while ago to watch as little TV as possible.

Several things to say about the material:

(1) the scope is incredible
(2) the depth is also incredible
(3) the quality is excellent

I'm using it to flesh out what I'm getting from Gil Strang on Linear Algebra.

He also seems to be a really nice guy. Check it out!

Wednesday, January 19, 2011


I just got a helpful comment on an old post (here) suggesting the use of an NSCountedSet. I'd never heard of this class, but it's described in the Apple docs about Collections (here). Besides the usual characters


there are several others including:


Let's show off the last one, from PyObjC. Here's the listing:

from Foundation import *

L = list('xyyzzz')
S = NSCountedSet.alloc().initWithArray_(L)

def show(S):
e = S.objectEnumerator()
while True:
obj = e.nextObject()
if not obj: break
print obj, S.countForObject_(obj)

N = S.countForObject_('z')
for i in range(N):
print 'z', S.countForObject_('z')
print 'w', S.countForObject_('w')

and the output:

x 1
y 2
z 3

z 2
z 1
z 0
w 0

It feels a bit strange, not only can you get the count of an object that's not in the set (as in Python), but as the docs say:
removeObject: does nothing if anObject is not present in the set

Sunday, January 16, 2011

One space or two

It's a little depressing but also undeniable that I am getting to be an old fart. My birthyear satisfies this equation:

2010 - b == b % 100

For today what it means is that I learned to type on a typewriter, an archaic device you can read about in wikipedia (here). My first two were unpowered. In typing class, my teacher was very strict: it was an error if you did not have two spaces separating sentences. In typography, or even when typing on a computer, the rules have changed, but legions of us still type to please the critical eye of someone like Brother McDermott.

Apparently that drives certain people crazy (here) or just leaves them bemused (here and here).

I still do it, as you can see it in the source for the blog. It's arguably correct for non-proportional fonts. But, in the end it doesn't matter because html simply ignores the extra space.

What does matter and I find incredibly useful is to separate groups of multi-line data by a double newline. Then I can just do:

L = data.strip().split('\n\n')

It's mildly irritating when I send off a FASTA file in that format and it comes back without the extra newline. Then I have to split on '>', throw the first item in the list away, and add back the '>'. Arggh.

Another useful Python idiom is:

title, seq = item.strip().split('\n',1)

I always strip, just to be safe.

Saturday, January 15, 2011

A series for pi

I saw something really nice in Strang (Calculus). It starts with inverse trigonometric functions, which are hard for me to think about, but let's go slowly and hope for the best:

y = sin x
x = sin-1 y (-π/2 < x < π/2)

x is the angle whose sin is y. We draw a picture, and see that:

sin-1 y + cos-1 y = π/2


cos x = √ (1-y2)

since we're dealing with the inverse function sin-1 y:
slope of inverse = 1 / slope of original function

y = sin x
dy/dx = cos x
dx/dy = 1/cos x = 1/√ (1-y2)

Since sin-1 y + cos-1 y = π/2, if:

x = cos-1 y
d/dy (cos-1 y) = - d/dy(sin-1 y)
dx/dy = -1/√(1 - y2)

And our goal, remembering from here:

(u/v)' = (v u' - u v')/v2

y = tan x
= sin x / cos x
dy/dx = (cos2 x + sin2 x) / cos2 x
dy/dx = sec2 x

sec2 x = 1 / cos2 x
= (sin2 x + cos2 x) / cos2 x
= 1 + tan2 x

x = tan-1 y
dx/dy = 1 / sec2 x
= 1/(1 + tan2 x)
= 1/(1 + y2)

So, if we integrate 1/(1 + y2) dy, we get tan-1 y. We're going to use that.

The geometric series is:

1 + x + x2 + x3 + .. = 1/(1 - x)
(converges for -1 < x < 1)

How do we derive this? One way (not legal) is to assume that the series really does converge to a sum S

1 + x + x2 + .. = S
x + x2 + x3 + .. = xS
1 + x + x2 + x3 + .. = 1 + xS
1 + xS = S
S - xS = 1
S = 1/(1 - x)

Or we can check it by just multiplying out:

(1 - x) (1 + x + x2 + x3 + ..) = 1
= 1 + x + x2 + x3 + ..
- x - x2 - x3 + ..
= 1

Replace x by -x2

1 - x2 + x4 - x6 .. = 1/(1 + x2)

If we integrate both sides, the rhs is tan-1 x. And the lhs is:

x - x3/3 + x5/5 - x7/7

The magic: if we let x = 1, then:

1 - 1/3 + 1/5 - 1/7 + .. = tan-1 1 = π/4

Another series for π, how cool is that?

Thursday, January 13, 2011

Finding Python (for PyObjC) 3

Just a short post to say that I asked about this on Stack Overflow and got a very nice answer from Ned Deily (here). He explains clearly what the different "executables" are about, and reminds me of a tool I'd overlooked: otool. For example

> otool -L /System/Library/Frameworks/Python.framework/Versions/2.6/Resources/
/System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

Although it is not so helpful here:

> otool -L /System/Library/Frameworks/Python.framework/Versions/2.6/bin/python
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

I still think the export DYLD_PRINT_LIBRARIES=1 trick is pretty cool.

You can also look into nm (like here).

I found at Source Forge (here) and tried it. It gave me quite a bit of insight into the structure of our simple examples from the other day (here). The screenshot above is from a run on its own executable. There is very detailed info on the object code (or images in geek-speak).

I had been a bit worried about it since I can't find out anything about the guy (Peter Saghelyi). But I found the source, which is not under files. Do:

svn co machoview

Wednesday, January 12, 2011

Matt Gallagher, artist

I hope you find this as entertaining as I did. A Cocoa App (with a window, even) but not using Xcode). The App itself is a curiosity---though not so much of one as Amit Singh's even shorter program (see the link for details)---but Matt Gallagher obviously knows his stuff, and I can see it serving as a harness for various simple tests.

Maybe I should make a sidebar populated with blogs like his. Check it out.

Tuesday, January 11, 2011

Build Python on OS X with clang

I've been fooling around with clang as a gcc replacement. (I'd love to have Xcode 4 but it's not worth the $99). Here we build Python 2.7 in record time:

mkdir ~/temp
cd ~/temp
curl -O
tar -zxf Python-2.7.1.tgz
mkdir Python

The date output:

Wed Jan 12 15:35:26 EST 2011

cd Python-2.7.1/
export CC=clang
./configure --prefix=/Users/telliott_admin/temp/Python --without-gcc
make && make install

Wed Jan 12 15:38:02 EST 2011

2 minutes and 36 seconds!

Python 2.7.1 (r271:86832, Jan 12 2011, 15:36:42)
[GCC 4.2.1 Compatible Clang Compiler] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version
'2.7.1 (r271:86832, Jan 12 2011, 15:36:42) \n[GCC 4.2.1 Compatible Clang Compiler]'
[1]+ Stopped ~/temp/Python/bin/python

The time stamp is interesting..

> file ~/temp/Python/bin/python
/Users/telliott_admin/temp/Python/bin/python: Mach-O 64-bit executable x86_64

cd ..
curl -O
> cd Python/bin
> ls
2to3 pydoc python2.7-config
easy_install python
easy_install-2.7 python-config
idle python2.7

~/temp/Python/bin/easy_install-2.7 numpy
ImportError: No module named numpy.distutils

just hit it again and it'll finish

Python 2.7.1 (r271:86832, Jan 12 2011, 15:36:42)
[GCC 4.2.1 Compatible Clang Compiler] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import *

However, I was unable to do a framework build to this location, and I was unable to build matplotlib either, at least so far.

Sunday, January 9, 2011

Python for PyObjC

This is a short post to document that I was able to download the's Framework build of Python 2.7 and then install matplotlib in it, as well as modify a Cocoa-Python application to link against that Framework. The only thing that looks a little shaky is the PyObjC install, but it still seems like it's working. Not every detail is given, but I hope it's enough.

The installer comes from here for python-2.7.1-macosx10.6.dmg (Mac Installer disk image (2.7.1) for OS X 10.6 and later).

I did a stock install except I unchecked Shell profile updater under Custom.. Not sure why, except I didn't want to spend time figuring out what it does. installs stuff in


I added this to .bash_profile

> export PATH=/$PATH:/Library/Frameworks/Python.framework/Versions/2.7/bin

I anticipated that easy_install would be difficult, but I got (something like) it from distribute:

> curl -O
> /usr/local/bin/python

I couldn't find any decent docs, though. It looks like they've just copied the easy_install documentation which I found impossible to figure out so far. But, poking around, I found easy_install and easy_install-2.7 in bin:

> which easy_install-2.7

The way my $PATH is set up, easy_install runs the wrong Python. That's OK, we just do:

> easy_install-2.7 -U numpy
> easy_install-2.7 pyobjc==2.2
> easy_install-2.7 -U pip

pyobjc has lots of warnings but looks OK

> /usr/local/bin/python 
Python 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
>>> import objc
>>> objc.__version__
>>> from Foundation import *
[1]+ Stopped /usr/local/bin/python

Next, I installed matplotlib using instructions from Gavin Huttley (here, here). Since matplotlib-1.0.0 is available, I went for it. In the make.osx file:

ARCH_FLAGS="-arch i386-arch x86_64"

I changed it to Python 2.7.

I also needed to update to libpng-1.2.39 and libfreetype-2.3.11 (although these are not the latest versions.. In fact, I had a bit of trouble with libpng since I got back a file that was really html, though titled as .gz or .bz2 or .xz. I finally found it here). For the other one:

> curl -O{$FREETYPEVERSION}.tar.bz2

As I said, the build instructions were as given at the link.

[UPDATE: Poking around in libpng file INSTALL, I notice that it says: "Before installing libpng, you must first install zlib, if it is not already on your system." So I'll do it in that order next time.]

I tested matplotlib by running the script from here.

Then, I got PyCogent (download link):

> cd PyCogent-1.5
> /usr/local/bin/python build
> sudo /usr/local/bin/python install
>>> from cogent import *
[2]+ Stopped /usr/local/bin/python

> cd tests
> /usr/local/bin/python "$@"

I had a few failures:

Ran 3602 tests in 184.766s

FAILED (failures=12, errors=4)

Some of these are related to executables not present. Looks pretty good. Finally, let's setup a new Xcode project:

Xcode > Cocoa-Python application

change SDK to 10.6
delete Python.framework
drag in new Python.framework
under Targets > X > Link Binary ..
drag in new Python.framework

Since our Python is a universal binary with 64-bit:

> /usr/local/bin/python
Python 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; print hex(sys.maxint)
[2]+ Stopped /usr/local/bin/python
> /usr/local/bin/python -c "import struct; print struct.calcsize('P')"
> /usr/local/bin/python -c "import platform; print platform.architecture()"
('64bit', '')

Build the Xcode project as 64-bit as well. Add this to

import sys
print sys.version
print hex(sys.maxint)

And run it from the Console:

2.7.1 (r271:86882M, Nov 30 2010, 10:35:34) 
[GCC 4.2.1 (Apple Inc. build 5664)]

It works!

Saturday, January 8, 2011

More hidden treasure in Python

Saw another "easter egg" for the first time this morning (here).

It was tantalizing enough that I grabbed Python 2.7 from Macports:

> sudo port install python27
> /opt/local/bin/python2.7
Python 2.7.1 (r271:86832, Jan 8 2011, 09:26:04)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import antigravity

But I don't want to spoil the secret completely.

Hidden treasure in Python

Some fun Python history on this:

> python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

How was this ever found? It's not in
>>> dir(__builtins__)

Finding Python (for PyObjC) 2

One last post about PyObjC issues and finding which Python is running (under OS X) and then I'm done with it. Until I have another problem.. The structure of a framework is like this:

MyFramework -> Versions/Current/MyFramework
Resources -> Versions/Current/Resources
Current -> A

From the docs, a framework normally has a structure like that shown above, where (if A is the current version) the actual library would be:


The reason for the post is that I realized I've been overlooking something obvious: the existence of an Info.plist file. Just as we set the key: NSPrincipalClass, to have the value: SimpleMessage in our bundle (here), there is a key in the Python.framework:

cat Resources/Info.plist

This key is not required to be set for a framework (a framework doesn't have to contain any code), but if it is, you'd expect it to be the path to the executable. Although there is something in the Framework docs about this key must be the same as the name of the framework. So the executable should be:


As far as I can see, our best approach is still to snoop on the loader. And to solve the original problem (no PyObjC) by making sure that we're running System Python.

Friday, January 7, 2011

Finding Python (for PyObjC)

This is a post exploring how to hunt down which Python is launched when we build and run a Python Cocoa application using Xcode and the templates from here.

The problem to be solved is that in some situations, the wrong Python is run, and it doesn't have PyObjC installed, so the statement import objc fails with ImportError, and the App terminates.

In order to fix that, we need to figure out which Python is running and either (i) change which one is run, (ii) remove the "wrong" one, (iii) install PyObjC into the "wrong" one or (iv) provide a path to PyObjC. I fixed my problem before by (ii). I never got (iii) working because I haven't figured out yet how to install easy_install when it's not there yet, or alternatively build PyObjC from scratch.

What we'll see here leads to the recommendation that you try (i).

We can start our troubleshooting by inserting this code into before the import of objc, as shown:

import sys
print sys.version
import objc

2.5.4 (r254:67916, Jun 24 2010, 21:47:25) 

Where does this come from? To begin with, it does not depend on this line in main.m:


The line can be commented out and everything still runs fine. I think it may be roadkill left over from the old days.

The second thing is that if we look at the Xcode build settings for either the project or the target we see:

Base SDK Mac OS X 10.5

The name or path of the base SDK being used during the build. The product will be built against the headers and libraries located inside the indicated SDK. This path will be prepended to all search paths, and will be passed through the environment to the compiler and linker. Normally, this path is set at the project level via the "Cross-Develop Using Target SDK" popup in the General tab of the project inspector. Additional SDKs can be specified in the ADDITIONAL_SDKS setting. [SDKROOT]

Since the default SDK was set for 10.5, at link time we expect to link against:


If we double-click on the Python.framework (under Frameworks > Linked Frameworks in the Xcode project) we get a Finder window open to:


If we do get info with the Python.framework selected we get

Name: Python.framework
Path: /System/Library/Frameworks/Python.framework

but all of this says nothing about the version so one might expect it to go with Current, which is actually Python 2.6:


I've looked at environment info. Put this in

import os
for k in os.environ:
print k, os.environ[k]

The only thing interesting is:

PATH /Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin
PYTHONPATH /Users/telliott_admin/Desktop/X/build/Debug/
DYLD_FRAMEWORK_PATH /Users/telliott_admin/Desktop/X/build/Debug
DYLD_LIBRARY_PATH /Users/telliott_admin/Desktop/X/build/Debug

The PATH variable is not the one from my shell. But there isn't any Python in /Developer/usr/bin and both Python and python in /usr/bin give 2.6. So that's not where it comes from. $PYTHONPATH just points to the App.

Let's try to find out more using this trick:

> ~/Desktop/X/build/Debug/
dyld: loaded: /System/Library/Frameworks/Python.framework/Versions/2.5/Python
2.5.4 (r254:67916, Jun 24 2010, 21:47:25)
[GCC 4.2.1 (Apple Inc. build 5646)]
dyld: loaded: /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC/objc/

We're clearly loading the 2.5 framework. And this behavior can be changed by changing the SDK in Xcode. However, I can't show you anything more right now because I no longer have a failing example! (here)

I don't see anything in any of the settings for the Project, Target or Executable that would explain which Python is actually launched when the App runs. (As you'll see in a minute, there are two choices even for the Python.framework, for example in Version 2.5).

There is some weirdness about the system framework. On three different machines running OS X 10.6.5: Python at top-level in the 2.5 framework seems to be a dynamically linked library.

But it actually runs /usr/bin/Python which is 2.6!:

> cd /System/Library/Frameworks/Python.framework/Versions/2.5
> ls -al P*
-rwxr-xr-x 1 root wheel 3702720 Nov 6 21:53 Python
> Python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
> file Python
Python: Mach-O universal binary with 3 architectures
Python (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
Python (for architecture i386): Mach-O dynamically linked shared library i386
Python (for architecture ppc7400): Mach-O dynamically linked shared library ppc

The whole second half of the post, and the part just above, is a hideous mistake. See here for details.

The stupid, it burns.

Or as my boss likes to tell me, "some days you're the dog, and some days you're the hydrant"---he's amused, me not so much.