Allow use of depends classes in the namespace of setup, compute and cleanup; and allow injection of pickleable python objects into the global namespace of the cluster.

[matthewwardrop]

This is my old patch from #11 revived against the current master.

As per #18 the first commit "fixes" the use of classes passed via the depends argument of JobCluster. The second commit allows users to inject pickleable python objects into the global namespace of the computation (which will be performed on the server).

With partial functions implemented, the second commit is not strictly required, but as in #17, I feel that this reduces the boilerplate significantly, and is therefore worth consideration.

A simple example demonstrating its (potential) usefulness is typeset below:

class Greeter(object):

    def __init__(self, greeting='Hello'):
        self.greeting = greeting

    def greet(self, name="Matthew"):
        print "%s %s" % ( self.greeting, name)

def compute(name):
    greeter.greet(name)

def callback(job):
    print job.stdout

def setup_simple():
    global greeter
    greeter = Greeter(salutation)
    return 0

from dispy import JobCluster

class BigComplexClass(object):
    greeting = "Hiya"
    names = ['Matthew', 'Mark', 'Luke', 'John']*100

    def run_jobserver(self):

        g = Greeter(greeting='best')

        cluster = JobCluster(compute,
            setup=setup_simple, # This will work
            globals={'salutation': self.greeting},
            depends=[Greeter],
            callback=callback)

        for name in self.names:
            cluster.submit(name)

        cluster.wait()
        cluster.stats()
        cluster.close()

if __name__ == "__main__":
    bcc = BigComplexClass()
    bcc.run_jobserver()

[matthewwardrop]

I should also note that this significantly reduces the work necessary in "cleanup"; since everything created by setup is thrown away at the end of the computations (except for modules).