Parallelizing Evaluations During Search via MongoDB

Hyperopt is designed to support different kinds of trial databases. The default trial database (Trials) is implemented with Python lists and dictionaries. The default implementation is a reference implementation and it is easy to work with, but it does not support the asynchronous updates required to evaluate trials in parallel. For parallel search, hyperopt includes a MongoTrials implementation that supports asynchronous updates.

To run a parallelized search, you will need to do the following:

1. Install mongodb
2. Start a mongod process somewhere network-visible.
3. Modify your call to hyperopt.fmin to use a MongoTrials backend connected to that mongod process.
4. Start one or more hyperopt-mongo-worker processes that will also connect to the mongod process, and carry out the search while fmin blocks.

Parallel fmin() with MongoTrials

import hyperopt
from hyperopt import fmin, tpe, hp

from hyperopt import Trials
from hyperopt.mongoexp import MongoTrials

trials = Trials()

trials2 = MongoTrials('mongo://localhost:1234/my_db')

def fn(x, y):
     return {
         'loss': x ** 2 + y ** 2,
         'status': hyperopt.STATUS_OK,
     }

fmin(fn,
    space=[hp.uniform('a', -3, 3), hp.normal('b', 1, 1)],
    algo=tpe.suggest,
    max_evals=100,
    #trials=trials,  # -- runs locally, synchronously
    trials=trials2,  # -- runs async
    )

print trials