Skip to main content

Distributed global function search via dlib

Project description

distgfs

Distributed computing framework for the Global Function Search (GFS) hyperparameter optimizer from the Dlib library. Based on gfsopt.

Provides the following features:

  • Parallel optimization: Run distributed hyperparameter searches via mpi4py.
  • Save and restore progress: Save/restore settings, parameters and optimization progress to/from HDF5 file.
  • Average over multiple runs: Run a stochastic objective function using the same parameters multiple times and report the average to Dlib's Global Function Search. Useful in highly stochastic domains to avoid biasing the search towards lucky runs.

For theoretical background of GFS, see 'A Global Optimization Algorithm Worth Using' and Malherbe & Vayatis 2017: Global optimization of Lipschitz functions

Example usage

A basic example where we maximize Levi's function with as many parallel processes as there are logical cores, and save progress to file.

import math, distgfs

def levi(x, y):
    """
    Levi's function (see https://en.wikipedia.org/wiki/Test_functions_for_optimization).
    Has a global _minimum_ of 0 at x=1, y=1.
    """
    a = math.sin(3. * math.pi * x)**2
    b = (x - 1)**2 * (1 + math.sin(3. * math.pi * y)**2)
    c = (y - 1)**2 * (1 + math.sin(2. * math.pi * y)**2)
    return a + b + c


def obj_fun(pp, pid):
    """ Objective function to be _maximized_ by GFS. """
    x = pp['x']
    y = pp['y']

    res = levi(0.4*x, y)
    print(f"Iter: {pid}\t x:{x}, y:{y}, result:{res}")
    # Since Dlib maximizes, but we want to find the minimum,
    # we negate the result before passing it to the Dlib optimizer.
    return -res

# For this example, we pretend that we want to keep 'y' fixed at 1.0
# while optimizing 'x' in the range -4.5 to 4.5
space = {'x': [-4.5, 4.5]}
problem_parameters = {'y': 1.}
    
# Create an optimizer parameter set
distgfs_params = {'opt_id': 'distgfs_levi',
                  'obj_fun_name': 'obj_fun',
                  'obj_fun_module': 'example_distgfs_levi_file',
                  'problem_parameters': problem_parameters,
                  'space': space,
                  'n_iter': 10,
                  'file_path': 'distgfs.levi.h5',
                  'save': True,
                 }

distgfs.run(distgfs_params, verbose=True)

For additional examples, see examples.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

distgfs-1.1.0.tar.gz (24.6 kB view hashes)

Uploaded Source

Built Distribution

distgfs-1.1.0-py3-none-any.whl (23.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page