mlens.parallel package

Submodules

• mlens.parallel.blend module
• mlens.parallel.estimation module
• mlens.parallel.evaluation module
• mlens.parallel.manager module
• mlens.parallel.single_run module
• mlens.parallel.stack module
• mlens.parallel.subset module

Module contents

ML-ENSEMBLE

author:Sebastian Flennerhag copyright:2017 licence:MIT

class mlens.parallel.ParallelProcessing(caller)

Bases: mlens.parallel.manager.BaseProcessor

Parallel processing engine.

Engine for running ensemble estimation.

Parameters:layers (mlens.ensemble.base.LayerContainer) – The LayerContainer that instantiated the processor.

get_preds(dtype=None, order='C')

Return prediction matrix.

Parameters:

• dtype (numpy dtype object, optional) – data type to return
• order (str (default = 'C')) – data order. See numpy.asarray for details.

process()

Fit all layers in the attached LayerContainer.

class mlens.parallel.ParallelEvaluation(caller)

Bases: mlens.parallel.manager.BaseProcessor

Parallel cross-validation engine.

Parameters:caller (Evaluator) – The Evaluator that instantiated the processor.

process(attr)

Fit all layers in the attached LayerContainer.

class mlens.parallel.Stacker(job, layer)

Bases: mlens.parallel.estimation.BaseEstimator

Stacked fit sub-process class.

Class for fitting a Layer using Stacking.

run(parallel)

Execute stacking.

class mlens.parallel.Blender(job, layer)

Bases: mlens.parallel.estimation.BaseEstimator

Blended fit sub-process class.

Class for fitting a Layer using Blending.

run(parallel)

Execute stacking.

class mlens.parallel.SubStacker(job, layer)

Bases: mlens.parallel.estimation.BaseEstimator

Stacked subset fit sub-process class.

Class for fitting a Layer using Subsemble.

run(parallel)

Execute subsembling

class mlens.parallel.SingleRun(job, layer)

Bases: mlens.parallel.estimation.BaseEstimator

Single run fit sub-process class.

Class for fitting a estimators in a layer without any sub-fits.

run(parallel)

Execute blending.

class mlens.parallel.Evaluation(evaluator)

Bases: object

Evaluation engine.

Run a job for an Evaluator instance.

Parameters:evaluator (Evaluator) – Evaluator instance to run job for.

evaluate(parallel, X, y, dir)

cross-validation of estimators.

Parameters:

• parallel (joblib.Parallel) – The instance to use for parallel fitting.
• X (array-like of shape [n_samples, n_features]) – Training set to use for estimation. Can be memmaped.
• y (array-like of shape [n_samples, ]) – labels for estimation. Can be memmaped.
• dir (str) – directory of cache to dump fitted transformers before assembly.

preprocess(parallel, X, y, dir)

Fit preprocessing pipelines.

Fit all preprocessing pipelines in parallel and store as a preprocessing_ attribute on the Evaluator.

Parameters:

• parallel (joblib.Parallel) – The instance to use for parallel fitting.
• X (array-like of shape [n_samples, n_features]) – Training set to use for estimation. Can be memmaped.
• y (array-like of shape [n_samples, ]) – labels for estimation. Can be memmaped.
• dir (directory of cache to dump fitted transformers before assembly.) –

class mlens.parallel.BaseEstimator(layer)

Bases: object

Base class for estimating a layer in parallel.

Estimation class to be used as based for a layer estimation engined that is callable by the ParallelProcess job manager.

A subclass must implement a constructor that accepts the following args:

• job : the Job instance containing relevant data
• layer: the Layer instance to estimate
• n: the position in the LayerContainer stack of the layer

as well as a run method that accepts a Parallel instance.

Parameters:layer (Layer) – layer to be estimated.

run(parallel)

Method for executing estimation.

Default method relies on the default constructor. Both can be replaced if desired.

Parameters:parallel (object) – Parallel instance.