mlens.parallel package

Module contents

ML-ENSEMBLE

author:Sebastian Flennerhag
copyright:2017
licence:MIT
class mlens.parallel.ParallelProcessing(caller)[source]

Bases: mlens.parallel.manager.BaseProcessor

Parallel processing engine.

Engine for running ensemble estimation.

Parameters:layers (mlens.ensemble.base.LayerContainer) – The LayerContainer that instantiated the processor.
get_preds(dtype=None, order='C')[source]

Return prediction matrix.

Parameters:
  • dtype (numpy dtype object, optional) – data type to return
  • order (str (default = 'C')) – data order. See numpy.asarray for details.
process()[source]

Fit all layers in the attached LayerContainer.

class mlens.parallel.ParallelEvaluation(caller)[source]

Bases: mlens.parallel.manager.BaseProcessor

Parallel cross-validation engine.

Parameters:caller (Evaluator) – The Evaluator that instantiated the processor.
process(attr)[source]

Fit all layers in the attached LayerContainer.

class mlens.parallel.Stacker(job, layer)[source]

Bases: mlens.parallel.estimation.BaseEstimator

Stacked fit sub-process class.

Class for fitting a Layer using Stacking.

run(parallel)[source]

Execute stacking.

class mlens.parallel.Blender(job, layer)[source]

Bases: mlens.parallel.estimation.BaseEstimator

Blended fit sub-process class.

Class for fitting a Layer using Blending.

run(parallel)[source]

Execute stacking.

class mlens.parallel.SubStacker(job, layer)[source]

Bases: mlens.parallel.estimation.BaseEstimator

Stacked subset fit sub-process class.

Class for fitting a Layer using Subsemble.

run(parallel)[source]

Execute subsembling

class mlens.parallel.SingleRun(job, layer)[source]

Bases: mlens.parallel.estimation.BaseEstimator

Single run fit sub-process class.

Class for fitting a estimators in a layer without any sub-fits.

run(parallel)[source]

Execute blending.

class mlens.parallel.Evaluation(evaluator)[source]

Bases: object

Evaluation engine.

Run a job for an Evaluator instance.

Parameters:evaluator (Evaluator) – Evaluator instance to run job for.
evaluate(parallel, X, y, dir)[source]

cross-validation of estimators.

Parameters:
  • parallel (joblib.Parallel) – The instance to use for parallel fitting.
  • X (array-like of shape [n_samples, n_features]) – Training set to use for estimation. Can be memmaped.
  • y (array-like of shape [n_samples, ]) – labels for estimation. Can be memmaped.
  • dir (str) – directory of cache to dump fitted transformers before assembly.
preprocess(parallel, X, y, dir)[source]

Fit preprocessing pipelines.

Fit all preprocessing pipelines in parallel and store as a preprocessing_ attribute on the Evaluator.

Parameters:
  • parallel (joblib.Parallel) – The instance to use for parallel fitting.
  • X (array-like of shape [n_samples, n_features]) – Training set to use for estimation. Can be memmaped.
  • y (array-like of shape [n_samples, ]) – labels for estimation. Can be memmaped.
  • dir (directory of cache to dump fitted transformers before assembly.) –
class mlens.parallel.BaseEstimator(layer)[source]

Bases: object

Base class for estimating a layer in parallel.

Estimation class to be used as based for a layer estimation engined that is callable by the ParallelProcess job manager.

A subclass must implement a constructor that accepts the following args:
  • job : the Job instance containing relevant data
  • layer: the Layer instance to estimate
  • n: the position in the LayerContainer stack of the layer

as well as a run method that accepts a Parallel instance.

Parameters:layer (Layer) – layer to be estimated.
run(parallel)[source]

Method for executing estimation.

Default method relies on the default constructor. Both can be replaced if desired.

Parameters:parallel (object) – Parallel instance.