mlens.parallel.manager module

ML-Ensemble

author:Sebastian Flennerhag copyright:2017 licence:MIT

Parallel processing job managers.

class mlens.parallel.manager.BaseProcessor(caller)

Bases: object

Parallel processing base class.

Base class for parallel processing engines.

caller

initialize(job, X, y=None, dir=None)

Create a job instance for estimation.

job

terminate()

Remove temporary folder and all cache data.

class mlens.parallel.manager.Job(job)

Bases: object

Container class for holding job data.

See also

ParallelProcessing, ParallelEvaluation

dir

job

predict_in

predict_out

tmp

update()

Shift output array to input array.

y

class mlens.parallel.manager.ParallelEvaluation(caller)

Bases: mlens.parallel.manager.BaseProcessor

Parallel cross-validation engine.

Parameters:caller (Evaluator) – The Evaluator that instantiated the processor.

process(attr)

Fit all layers in the attached LayerContainer.

class mlens.parallel.manager.ParallelProcessing(caller)

Bases: mlens.parallel.manager.BaseProcessor

Parallel processing engine.

Engine for running ensemble estimation.

Parameters:layers (mlens.ensemble.base.LayerContainer) – The LayerContainer that instantiated the processor.

get_preds(dtype=None, order='C')

Return prediction matrix.

Parameters:

• dtype (numpy dtype object, optional) – data type to return
• order (str (default = 'C')) – data order. See numpy.asarray for details.

process()

Fit all layers in the attached LayerContainer.

mlens.parallel.manager.dump_array(array, name, dir)

Dump array for memmapping.