Parallel
Tweak of joblib.Parallel (opens in a new tab) that propagates the scikit-learn configuration.
This subclass of joblib.Parallel (opens in a new tab) ensures that the active configuration (thread-local) of scikit-learn is propagated to the parallel workers for the duration of the execution of the parallel tasks.
The API does not change and you can refer to joblib.Parallel (opens in a new tab) documentation for more details.
Python Reference (opens in a new tab)
Constructors
constructor()
Signature
new Parallel(opts?: object): Parallel;Parameters
| Name | Type |
|---|---|
opts? | object |
Returns
Defined in: generated/utils/parallel/Parallel.ts:25 (opens in a new tab)
Methods
dispatch_next()
Dispatch more data for parallel processing
This method is meant to be called concurrently by the multiprocessing callback. We rely on the thread-safety of dispatch_one_batch to protect against concurrent consumption of the unprotected iterator.
Signature
dispatch_next(opts: object): Promise<any>;Parameters
| Name | Type |
|---|---|
opts | object |
Returns
Promise<any>
Defined in: generated/utils/parallel/Parallel.ts:99 (opens in a new tab)
dispatch_one_batch()
Prefetch the tasks for the next batch and dispatch them.
The effective size of the batch is computed here. If there are no more jobs to dispatch, return false, else return true.
The iterator consumption and dispatching is protected by the same lock so calling this function should be thread safe.
Signature
dispatch_one_batch(opts: object): Promise<any>;Parameters
| Name | Type |
|---|---|
opts | object |
Returns
Promise<any>
Defined in: generated/utils/parallel/Parallel.ts:129 (opens in a new tab)
dispose()
Disposes of the underlying Python resources.
Once dispose() is called, the instance is no longer usable.
Signature
dispose(): Promise<void>;Returns
Promise<void>
Defined in: generated/utils/parallel/Parallel.ts:80 (opens in a new tab)
format()
Return the formatted representation of the object.
Signature
format(opts: object): Promise<any>;Parameters
| Name | Type |
|---|---|
opts | object |
Returns
Promise<any>
Defined in: generated/utils/parallel/Parallel.ts:155 (opens in a new tab)
init()
Initializes the underlying Python resources.
This instance is not usable until the Promise returned by init() resolves.
Signature
init(py: PythonBridge): Promise<void>;Parameters
| Name | Type |
|---|---|
py | PythonBridge |
Returns
Promise<void>
Defined in: generated/utils/parallel/Parallel.ts:43 (opens in a new tab)
print_progress()
Display the process of the parallel execution only a fraction of time, controlled by self.verbose.
Signature
print_progress(opts: object): Promise<any>;Parameters
| Name | Type |
|---|---|
opts | object |
Returns
Promise<any>
Defined in: generated/utils/parallel/Parallel.ts:181 (opens in a new tab)
Properties
_isDisposed
boolean=false
Defined in: generated/utils/parallel/Parallel.ts:23 (opens in a new tab)
_isInitialized
boolean=false
Defined in: generated/utils/parallel/Parallel.ts:22 (opens in a new tab)
_py
PythonBridge
Defined in: generated/utils/parallel/Parallel.ts:21 (opens in a new tab)
id
string
Defined in: generated/utils/parallel/Parallel.ts:18 (opens in a new tab)
opts
any
Defined in: generated/utils/parallel/Parallel.ts:19 (opens in a new tab)
Accessors
py
Signature
py(): PythonBridge;Returns
PythonBridge
Defined in: generated/utils/parallel/Parallel.ts:30 (opens in a new tab)
Signature
py(pythonBridge: PythonBridge): void;Parameters
| Name | Type |
|---|---|
pythonBridge | PythonBridge |
Returns
void
Defined in: generated/utils/parallel/Parallel.ts:34 (opens in a new tab)