# draugr.multiprocessing_utilities.pooled_queue_processor.PooledQueueProcessor

class draugr.multiprocessing_utilities.pooled_queue_processor.PooledQueueProcessor(func, args: Iterable = (), kwargs=None, max_queue_size=100, n_proc=None, max_tasks_per_child=None, fill_at_construction=True, blocking=True)

  Bases: `object`

  This is a workaround of Pythons extremely slow interprocess communication pipes. The ideal solution would be to use a multiprocessing.queue, but it apparently communication is band limited. This solution has processes complete tasks (batches) and a thread add the results to a queue.queue.

  __init__(func, args: Iterable = (), kwargs=None, max_queue_size=100, n_proc=None, max_tasks_per_child=None, fill_at_construction=True, blocking=True)

  Methods

  `__init__`(func[, args, kwargs, ...]) |
  `close`() | close pool
  `fill`() | fill queue

  `get`()

  return

  `maybe_fill`() | fill queue if not full

  `put`(res)

  param res

  `raise_error`(excptn)

  param excptn

  `terminate`() | terminate pool

  Attributes

  `queue_size` | return: :rtype:

  close() → None

    close pool

  fill() → None

    fill queue

  get() → Any
    Returns

  maybe_fill() → None

    fill queue if not full

  put(res) → None
    Parameters

      res –

  property queue_size: int

    return: :rtype:

  raise_error(excptn) → None
    Parameters

      excptn –

  terminate() → None

    terminate pool