Callbacks

The following actions can be registered with a node event. When that event fires, Conducto will complete the action.

Pipeline Control

conducto.callback.retry(max_num_retries, *, inherited=False, user_display=True)

If the node doesn’t succeed, retry it max_num_retries times.

Parameters

max_num_retries – Max iterations.

Returns

a callback object to associate with a node event

conducto.callback.retry_then_skip(max_num_retries, *, inherited=False, user_display=True)

If the node doesn’t succeed, retry it max_num_retries number of times. If it still hasn’t succeeded, skip it.

Parameters

max_num_retries – Max iterations.

Returns

a callback object to associate with a node event

conducto.callback.retry_with_double_mem(max_num_retries, *, inherited=False, user_display=True)

Retry the given node max_num_retries times, doubling the memory estimate each time. The memory request will be capped at the limits permitted by the underlying service.

Parameters

max_num_retries – Max iterations.

Returns

a callback object to associate with a node event

conducto.callback.handle_memory_errors(mem_limits=(8, 32), max_duration=3600, *, inherited=False, user_display=True)
Logic:
  • If not a memoryError, abort.

  • Don’t retry if memory would be above upper bound of mem_limit.

  • Don’t retry if duration has already exceeded max_duration.

  • Retry if memory is below lower bound of mem_limit.

  • Retry if memory is above lower bound of mem_limit and we haven’t retried it yet (try at least once).

conducto.callback.skip_some_errors(max_num_errors, *, inherited=False, user_display=True)

If the node fails with <= max_num_errors descendants that have errored, skip those nodes and reset them, thereby ignoring their errors.

Parameters

max_num_errors – Max errors to skip + reset.

Returns

a callback object to associate with a node event

conducto.callback.email(to=None, cc=None, *, inherited=False, user_display=True)

When the node finishes, send a summary email to the specified recipients.

Parameters
  • to – Recipients in the “To” field

  • cc – Recipients in the “CC” field

Returns

a callback object to associate with a node event

Slack

conducto.callback.slack_status(recipient, message=None, node_summary=False, *, inherited=False, user_display=True)

Post a summary of the node in the specified slack channel via the Slack integration.

Parameters
  • recipient – where to send the updates to, channel id or user id

  • message – message to print, optional

  • node_summary – print node summary, defaults to True if no message, False otherwise

Returns

a callback object to associate with a node event