Examples

Overview

Tinkering with small pieces of working code is a fun way to learn. Below you will find several examples that you can play with in this way.

If you see a snippet of pipeline code elsewhere in this documentation, there might be an "Examples" section at the bottom of the page. If so, it will link you to a git repo which contains a pipeline that uses the snippet.

This page traces those links backwards. Here we list each example and links them back to the relevant documentation.

Example Pipelines in Git.

Most of the examples referenced by the docs are in a single git repo: github.com/conducto/examples. It's also worth checking out the demo repo, which exercises much of the Conducto API and documents it right there in the code (and also in the resulting pipeline).

Here's a convenient way to use these examples:

  1. Find one that resembles what you want to do
  2. Copy it into your project
  3. Make sure it works as expected
  4. Make minor changes until it does what you want

If you run in to questions along the way, feel free to ask.

The Examples

Compression Race

pipeline.py

Referenced by Controlling a Pipeline, this pipeline explores Serial and Parallel nodes as well as the cpu node parameter. It's a race, and varying the node parameters will change who wins.

scale.py

Referenced by Local vs Cloud as an example of using cloud mode to scale beyond what is possible locally, this pipeline is like the one above, except it takes a parameter and runs a race with that many nodes.

Container Reuse

Referenced by Container Reuse, this pipeline explores how Conducto retains or clears container state depending on how the pipeline definition author lays out container reuse contexts.

Counter

This pipeline deploys two services to your local docker daemon and tests them. It leaves the last node skipped so you can visit the deployed "site" in your browser. Unskipping them will clean up the deployment.

Disambiguate

Referenced by Native Functions, this pipeline shows how to pass node parameters to an Exec node which calls a native function. Doing so requires special syntax because otherwise it's not clear which arguments belong to the function, and which Conducto should interpret.

Eratosthenes

Referenced by both Native Functions and Debugging, pipeline.py and pipeline_onefile.py contrast using a command to run python code with using a native function to do so.

Fortune (messy)

This is the first part of an (unfinished) series. It uses local system utilities to generate a fortune for every day of the month, and then injects those fortunes into a pipeline. If you don't have the right utilities, it will break.

Fortune (clean), when it exists, will achieve the same task--but it will do so entirely within the pipeline containers. Contrasting these two will help explain how to keep your pipelines portable.

Hello Dockerfile

Referenced by Images, this pipeline demonstrates the syntax for using a Dockerfile to define an image for use with Conducto.

Hello Git

Referenced by Images, this pipeline demonstrates the syntax for including the contents of a git repo in an image for use with Conducto.

Hello Image

Referenced by Images, this pipeline demonstrates the syntax for including using a named image from DockerHub with Conducto.

Hello Native

Referenced by Images, this pipeline demonstrates the syntax for adding python packages to an image so that you can reference them in a native function.

Hello Py

Referenced by Images, this pipeline demonstrates the syntax for adding local files to an image so that you can reference them in a command.

Hello Py Js

This pipeline is like "Hello World" except that the commands run scripts instead of executing code directly.

Hello World

Referenced by Hello World, which encourages the user to run this pipeline, notice an error, fix it, and rerun the pipeline.

Image Version

Referenced by Access and Connectivity, this pipeline shows how to reference a specific a docker image instead of using a tag.

Stop on Error

Referenced by Pipeline Structure, this pipeline shows how you can influence the way that serial nodes handle failed children via the stop_on_error node parameter.

Tree Syntax

Referenced by Pipeline Structure, this pipeline shows how node parameters inheritance and overriding works in the pipeline tree.

Conclusion

Those are the examples so far. In the future, there will be more.

If you write a pipeline definition that might be useful as a reference, let us know and we'll consider adding it to this list.

Chat with us for a live demo right now!
(If we're awake 😴)

avatar