There are many ways to get data into a pipeline:

  1. as files in a node's image
  2. as files in /conducto/data
  3. as part of the container, via environment variables
  4. as part of the node's command string
  5. via a runtime network connection

The third way is most appropriate for secrets, hostnames, and other context-sensitive data. This article will show you how.

The env Node Parameter

Conducto provides explicit control of node environment variables with the env node parameter. It takes a dictionary where the keys are variable names, and the values are their contents.

node = co.Exec("echo hello $NAME", env={"NAME" : "James"})

The node above will run its command in a shell where the variable called NAME stores the value James. Stdout will look like this:

hello James

Like most node parameters, you can also assign it at the parent level and expect it to affect a node's children.

crew = { "CAPTAIN"  : "James",
         "XO"       : "Naomi",
         "MECHANIC" : "Amos" }

greet = co.Serial(env=crew)
greet["leaders"] = co.Exec("echo $CAPTAIN and $XO are in charge")
greet["grunts"] = co.Exec("echo but we're going nowhere without $MECHANIC")

Conducto Secrets

Your conducto user Profile contains two Secrets sections. One for your User and one for your Org.

Conducto Profile Secrets

This is similar to the env node parameter: the keys refer to environment variable names. The User secrets are visible only to you (and your pipelines). Similarly, the Org secrets are shared across your whole Conducto Org (and their pipelines).

If you enable masking, Conducto will hide the secret both in the UI, and whenever it appears in your node's stdout and stderr.

Summary

Conducto sets environment variables when it runs your nodes. You can define these explicitly in the pipeline definition, or you can set them in your Conducto profile and they'll be automatically made available to your nodes.