Security and Connectivity

Overview

When you're working on a team, it's nice to be on the same page. That's why each Conducto pipeline gets a page that is visible to the whole Conducto org. If you want to know about its status, that's where you look.

You can also control the pipeline from that page. If it's a cloud pipeline, then the whole org gets the same level of control.

For pipelines in local mode, certain types of control are only available to whoever launched the pipeline. These restrictions let that person control which code runs on their machine.

This article show how Conducto uses local resources. Specifically, we're looking for cases where malicious pipeline code could cause local trouble.

Understanding the potential attack surface will help you decided if a pipeline is worth running on locally. Finally, we'll talk about starting an agent, since that's what connects your machine to the Conducto ecosystem.

Pipeline Owners

If you launch a pipeline, then you're its owner. Whether it's local or in the cloud, any filesystem-dependent Conducto features will reference your local filesystem. These are indicated by kwargs like (copy_dir). See User-Launched Cloud Pipelines for more about this.

Note that these features act before a node's command runs. For cloud pipelines, it doesn't matter who owns it after that point--the computation is happening elsewhere. But if you're considering launching a local pipeline, then you should be aware that Conducto pipelines are not security sandboxes. Running untrusted pipeline code can put the machine that hosts it at risk.

Restrictions on Non-Owners

Conducto will restrict some actions to the pipeline's owner if it's in local mode.

  • change the command for an exec node
  • wake a sleeping pipeline

These limitations are intended to prevent users from running malicious code on someone else's machine via an already-launched local pipeline. They aren't a perfect solution: Some vigilance is necessary on the part of the pipeline owner.

Potentially Malicious Pipelines

CI/CD pipelines commonly pull updated code as part of their operation. This means that a pipeline definition could be harmless as written, but its pipeline might later pull in dangerous code after the pipeline was initially launched.

You can configure Conducto to download new code while rebuilding a node's image, which is not restricted to pipeline owners. It can come in either as source code (via git or somesuch), or as part of a docker image.

Code Changes

If your pipeline depends on updated code from a repository, then we assume that you trust whoever is allowed to push code to that repo. The Image kwargs copy_url and copy_repo indicate that a pipeline definition will pull new code when images are rebuilt. Fresh code might also come in while the pipeline is running, so you'll also want to keep an eye out for commands like git clone.

Pipelines that pull in code and execute it are not necessarily problematic, but you should ask yourself if you trust the source of that code before you run them.

Image Changes

The same reasoning applies to any docker images that you reference by name. If the maintainers update the image, then Conducto will pull the update whether you've decided to trust it or not.

Usually these updates are helpful, but if you're concerned about this you can pin your images to a version. So instead of using a name and a tag.

image="alpine:latest"

You can use a name and a hash.

image="alpine@sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321"

This way Docker will refuse to pull any image other than the one indicated by the hash. To get the hash for an image, first pull it, then inspect it for the "Repo Digests" field.

$ docker pull alpine:latest
$ docker inspect alpine:latest --format '{{json .RepoDigests}}'

    ["alpine@sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321"]

Local Access

This section will explain the parts of the Conducto ecosystem that run on your machine. Each of them runs in a docker container.

In brief: Workers run your pipeline's commands. Each pipeline gets a manager which creates workers. Agents handle any remote action that a manager isn't available for.

The subsections below will explain each of these in more detail. Specifically we'll be focused on which local resources they have access to, and whether that access represents a security concern.

Generally speaking, you can run local pipelines without worry if you and your team follows these recommendations:

  1. Be mindful of how you're using a teammate's machine
  2. Don't work with malicious teammates
  3. Don't launch untrustworthy pipelines

Workers

When somebody runs an Exec node via the Conducto web app, the manager for that pipeline will create a worker container. That container runs the node's command until it completes.

Workers are the vehicle for running your pipeline's commands, so any trouble that a pipeline definition can cause will likely be caused by a worker.

Compute Resources

The primary resources consumed by Exec nodes are computational--controlled by the node parameters cpu and mem. For more about allocating computational resources to Exec nodes, see Pipeline Structure and Controlling a Pipeline.

If your everyday workflow significantly taxes your processor or memory, then local pipelines might slow things down when they run on your machine. If this happens, consider relaunching the pipelines elsewhere (either in cloud mode or in local mode on a different machine). Alternatively, talk to your team about recommendation #1 (above).

Network Access

Another resource that workers share with your local machine is network access. If you have network resources on your LAN that aren't usually accessible from the internet, those resources might be accessible from your pipeline's nodes.

Recommendation #3 should prevent any problems here, and if your pipeline pulls updates from elsewhere then you'll want to consider #2 as well.

Docker Socket

If your pipeline uses the requres_docker node parameter, then the Docker daemon's socket (typically /var/run/docker.sock) will be mounted in its containers. This is what lets you manipulate docker from inside your container.

Docker (usually) runs as root, so malicious code can be especially damaging if it is allowed access to the docker socket.

Mitigating these risks comes down to following recommendations #2 and #3, like in the Network Access subsection.

Keep in mind that some installations of Docker run the Docker daemon in a VM. In these cases, malicious code that targets the docker socket can only affect that VM's filesystem.

Managers

Each Conducto pipeline gets a manager, which is responsible for things like running that pipeline, resetting nodes, storing its runtime details, and putting it to sleep. For more about this comes together, see How Local Mode Works.

Managers have access to the Docker daemon's socket, which is how they create workers.

If you put a pipeline to sleep, the manager exits. This will stop the manager from being able to create workers. Be aware that an agent can wake a sleeping manager—so if you want to limit remote access you should stop the agent too.

Agents

Agents are responsible for events from conducto.com that need to be handled on a local machine which aren't already handled by a manager:

  • updating sandbox project files in ~/.conducto
  • starting managers for sandbox project pipelines
  • waking up sleeping managers
  • building images for cloud pipelines

If you run a command that requires an Agent, we'll start one for you. And if you try to do something in the browser that needs one, we'll provide you with a command to start one.

Since they start manager containers, they need access to the docker socket. They also need access to ~/.conducto so that they have access to the local machine's sandbox projects.

Start an Agent

If the Conducto web app doesn't have an agent to use, then it will provide a command to start one.

Copy this command to start an agent

You'll need to paste the command in a shell with access to Docker. The rest of this section will explain how it works.

The Agent Start Command

The goal of this command is to leave an agent running so that your machine can respond to events from the Conducto web app. Let's look at the parts of the command to see how it works.

docker run … conducto/start

The run verb tells Docker to create a new container. See the Docker docs for a more detailed description of run.

In this case, our new container will run a command that is specified by the image: conducto/start. You can see the command like this:

$ docker pull conducto/start
$ docker inspect conducto/start --format "{{.Config.Cmd}}"

    [/bin/sh -c conducto-profile start-agent]

The docker run command is just a way to avoid installing the conducto Python package. If you already have it installed and set up, then you can achieve the same result by running that command locally:

conducto-profile start-agent

However you run it, you will end up with an agent running in a container (unless you already have one). The remaining subsections expand on the arguments that docker needs to make this happen.

-it --rm -e CONDUCTO_…

-it connects a terminal to the container in interactive mode. This lets the container know that there's a human on the outside, which affects how it will communicate with us.

--rm tells docker to remove the container once it finishes running. This prevents you from having a pile of sopped docker containers that you would occasionally need to clean up.

The -e flag defines an environment variable inside the container. The meaning of each variable is in the table below.

Variable Name Description
CONDUCTO_TOKEN This token identifies you to conducto.com and ensures that your local machine is associated with the right user.
CONDUCTO_OS Your local operating system is stored as metadata on the agent, it is used for keeping track of multiple agents.
CONDUCTO_LOCAL_HOSTNAME Your machine's hostname is stored as metadata on the agent for the same reason.
CONDUCTO_OUTER_OWNER The agent creates local files on your behalf. It needs some details about your operating system user so that those files don't have permissions problems later on.

-v /var/run/docker.sock:/var/run/docker.sock

This makes your docker socket available from within the container, which is how a command inside a container can create a separate container on the outside.

You should be aware that this gives Conducto control over Docker, which commonly runs with elevated permissions. Conducto uses it create the containers described in the Local Access section.

It's possible to write pipeline definitions that pass this socket into your Worker Containers. See the Docker Socket for the security implications of running local pipelines that do so.

-v $HOME/.conducto:/root/.conducto

This makes ~/.conducto on your local machine show up at /root/.conducto in the container. If you have multiple Conducto accounts, files in this directory will help keep them straight.

This mount is also applied to the agent container, which it uses to manage your sandbox files.

Up and running

Now that your agent is started, you can use the Conducto web app to leverage local computing power from anywhere. If you use this machine remotely, don't forget to adjust its power settings to prevent it from going to sleep.

To see which agents are running, consult the computer icon in the upper right.

Manage Agents

You can get more detail by selecting "Manage Agents".

Stop an Agent

If you have the conducto Python package installed and set up, you can stop your local agent with the command:

conducto-profile stop-agent

Otherwise run docker ps and look for the container named "conducto_agent".

$ docker ps

    CONTAINER ID     NAMES
    5371b7167eb9     conducto_agent_hc5bb75a4_pc5294125

Then stop that container.

$ docker stop 537

(Here's a tip: You only need to type enough of the container ID to avoid ambiguity.)

Keep in mind that managers also relay events from conducto.com to your local machine, so if you want to prevent remote access you'll want to stop them too (or just sleep their corresponding pipelines).

Conclusion

We built Conducto to stay out of your way so that you can focus on the task at hand. A side effect is that using local pipelines can sometimes feel like magic.

You should be able to de-mystify any process that runs on your machine, so in this article we've laid all our cards on the table: Nothing up our sleeve—just a web app that controls trees made out of containers.

Malicious code is still malicious when it's in a pipeline, so we discussed the ways that pipeline code can cause local problems, and some strategies for preventing those problems. This discussion focused on worker containers, which is where your code runs. For a sense of context, we also explained managers and agents, which is where our code runs.

Starting an agent involves a somewhat complex command, so we finished by describing how it works so that you can use Conducto on your local machine with confidence.

Concepts

Examples

Chat with us for a live demo right now!
(If we're awake 😴)

avatar