2.13. Using Containers

2.13.1. Running Tools Inside Docker

Docker containers simplify software installation by providing a complete known-good runtime for software and its dependencies. However, containers are also purposefully isolated from the host system, so in order to run a tool inside a Docker container there is additional work to ensure that input files are available inside the container and output files can be recovered from the container. A CWL runner can perform this work automatically, allowing you to use Docker to simplify your software management while avoiding the complexity of invoking and managing Docker containers.

One of the responsibilities of the CWL runner is to adjust the paths of input files to reflect the location where they appear inside the container.

This example runs a simple Node.js script inside a Docker container which will then print “Hello World” to the standard output.

docker.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: CommandLineTool
baseCommand: node
hints:
  DockerRequirement:
    dockerPull: node:slim
inputs:
  src:
    type: File
    inputBinding:
      position: 1
outputs:
  example_out:
    type: stdout
stdout: output.txt

docker-job.yml

src:
  class: File
  path: hello.js

Before we run this, let’s just break it down and see what some bits do. Most of this has been explained in previous sections, the only part that is really new is the dockerRequirement section.

baseCommand: node
hints:
  DockerRequirement:
    dockerPull: node:slim

baseCommand: node tells CWL that we will be running this command using the Node Js runtime that is meant for Javascript files. We then need to specify some hints for how to find the container we want. In this case we list just our requirements for the docker container in DockerRequirements. The dockerPull: parameter takes the same value that you would pass to a docker pull command. That is, the name of the container image (you can even specify the tag, which is good idea for best practices when using containers for reproducible research). In this case we have used a container called node:slim.

Create a Javascript file named “hello.js” and invoke cwltool providing the tool description and the input object on the command line:

hello.js

console.log("Hello World");

$ cwltool docker.cwl docker-job.yml
INFO /opt/hostedtoolcache/Python/3.9.19/x64/bin/cwltool 3.1.20240508115724
INFO Resolved 'docker.cwl' to 'file:///home/runner/work/user_guide/user_guide/src/_includes/cwl/using-containers/docker.cwl'
INFO [job docker.cwl] /tmp/dusvdol8$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/dusvdol8,target=/vzSlrf \
    --mount=type=bind,source=/tmp/qhzbp__6,target=/tmp \
    --mount=type=bind,source=/home/runner/work/user_guide/user_guide/src/_includes/cwl/using-containers/hello.js,target=/var/lib/cwl/stg85a27c72-1bb8-46e1-bd7d-254b6b75587c/hello.js,readonly \
    --workdir=/vzSlrf \
    --read-only=true \
    --net=none \
    --log-driver=none \
    --user=1001:127 \
    --rm \
    --cidfile=/tmp/ilevkccw/20240518114125-414607.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/vzSlrf \
    node:slim \
    node \
    /var/lib/cwl/stg85a27c72-1bb8-46e1-bd7d-254b6b75587c/hello.js > /tmp/dusvdol8/output.txt
INFO [job docker.cwl] completed success
{
    "example_out": {
        "location": "file:///home/runner/work/user_guide/user_guide/src/_includes/cwl/using-containers/output.txt",
        "basename": "output.txt",
        "class": "File",
        "checksum": "sha1$648a6a6ffffdaa0badb23b8baf90b6168dd16b3a",
        "size": 12,
        "path": "/home/runner/work/user_guide/user_guide/src/_includes/cwl/using-containers/output.txt"
    }
}INFO Final process status is success

$ cat output.txt
Hello World

Notice the CWL runner has constructed a Docker command line to run the script.

In this example, the path to the script hello.js is /home/me/cwl/user_guide/hello.js outside the container but /var/lib/cwl/job369354770_examples/hello.js inside the container, as reflected in the invocation of the node command.