2.14. Creating Files at Runtime

Sometimes you need to create a file on the fly from input parameters, such as tools that expect to read their input configuration from a file rather than the command line parameters, or need a small wrapper shell script.

To generate such files, we can use the InitialWorkDirRequirement.

createfile.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: CommandLineTool
baseCommand: ["sh", "example.sh"]

requirements:
  InitialWorkDirRequirement:
    listing:
      - entryname: example.sh
        entry: |-
          PREFIX='Message is:'
          MSG="\${PREFIX} $(inputs.message)"
          echo \${MSG}

inputs:
  message: string
outputs:
  example_out:
    type: stdout
stdout: output.txt

Any expressions like $(inputs.message) are expanded by the CWL engine before creating the file. Here, insert the value at the input message.

Tip

The CWL expressions are independent of any shell variables used later during command line tool invocation. That means that any genuine need for the character $ must be escaped with \. For instance, \${PREFIX} above is expanded to ${PREFIX} in the generated file to be evaluated by the shell script instead of the CWL engine.

To test the above CWL tool, use this job to provide the input value message:

echo-job.yml

message: Hello world!

Before we run this, let us look at each step in a little more detail. The base command baseCommand: ["sh", "example.sh"] will execute the command sh example.sh. This will run the file we create in the shell.

InitialWorkDirRequirement requires a listing. As the listing is a YAML array, we need a - on the first line of each element of the array, in this case we have just one element. entryname: can have any value, but it must match what was specified in the baseCommand. The final part is entry:, this is followed by |- which is YAML quoting syntax, and means that you are using a multiline string (without it, we would need to write the whole script on one line).

Note

See the YAML Guide for more about the formatting.

Now invoke cwltool with the tool description and the input object on the command line:

$ cwltool createfile.cwl echo-job.yml
INFO /opt/hostedtoolcache/Python/3.9.19/x64/bin/cwltool 3.1.20240508115724
INFO Resolved 'createfile.cwl' to 'file:///home/runner/work/user_guide/user_guide/src/_includes/cwl/creating-files-at-runtime/createfile.cwl'
INFO [job createfile.cwl] /tmp/li93ouor$ sh \
    example.sh > /tmp/li93ouor/output.txt
INFO [job createfile.cwl] completed success
{
    "example_out": {
        "location": "file:///home/runner/work/user_guide/user_guide/src/_includes/cwl/creating-files-at-runtime/output.txt",
        "basename": "output.txt",
        "class": "File",
        "checksum": "sha1$9045abe4bd04dd8ccfe50c6ff61820b784b64aa7",
        "size": 25,
        "path": "/home/runner/work/user_guide/user_guide/src/_includes/cwl/creating-files-at-runtime/output.txt"
    }
}INFO Final process status is success

$ cat output.txt
Message is: Hello world!