Developing for Relay

Developing steps with the Relay CLI

Relay has an internal API called the metadata service, which presents a REST endpoint to step containers. It serves up dynamic information from workflow steps' spec sections and receives outputs from steps as well as events from trigger containers.

In order to shorten the development loop for writing, testing, and debugging step code, the relay CLI tool has a built-in version of the metadata service that you can run locally. This will allow your step entrypoint code to run unmodified from the shell, rather than requiring a container build, push, and execute cycle up to the Relay service.

The metadata service is available under the relay dev metadata subcommand. It reads in a YAML file containing mock data that you supply, then starts an HTTP server. The server responds to the API endpoints that the Relay client SDKs expect to query on the service, supplying your mock values. This enables you to rapidly test and debug how your code will work when presented with user input on the service.

Because Relay steps don't normally exist in isolation, the YAML input is set up to mock an entire workflow's configuration and includes provisions for secrets, connections, and outputs. As such, the lexical structure of the YAML looks like this:

# filename: test-metadata.yaml
# map of connections, named '<type>/<name>'
connections:
  aws/my-aws-connection:
    accessKeyID: AKIASAMPLEKEY
   secretAccessKey: ...

# single string valued secrets
secrets:
  mySecretKey: s00pers33kr1t

# now the data to be fed into pseudo-run 1 of this workflow
runs:
  '1':      # quoted string, not numeric! thanks YAML
    steps:
      first-step:
        # the `spec` keys match what you'd see in a real workflow
        spec:
          aws: !Connection [aws, test]
          message: My test message
          secret: !Secret mySecretKey
        # outputs can be strings or structured data
        outputs:
          instanceIDs:
          - 125252025
          - 252598675
          outputMessage: "This came from the step!"

Additional steps may then reference !Output [first-step, instanceIDs] in their spec section to retrieve the array of IDs.

After constructing the YAML input, start up the metadata service with:

relay dev metadata --input test-metadata.yaml --run 1 --step first-step

The last line of output contains an environment variable that you'll need to copy-and-paste into the shell where you're debugging you entrypoint:

No command was supplied, awaiting requests. Set environment with:
export METADATA_API_URL='http://:VeRyLoNgJWT[::]:59025'

This is the environment variable that the Python and Go SDKs as well as the ni shell tool use to communicate with the metadata service in Relay; setting it locally will point your code at the mock service. Running your code should result in HTTP GETs against the /spec path for each parameter and HTTP PUTs against the /output path if the step sets output variables.

Alternately, you can append the command line you'd run to test your code to the invocation of relay dev metadata and the environment variable will be set for you. This might be preferable if you are making changes to the YAML input file instead of, or concurrently with, changing the entrypoint code.

relay dev metadata --input test-metadata.yaml --run 1 --step first-step python3.8 ./my-entrypoint.py