Developing with Relay

Developing steps with the Relay CLI

Relay has an internal API called the metadata service, which presents a REST endpoint to step containers. It serves up dynamic information from workflow steps' spec sections and receives outputs from steps as well as events from trigger containers.

In order to shorten the development loop for writing, testing, and debugging step code, the relay CLI tool has a built-in version of the metadata service that you can run locally. This will allow your step entrypoint code to run unmodified from the shell, rather than requiring a container build, push, and execute cycle up to the Relay service.

The metadata service is available under the relay dev metadata subcommand. It reads in a YAML file containing mock data that you supply, then starts an HTTP server. The server responds to the API endpoints that the Relay client SDKs expect to query on the service, supplying your mock values. This enables you to rapidly test and debug how your code will work when presented with user input on the service.

Because Relay steps don't normally exist in isolation, the YAML input is set up to mock an entire workflow's configuration and includes provisions for secrets, connections, and outputs. As such, the lexical structure of the YAML looks like this:

# filename: test-metadata.yaml
# map of connections, named '<type>/<name>'
    secretAccessKey: test

# single string valued secrets
  test: test

# now the data to be fed into pseudo-run 1 of this workflow
  '1':      # quoted string, not numeric! thanks YAML
        # the `spec` keys match what you'd see in a real workflow
          aws: ${}
          message: Test specification message
          secret: ${secrets.test}
        # outputs can be strings or structured data
          - 125252025
          - 252598675
          outputMessage: "Test output message"

Additional steps may then reference ${outputs.'first-step'.instanceIDs} in their spec section to retrieve the array of IDs.

After constructing the YAML input, start up the metadata service with:

relay dev metadata --input test-metadata.yaml --run 1 --step first-step

The last line of output contains an environment variable that you'll need to copy-and-paste into the shell where you're debugging you entrypoint:

No command was supplied, awaiting requests. Set environment with:
export METADATA_API_URL='http://:VeRyLoNgJWT[::]:59025'

This is the environment variable that the Python and Go SDKs as well as the ni shell tool use to communicate with the metadata service in Relay; setting it locally will point your code at the mock service. Running your code should result in HTTP GETs against the /spec path for each parameter and HTTP PUTs against the /output path if the step sets output variables.

Alternately, you can append the command line you'd run to test your code to the invocation of relay dev metadata and the environment variable will be set for you. This might be preferable if you are making changes to the YAML input file instead of, or concurrently with, changing the entrypoint code.

relay dev metadata --input test-metadata.yaml --run 1 --step first-step python3.8 ./