What’s Going on With Tekton? (Part 2)
The Story So Far…
In Part 1 of this two-part blog, I talked about how we’re using Tekton Pipelines in building relay.sh, how Jenkins X is using it, and a brief history of the project and its terminology. In this part, I’ll dig into some of the cool advancements and features in the core Pipelines project and the broader ecosystem.
tkn CLI tool
There’s been a ton of work lately going into the
tkn command-line interface. I wanted to introduce it first, not just because there’ve been a number of very cool improvements in it recently, but also because we at Puppet are heavily invested in awesome command-line user experiences for our users, and
tkn exhibits some great UX patterns that are worth looking at, even if you’re not using it directly.
You can install
tkn via Homebrew or tarball; check out the README in the tektoncd/cli repo for the details. It uses the kubernetes client API, so if
kubectl get pods --namespace=tekton-pipelines works for you, so will
tkn. The first cool thing about the CLI that it supports both bash and zsh command completion, so typing
tkn <Tab> will show you the list of subcommands it supports:
> tkn <Tab> -- command -- clustertask -- Manage clustertasks completion -- Prints shell completion scripts condition -- Manage conditions eventlistener -- Manage eventlisteners help -- Help about any command pipeline -- Manage pipelines pipelinerun -- Manage pipelineruns resource -- Manage pipeline resources task -- Manage tasks taskrun -- Manage taskruns triggerbinding -- Manage triggerbindings triggertemplate -- Manage triggertemplates version -- Prints version information
Many of these subcommands map directly back to the terminology and underlying CRDs I mentioned in the opening. It’s important to note that, by design, the
tkn CLI is primarily a read-only interface, not a read-write one, so creation and editing of the objects still needs to go through
kubectl. That’s because the creation of Tekton objects boils down to feeding YAML descriptions into the Kubernetes cluster API; modulo introducing a higher-order language like Relay or Jenkins X, there’s not much
tkn can add over raw
kubectl apply -f mypipeline.yaml. There are, however, several commands for controlling existing objects that are substantially nicer to use than their
kubectl equivalents. For example,
tkn pipeline start will kick off a new run of an existing pipeline, without you needing to craft a new YAML file for each one.
In addition to execution, the detailed output you get from
tkn is vastly improved over the
kubectl describe equivalents. For example, after running an example pipeline using conditions (more on those in a moment), I can get a compact list of recent runs and see useful information about the one I’m most interested in:
> tkn taskrun list NAME STARTED DURATION STATUS condtional-pr-then-check-gtp96 27 minutes ago 7 seconds Succeeded condtional-pr-then-check-gtp96-file-exists-v9m2t 27 minutes ago 14 seconds Succeeded condtional-pr-first-create-file-4n9zh 27 minutes ago 22 seconds Succeeded steps-run-in-order-p6v7q 5 hours ago 9 seconds Succeeded echo-hello-world-task-run 1 day ago 9 seconds Succeeded > tkn taskrun describe condtional-pr-first-create-file-4n9zh [0/3547] Name: condtional-pr-first-create-file-4n9zh Namespace: default Task Ref: create-readme-file Service Account: default 🌡️ Status STARTED DURATION STATUS 4 hours ago 22 seconds Succeeded [ ... ] 🦶 Steps NAME STATUS ∙ write-new-stuff Completed ∙ create-dir-workspace-868lz Completed ∙ source-copy-pipeline-git-bpt22 Completed ∙ source-mkdir-pipeline-git-4lg5b Completed 🚗 Sidecars No sidecars
(Yes, the emoji are part of the output! 😺)
kubectl describe commands would spew out some verbose, unhelpful output and require lots of
awk-ward manipulation in order to see the same level of usable info we see here.
Pipeline Improvements: Conditions
I mentioned above that the sample pipeline run we’re looking at makes use of Conditions. Conditions landed recently and are a welcome addition to Pipeline capabilities. Conditions extend Tekton’s core concept — launching task-specific containers and managing their success or failure — to enable decision-making in the middle of a pipeline run. Normally, a container’s entrypoint exiting with a non-zero exit code indicates that something went horribly wrong — it couldn’t clone a git repository, or unit tests failed, perhaps — but a container flagged as a condition failing just means that other Tasks marked as dependent on that container’s success will not be executed.
The pipeline run that generated the output above contains a condition that shows the utility and potential power of the feature.
apiVersion: tekton.dev/v1alpha1 kind: Condition metadata: name: file-exists spec: params: - name: 'path' resources: - name: workspace type: git check: image: alpine script: 'test -f $(resources.workspace.path)/$(params.path)'
This Condition uses the
alpine image to run a really simple inline script to make sure a given file exists.
--- apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: conditional-pipeline spec: resources: - name: source-repo type: git params: - name: 'path' default: 'README.md' tasks: - name: first-create-file taskRef: name: create-readme-file resources: outputs: - name: workspace resource: source-repo - name: then-check conditions: - conditionRef: 'file-exists' params: - name: 'path' value: '$(params.path)' resources: - name: workspace resource: source-repo from: [first-create-file] taskRef: name: echo-hello
The Pipeline then creates a
README.md file on line 14, then uses
file-exists condition (starting on line 22) to test whether the file was created successfully. If so, it executes the
echo-hello task. Both of these individual tasks are defined as separate Task objects to enable reuse:
--- apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: create-readme-file spec: resources: outputs: - name: workspace type: git steps: - name: write-new-stuff image: ubuntu script: 'touch $(resources.outputs.workspace.path)/README.md' --- apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: echo-hello spec: steps: - name: echo image: ubuntu script: 'echo hello'
There’s an elegance to this design that, to me anyway, expresses one of the coolest parts of working with Kubernetes. It distills a problem down to a set of declarative resources: has this container run? If not, run it. Then builds upwards: Did it exit successfully? If so, proceed with the next task.
The specification is described in the Conditions doc and there are several more conditional examples in the tektoncd/pipeline repo. Make sure you’re running the latest version as some of the implementation details are still solidifying as Tekton moves towards its beta release in early 2020.
Dashboard and Trigger Deliciousness
One of the best things about the Tekton community is the weekly working group calls. They’re open to anyone who joins the tekton-dev Google group (check out the details in the tektoncd/community repo), stay on-topic more than 95% of meetings I’ve seen, and present a really powerful and positive example of how multi-vendor open source ought to work. On any given Wednesday, there’ll be representatives from Google, VMware, IBM, Red Hat (well, a different part of IBM, anyway!), eBay, Apple, D2iQ, and many other vendor and end user organizations. While some of these companies’ agendas might diverge, the team overall works in a remarkably healthy and cooperative manner to get stuff done: from collaborative comments on design docs to encouraging messages on pull requests, from my perspective Tekton is a burgeoning open-source success story.
On a recent working group call, Andrea Frittoli from IBM did a two-fer demo, showing not only how Tekton dogfoods Tekton for its own build and deployment pipeline, but how far the Tekton Dashboard has come in displaying the inner workings of the infrastructure.
Dashboard is an graphical add-on to Tekton, primarily developed by IBM folks as an upstream component of the Kabanero.io project, which (as you might expect) provides a graphical view into what’s going on across your Tekton deployment. I’ll give a brief rundown of Dashboard’s capabilities here; for an in-depth look, check out Adam Roberts’ blog post on the IBM developer blog “Why Now’s a Great Time to Use the Tekton Dashboard” (for whatever value of “now” happens to be true for you!)
As you can see from the sidebar, the Dashboard has kept pace with the new CRDs that the platform implements. One that’s particularly interesting is the highlighted
EventListeners resources — the ones in the dogfooding repo are “raw” resources, but there’s an extension to the Dashboard that allows you to easily configure webhook events from external services so you can (for example) send Tekton an event when a Github PR is merged.
The Dashboard’s webhooks extension provides a friendly interaction layer into the Triggers system, but even without it the Tekton Triggers project is very cool. It allows Tekton to become responsive to events that originate outside of your Kubernetes cluster. This makes it much easier to integrate Tekton into your broader infrastructure, instead of requiring explicit requests against the k8s cluster API to trigger task and pipeline runs. The
EventListeners , as in the screenshot above, provide an endpoint for external systems to talk to. As events come in,
TriggerBindings transform fields form them into parameters for
TriggerTemplates. The templates, in turn, generate resources like the Task and Pipeline runs that we’ve already seen. This level of dynamic instantiation truly levels Tekton’s capabilities up to the point where it can handle modeling the complexity of modern deployment scenarios.
As you can tell by the length of this post, I’m really excited about all of the advancement in Tekton. As part of Relay, we’ve been trying to both keep abreast of the new features and contribute back wherever we can (in fact the cute kitten pic was from one of our pull requests!). The
tkn CLI is an awesome example of using a familiar shell-based paradigm to interact with an API-driven service, Conditions bring a much-needed level of expressiveness to pipeline definitions, and we believe deeply in the future of event-driven automation.
Here are some links to follow up with the Tekton community and the related projects I mentioned in this series:
- The tektoncd/community repo has links to our Slack workspace and mailing lists.
- The Jenkins X community page has office hours and contact info.
- The Kabanero project is a spicy way to get teams deploying Kubernetes apps faster.
- We’d love to get your input on which services and tools you’d like to connect with Relay.