Contribution Guide

• 1: Development setup
• 2: Making changes
• 3: Running tests
• 4: Documentation
• 5: Submitting a pull request
• 6: Developer's certificate of origin
• 7: Release & upgrade notes
• 8: Contributor Ladder

1 - Development setup

Building and running Tetragon

For local development, you will likely want to build and run bare-metal Tetragon.

Requirements

• A Go toolchain with the version specified in the main go.mod;
• GNU make;
• A running Docker service (you can use Podman as well);
• The docker-buildx-plugin (you may already have this);
• For building tests, libcap and libelf (in Debian systems, e.g., install libelf-dev and libcap-dev).

Build everything

You can build most Tetragon targets as follows (this can take time as it builds all the targets needed for testing, see minimal build):

make

If you want to use podman instead of docker, you can do the following (assuming you need to use sudo with podman):

CONTAINER_ENGINE='sudo podman' make

You can ignore /bin/sh: docker: command not found in the output.

To build using the local clang, you can use:

CONTAINER_ENGINE='sudo podman' LOCAL_CLANG=1 LOCAL_CLANG_FORMAT=1 make

See Dockerfile.clang for the minimal required version of clang.

Minimal build

To build the tetragon binary, the BPF programs and the tetra CLI binary you can use:

make tetragon tetragon-bpf tetra

Run Tetragon

You should now have a ./tetragon binary, which can be run as follows:

sudo ./tetragon --bpf-lib bpf/objs

Notes:

1. The --bpf-lib flag tells Tetragon where to look for its compiled BPF programs (which were built in the make step above).

2. If Tetragon fails with an error "BTF discovery: candidate btf file does not exist", then make sure that your kernel support BTF, otherwise place a BTF file where Tetragon can read it and specify its path with the --btf flag. See more about that in the FAQ.

Building and running a Docker image

The base kernel should support BTF or a BTF file should be bind mounted on top of /var/lib/tetragon/btf inside container.

To build Tetragon image:

make image

To run the image:

docker run --name tetragon \
   --rm -it -d --pid=host \
   --cgroupns=host --privileged \
   -v /sys/kernel/btf/vmlinux:/var/lib/tetragon/btf \
   cilium/tetragon:latest

Run the tetra binary to get Tetragon events:

docker exec -it tetragon \
   bash -c "/usr/bin/tetra getevents -o compact"

Building and running as a systemd service

To build Tetragon tarball:

make tarball

Running Tetragon in kind

This command will setup tetragon, kind cluster and install tetragon in it. Ensure docker, kind, kubectl, and helm are installed.

# Setup tetragon on kind
make kind-setup

Verify that Tetragon is installed by running:

kubectl get pods -n tetragon

Local Development with Apple Silicon Mac

Use Lima to create a Linux VM if you are using a Mac with Apple silicon. For example:

First create a VM using Lima:

brew install lima
limactl create --mount-writable --tty=false --name=tetragon
limactl start tetragon
limactl shell tetragon

Then install needed dependencies inside the VM:

sudo add-apt-repository -y ppa:longsleep/golang-backports
sudo apt update
sudo apt install -y golang-1.23 libelf-dev libcap-dev make
export CONTAINER_ENGINE=nerdctl
export PATH=$PATH:/usr/lib/go-1.23/bin

You can now build Tetragon in your VM:

make -j3 tetragon-bpf tetragon tetra

What’s next

• See how to make your first changes.

2 - Making changes

1. Make sure the main branch of your fork is up-to-date:

   git fetch upstream
   git checkout main
   git merge upstream/main
   

   For further reference read GitHub syncing a fork documentation.

2. Create a PR branch with a descriptive name, branching from main:

   git switch -c pr/${GITHUB_USERNAME_OR_ORG}/changes-to-something main
   

3. Make the changes you want.

4. Test your changes. Follow Development setup and Running tests guides to build and test Tetragon.

   • Make sure that all new code is covered by unit and/or end-to-end tests where feasible.
   • Run Tetragon locally to validate everything works as expected.
   • If adding/extending tests is not required, mention in the commit message what existing test covers the new code or how you validated the change.

5. Run code/docs generation commands if needed (see the sections below for specific code areas).

6. Run git diff --check to catch obvious white space violations.

7. Follow Submitting a pull request guide to commit your changes and open a pull request.

Making changes to documentation

To improve Tetragon documentation (https://tetragon.io/), please follow the documentation contribution guide.

Adding dependencies

Tetragon vendors Go dependencies. If you add a new Go dependency (go.mod), run:

make vendor

Most dependencies are updated automatically using Renovate. If this is not the desired behavior, you will need to update the Renovate configuration (.github/renovate.json5).

Making changes to protobuf API

Tetragon contains a protobuf API and uses code generation based on protoc to generate large amounts of boilerplate code. Whenever you make changes to these files (api/) you need to run code generation:

make protogen

Should you wish to modify any of the resulting codegen files (ending in .pb.go), do not modify them directly. Instead, you can edit the files in tools/protoc-gen-go-tetragon/ and then re-run make protogen.

Making changes to CRDs

Kubernetes Custom Resource Definitions (CRDs) are defined using Kubebuilder framework and shipped with generated Go client and helpers code. They are also included in the Helm chart for easy installation. Whenever you make changes to these files (pkg/k8s/), you need to run code generation:

make crds

Making changes to Helm chart

If you make changes to the Helm values (install/kubernetes/tetragon/values.yaml), you need to update the generated Helm values reference:

make -C install/kubernetes docs

Making changes to Prometheus metrics

If you add, change or delete metrics, you need to update the generated metrics reference:

make metrics-docs

What’s next

• See how to run the tests of the project.
• See how to submit your first pull request.

3 - Running tests

Tetragon has several types of tests:

• Go tests, composed of unit tests for userspace Go code and Go and BPF code.
• BPF unit tests, testing specifing BPF functions.
• E2E tests, for end-to-end tests, installing Tetragon in Kubernetes clusters and checking for specific features.

Those tests are running in the Tetragon CI on various kernels¹ and various architectures (amd64 and arm64).

Go tests

To run the Go tests locally, you can use:

make test

Use EXTRA_TESTFLAGS to add flags to the go test command.

Test specific kernels

To run the Go tests on various kernel versions, we use vmtests with cilium/little-vm-helper in the CI, you can also use it locally for testing specific kernels. See documentation github.com/cilium/tetragon/tests/vmtests.

BPF unit tests

To run BPF unit tests, you can use:

make bpf-test

Those tests can be found under github.com/cilium/tetragon/bpf/tests. The framework uses Go tests with cilium/ebpf to run those tests, you can use BPFGOTESTFLAGS to add go test flags, like make BPFGOTESTFLAGS="-v" bpf-test.

E2E tests

To run E2E tests, you can use:

make e2e-test

This will build the Tetragon image and use the e2e framework to create a kind cluster, install Tetragon and run the tests. To not rebuild the image before running the test, use E2E_BUILD_IMAGES=0. You can use EXTRA_TESTFLAGS to add flags to the go test command.

What’s next

• See how to submit your first pull request.

4 - Documentation

Thank you for taking the time to improve Tetragon’s documentation.

Find the content

All the Tetragon documentation content can be found under github.com/cilium/tetragon/docs/content/en/docs.

Style to follow

We generally follow the Kubernetes docs style guide k8s.io/docs/contribute/style/style-guide.

Preview locally

To preview the documentation locally, use one of the method below. Then browse to localhost:1313/docs, the default port used by Hugo to listen.

Using Docker

With a Docker service available, from the root of the repository, use:

make docs

You can also use make from the Makefile at the /docs folder level.

To cleanup the container image built in the process, you can use:

make -C docs clean

Local Hugo installation

The documentation is a Hugo static website using the Docsy theme.

Please refer to dedicated guides on how to install Hugo+extended and how to tweak Docsy, but generally, to preview your work, from the /docs folder:

hugo server

5 - Submitting a pull request

Commit changes

Save your changes in one or more commits. If you are not comfortable with Git yet (in particular with git rebase), refer to the GitHub documentation.

Write a commit message

All commits must contain a well-written commit message:

1. Write a title, no longer than 72 characters. The title should ideally answer the question “what?”. If the commit covers one specific area, start the title with a prefix like helm:, bpf:, pkg/sensors:, or metrics:.

2. Describe the changes in the commit description. Focus on answering the question why the change is required and document anything that might be unexpected. If any explanation is required to understand your code, then it should be written in the code comments instead of the commit description. Please wrap your commit description to ~70/80 chars width lines (use gq in vim for example).

3. Add a Fixes: #XXX line if the commit addresses a particular GitHub issue identified by its number. Note that the GitHub issue will be automatically closed when the commit is merged.

4. If any of the commits fixes a particular commit already in the tree, that commit is referenced in the commit message of the bugfix. The proper format for the Fixes: tag referring to commits is to use the first 12 characters of the git SHA followed by the full commit title as seen above without breaking the line.

   Fixes: 29b76c402b68 ("Refactor CRD defaulting and validation as generic")
   

5. All commits must be signed off (git commit -s). See the section Developer’s Certificate of Origin.

Example commit messages

tetragon: Fix struct perf_event_info_type layout

We have a hole in perf_event_info_type which results in wrong number in
its go counterpart MsgGenericKprobePerfEvent which is aligned differently.

Make sure the C object does not have any holes.

Fixes: #4119
Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Commit can contain previous output of commands, and the result after the patch to explicit the benefit to the reviewer, it can also be logs output, benchmark results, etc.

cmd/tetra: use text/tabwriter to format tp list output

Also add the support of load error by reusing the metric state recap.

Previously, the output of `tetra tp list` was a bit chaotic, for
example, with 2 policies, with one with a loading error, it looked like
the following:

    [1] invalid enabled:false filterID:0 namespace:(global) sensors:
        loadError: "policy handler 'tracing' failed loading policy 'invalid': tracing [...]
    [2] block-binary enabled:true filterID:0 namespace:(global) sensors:gkp-sensor-1

Now, using text/tabwriter, we redact the errors (use -o json for full
output) and put everything in columns:

    ID   NAME           STATE        FILTERID   NAMESPACE   SENSORS
    1    invalid        load_error   0          (global)
    2    block-binary   enabled      0          (global)    gkp-sensor-1

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>

Commit message can contain the Reported-by, Suggested-by, etc., tags from the kernel documentation. You can also use the Co-authored-by tag to create commits with multiple authors.

bpf: fix tail call program types for usdt sensor

As reported by Mahe newest kernels check properly on programs using
(some) maps being same type[^1]. We violate that with usdt tail called
programs having just 'uprobe' type.

[^1]: 4540aed51b12 ("bpf: Enforce expected_attach_type for tailcall compatibility")

Reported-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Include a Fixes: tag and try to explain a bug you found out.

pkg/crdutils: fix standalone custom resources validation

This fixes commit 29b76c4 ("Refactor CRD defaulting and validation
as generic") that was enhancing commit a9c9a0b ("pkg/tracingpolicy:
add k8s validation for meta and spec").

I think it's a typo that was introduced because when we do the validation
of the object, we give the function two versions of that object, cr that
is typed and was created by yaml.UnmarshalStrict the JSON object with
K8s default, and unstr that is the unstructured object version of what
was passed as input, but with K8s default. We need these two objects to
do the validation respectively on the ObjectMeta and the Spec.

When Unmarshaling the JSON object to the GenericTracingPolicy (or
others) types, all fields are set to their Golang defaults (!) in
addition to the K8s default that were already applied. So some missing
fields, that were defaulted by the K8s default were injected in the
process and thus only partial validation was done.

Fixes: 29b76c402b68 ("Refactor CRD defaulting and validation as generic")

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>

Submit a pull request

Contributions must be submitted in the form of pull requests against the upstream GitHub repository at https://github.com/cilium/tetragon.

Please follow the checklist in the pull request template and write anything that reviewers should be aware of in the pull request description. After you create a pull request, a reviewer will be automatically assigned. They will provide feedback, add relevant labels and run the CI workflows if needed.

Changelog and breaking changes

1. Document any user-facing or breaking changes in contrib/upgrade-notes/latest.md.

2. Verify the release note text. If not explicitly changed, the title of the PR will be used for the release notes. If you want to change this, you can add a special section to the description of the PR. These release notes are primarily going to be read by users, so it is important that release notes for bugs, major and minor features do not contain internal details of Cilium functionality which sometimes are irrelevant for users.

   Example of a bad release note

   ```release-note
   Fix concurrent access in k8s watchers structures
   ```
   

   Example of a good release note

   ```release-note
   Fix panic when Tetragon received an invalid Tracing Policy from Kubernetes
   ```
   

   Note If multiple lines are provided, the PR title will be used and the lines will be added as sub items.

3. If you have permissions to do so, pick the right release-note label. These labels will be used to generate the release notes which will primarily be read by users.

   Labels	When to set
   release-note/bug	This is a non-trivial bugfix and is a user-facing bug
   release-note/major	This is a major feature addition, e.g. Add MongoDB support
   release-note/minor	This is a minor feature addition, e.g. Add support for a Kubernetes version
   release-note/misc	This is a not user-facing change, e.g. Refactor endpoint package, a bug fix of a non-released feature
   release-note/docs	This is a documentation change.
   release-note/ci	This is a CI feature or bug fix.

Frequently Asked Questions

CI is complaining about Go module vendoring, what do I do?

You can run make vendor then add and commit your changes.

CI is complaining about a missing “signed-off-by” line. What do I do?

You need to add a signed-off-by line to your commit messages. The easiest way to do this is with git fetch origin/main && git rebase --signoff origin/main. Then push your changes.

6 - Developer's certificate of origin

To improve tracking of who did what, we’ve introduced a “sign-off” procedure, make sure to read and apply the Developer’s Certificate of Origin.

7 - Release & upgrade notes

Tetragon release notes are published on the GitHub releases page. To ensure the release notes are accurate and helpful, contributors should write them alongside development. Then, at the time of release, the final notes are compiled and published.

This guide is intended for both Tetragon developers and reviewers. Please follow it when creating or reviewing pull requests.

release-note blurb in PR

When you create a pull request, the template will include a release-note blurb in the description. Write a short description of your change there. Focus on the user perspective - that is, what functionality is available, not how it’s implemented.

The release-note blurb will be compiled into the release notes as a bullet point. If you delete the release-note blurb from the PR description, then the PR title will be used instead (it’s reasonable to do so for example when the change has no user impact).

release-note/* label

Each pull request should have exactly one release-note/* label. The label will be added by a reviewer, but feel free to suggest one when you create a PR.

The following release-note/* labels are available:

• release-note/major - use it for changes you want to be highlighted in the release. Typically these are new features, but the question to answer is always if it’s a highlight of the release, not how big or new the change is.
• release-note/minor - use it for other user-visible changes, for example improving an existing functionality or adding a new config option
• release-note/bug - use it for bug fixes
• release-note/misc - use it for changes that don’t have any user impact, for example refactoring or tests
• release-note/ci - use it for CI-only changes (.github directory)
• release-note/docs - use it for documentation-only changes (docs directory)
• release-note/dependency - use it for PRs that only update dependencies. This label is added automatically to PRs created by Renovate bot and is rarely used by humans.

Upgrade notes

Upgrade notes highlight changes that require attention when upgrading Tetragon. They instruct users on how to adapt in case the change requires a manual intervention.

Examples of changes that should be covered in upgrade notes:

• renaming/removing config options
• renaming/removing API fields
• renaming/removing metrics or metric labels
• changes with a significant performance impact
• deprecations

If your change entails an upgrade note, write it in the contrib/upgrade-notes/latest.md file (if your change doesn’t fit nicely in the predefined sections, just add a note at the top). The upgrade notes will be included in the release, in addition to the regular release notes.

8 - Contributor Ladder

Tetragon is a sub-project of the Cilium project and follows the same governance model, community processes, and contributor growth paths as Cilium. To support contributors in gaining both privileges and responsibilities, we adopt the shared Contributor Ladder. This contributor ladder defines how contributors can grow from community participants to project maintainers, along with the expectations at each level.

Community members generally start at the first levels of the ladder and advance as their involvement deepens. Becoming a Cilium organization member grants additional privileges across the project ecosystem, such as the ability to review pull requests or trigger CI runs. If you are contributing regularly to Tetragon, we encourage you to join the Tetragon team to help review code and accelerate development.

Your contributions play a vital role in improving the project, and the community is here to support you every step of the way.