New Checks API public beta

Today we're adding new functionality which allows integrations and GitHub to communicate more comprehensively about the checks run against code changes. These changes will improve your workflow by allowing you to view feedback from code checks directly in the pull request view, see the line of code causing a problem in the diff view, “re-run” checks, and more-all within the GitHub user interface.

Instead of binary pass/fail commit statuses, integrators can now report more fine-grained outcomes, such as a neutral conclusion for more informational analysis or action_required if the integrator site requires the user's attention.

We've added first-class support for common workarounds, like the ability to skip or request checks via commit message, trigger checks with a dedicated check_suite webhook event, and set preferences for when checks are triggered.

Integrator workflows

Automatic flow

1. By default, we will automatically send installed applications that have the checks:write permission a check_suite webhook event with the action requested. When an app receives this event, it should perform its analysis on the code changes.
2. GitHub displays the apps that have been requested to run checks in the pull request view with a status of queued.
3. An app creates a check run with a started_at timestamp and a status (typically in_progress) for each analysis check that the app performs.
4. An app updates the check run and includes information about the output of its analysis.
5. GitHub displays the output of the check run in the pull request view.
6. Repeat steps four and five if needed. Apps can send data in stages until their analysis is complete—when it sends a completed_at timestamp and conclusion.
7. GitHub displays all the check suites and runs data in the pull request view.

Opt out of the automatic flow

Because some checks may be expensive to run, an application can opt out of the automatic flow and instead create check suites to their preferred timing. Application owners can control this by setting their check suite preferences on a per-app and repository basis. People with admin permission on a repository can override this setting.

User workflows

Check requests via the API

The automatic flow prompts installed apps to run checks against the last commit in code pushed to a repository. A user can request to run checks for any given commit with the rerequest a check suite endpoint:

POST /repos/:owner/:repo/check-suites/:id/rerequest

Check requests via commit message

A user can control the check suite request flow on a per-commit basis:

• To prevent a check suite from being created, include skip-checks: true in the trailers of the commit message.
• To request a check suite, when automatic creation is disabled, include request-checks: true in the trailers of the commit message.

Note: the trailer would need to be on the last commit of a push. If you supply both trailers, skip-checks wins.

See skipping and requesting checks for individual commits for more information.

Check requests via the Web UI

A user can re-run a check run or entire check suite in the pull request view on GitHub.com.

When a user requests to re-run a check, a check_run webhook event is delivered to that app's webhook with an action of rerequested. The app is then expected to create a new check run for the given head_sha.

When a user requests to re-run a check suite, a check_suite webhook event is delivered to that app's webhook with an action of rerequested. The app is then expected to create a new check run for all its runs in that suite.

Questions

How do checks work with protected branches?

If the names of check runs are the same as the context of prior commit statuses, and those statuses were required, then the new check runs are automatically required. If a commit status and check run are created with the same name or context, both the status and the check run will be required.

If the names of the new check runs are different from the old commit statuses, the new check runs will need to be selected as required.

How are check runs different than commit statuses?

Commit statuses allow for a simple pass or fail state. Check runs allow for much more granular information: they can conclude as either success, failure, neutral, cancelled, timed_out, or action_required. Check runs are more flexible than commit statuses. You can update the lifecycle state by indicating queued, in_progress, or completed through the status field.

Check runs can be created as simply as a commit status with just a name and conclusion for the given commit. They can also include a variety of output data: textual information, images, and feedback on specific lines of code.

Is this supported in the GraphQL API?

No, but we plan to add support in the near future.

Who can use it?

The Checks API is only available to GitHub Apps through a new granular permission: checks.

How can I try it?

Anyone can register a GitHub App on GitHub through Settings > Developer settings > GitHub Apps and manage an existing GitHub App from the same place. See how to get started building GitHub Apps.

See the full Checks API documentation for more details.

To access this functionality, you’ll need to provide the following custom media type in the Accept header:

application/vnd.github.antiope-preview+json

Share your feedback

We're excited to see what you build with these new improvements. If you have any questions, please let us know.

Looking for a new app to use? Browse GitHub Marketplace.