Snyk's Journey from CLI to GitHub Integration

This post is penned by Guy Podjarny, CEO & co-founder at Snyk - for the interest of the GitHub community

At Snyk, we help developers fix vulnerabilities in open source packages, such as npm and RubyGems. Snyk started as a CLI tool to support source code management as well as build and test tools. However, running an application in the terminal is fundamentally limited in terms of scope and UI, which led us to create a deeper integration with GitHub.

Less than 6 months after launching the integration, we now see the integrated GitHub experience as far superior to the CLI. This post shares why we integrated with GitHub and how it helped - which might benefit others facing similar choices.

How GitHub helped us simplify

Unlike changing your tool of choice for CI or linting, using Snyk means introducing a new activity to your dev flow. Developers have very little time to spend on additional activities, and so we put a lot of effort into making our CLI easy and intuitive, demonstrating value in seconds. While user feedback shows we succeeded in doing so, even a perfect CLI could not address some of the challenges we came across.

Integrating with GitHub was focused heavily on breaking through these challenges, further shrinking time to value and simplifying continuous integration into the dev workflow. Here are the key ways our GitHub integration helped this goal.

Get value without leaving the browser

Most developers discover Snyk via the web (e.g. through a badge), but to use the product they had to leave the browser and go to the terminal. This may seem easy, but it's still another step, and many visitors dropped off at this point. Integrating with GitHub let users see the value immediately (for example, find or fix vulnerable packages) without leaving the browser, which reduced the drop-off substantially.

Broad repository access

Snyk's CLI is easy to use on any given project, but running it on every repository requires more effort. As a result, we noticed many users didn't test all of their application, but only a select few. With our GitHub integration, we automatically identify the repositories we can protect and test all of them at once. A single click uncovers vulnerabilities in all repositories at once, driving the right behavior with increased ease. For more information, see "Get a repository."

Separate and persistent code changes

Snyk's CLI improves projects by fixing code (mostly package files) through discovered vulnerabilities. We noticed that changes not committed immediately are often forgotten forever. With GitHub, we can submit the same fixes as a well documented Pull Request, which doesn't go away until it's merged or closed. Pull Requests also allow us to proactively submit fixes to newly discovered issues, as opposed to emailing developers and requiring them to run the CLI to produce the necessary changes.

Single click integration

Lastly, while the CLI made a one-time test easy, many users didn't integrate the vulnerability testing as part of their CI process. As before, this isn't a complicated step, but it's yet one more thing users might forget. Integrating with GitHub, we've enabled developers to use their existing GitHub Flow and bring in Snyk with a single click. We embed the test logic in the Pull Request checks, and listen to commits to track the project's dependencies. This improvement did wonders to the number of users who made security testing a continuous activity.

Spreading the word

While secondary to ease of use improvements, the GitHub integration also helped grow awareness and visibility of Snyk and the problem it solves. We didn't really plan for it, but as a company solving a new problem it proved very helpful.

Pull Request visibility

Submitting Pull Requests was the best mechanism for growing awareness within the team. When we submit a Pull Request fix, everybody dealing with the repository can see it - not just the champion who integrated Snyk in the first place. Team mates get the opportunity to learn the problem existed, see how it was solved, and sometimes even act (merge). Beyond team members, public repository watchers are also notified about these Pull Requests, and we've often seen users comment asking for a Pull Request to be merged.

Integrations directory

Integrating with GitHub also meant being listed on their new-ish Integrations Directory. GitHub customers consult this directory when looking to bolster their development workflow with tools, and finding Snyk with that mindset makes them likely to act. The integrations page still a fairly modest traffic source for us, but GitHub is now proactively encouraging users to explore it, and we look forward to seeing it bear fruit!

Badges

In addition to being a platform, GitHub is also a social network, making it a great vehicle to spread the word about the problem and solution. To that effect, Snyk offers badges, indicating the number of vulnerable dependencies this project has. The badges have been very helpful in growing awareness, and users that click the badge and land on a Snyk test page are far more likely to register than those arriving through other venues.

Consistent workflow

What constitutes a good CLI tool differs by the programming language and surrounding dev tools. GitHub Flow, on the other hand, remains mostly consistent across languages. This dramatically reduces the client-side effort of adding another language, allowing us to expand our reach and value. The difference in effort is so big that we've decided to only support GitHub when we launched our initial Ruby support, buying over a month before also releasing CLI support

Integration challenges

Technically, integrating with GitHub wasn't terribly complex given the well documented API and many open source examples. Where we did encounter challenges was with authentication and access scopes.

Separating authentication from repository access

We actually integrate with GitHub twice - once for user authentication and another for repository access. This separation can be confusing to users who are prompted to give GitHub access twice. After many conversations we decided to keep the separation between authenticating a user and granting repository access, to allow future flexibility. To reduce confusion, we added an interim screen before the second OAuth screen, and are repeatedly tracking activity and tweaking our UI flows.

Overly broad repository access scope

GitHub's current OAuth scopes for repository access are very coarse. For instance, adding a webhook means requesting permissions for all repositories instead of a single repository. In fact, Snyk doesn't even need access to source code except a specific set of files, but GitHub OAuth forces us to require full access. This is the single biggest complaint we get, and the biggest drop off point in our flow.

At the moment, we compensate for this shortcoming using a self-hosted broker and our CLI. In addition, GitHub has recently launched early access to a new Integrations API, which gives users more control over which repositories and files an app can access. We plan to switch to this new API, and expect it to mitigate this concern substantially.

Summary

Creating a tight GitHub integration was one of the best decisions we've made to date. It helped us dramatically reduce the effort to start using Snyk, and let us guide users into using it correctly with minimal effort. In addition to usability, the integration also raised our visibility across the dev team and repository users, helping us spread the word. Lastly, the consistent GitHub workflow helps us expand faster into new languages, making us relevant to more developers and providing more value to existing users.

The integration was not without its challenges, and required a fair bit of UX design and implementation details to get right. However, the effort was more than worthwhile, and we intend to keep the integrated path as a core direction.

If you have a CLI tool, you should consider adding a tight GitHub integration too. And if you're using GitHub, don't forget to test your repositories for vulnerable dependencies too!