New Validation Rule for Beta Code Search API

As we prepare to end the preview period for the new search API, we're making sure that it's ready to handle the traffic from all the apps you'll build on top of it.

New Validation Rule

In order to support the expected volume of requests, we're applying a new validation rule to the Code Search API. Starting today, you will need to scope your code queries to a specific set of users, organizations, or repositories.

As usual, you specify the query via the q parameter. The value must include at least one user, organization, or repository.

For example, with this query, we're searching for code from @twitter or @facebook that uses an MIT License:

MIT License user:twitter user:facebook

And here, we're looking for uses of the underscore library in @mozilla's BrowserQuest repository:

underscore language:js repo:mozilla/BrowserQuest

To perform these queries via the API, we would use the following URLs (respectively):

https://api.github.com/search/code?q=MIT+License+user%3Atwitter+user%3Afacebook

https://api.github.com/search/code?q=underscore+language%3Ajs+repo%3Amozilla%2FBrowserQuest

All the various code search qualifiers are still available to you. A user, organization, or repository qualifier is now required. The other search qualifiers are still optional.

Other Search Types Not Affected

This new validation only applies to the Code Search API. It does not apply to the Search API for issues, users, or repositories.

This validation does not affect searches performed on github.com/search.

By ensuring that code queries are more targeted in nature, the API will be ready to meet the expected demand from all your apps. As we continue to tune the Search API, we hope to relax this validation in the future. There's no ETA, but we'd like to relax it as soon as it's feasible.

As always, if you have any questions or feedback, please get in touch.

List all teams for the authenticated user

We just added a new API method to list all the teams for the authenticated user across all organizations:

curl -H "Authorization: token [yours]" https://api.github.com/user/teams

[
  {
    "name": "Testing",
    "id": 396018,
    "slug": "testing",
   "permission": "pull",
   "url": "https://api.github.com/teams/396018",
   "members_url": "https://api.github.com/teams/396018/members{/member}",
   "repositories_url": "https://api.github.com/teams/396018/repos",
   "members_count": 1,
   "repos_count": 0,
   "organization": {
     "login": "dotfiles",
     "id": 1593590,
     "url": "https://api.github.com/orgs/dotfiles",
     "repos_url": "https://api.github.com/orgs/dotfiles/repos",
     "events_url": "https://api.github.com/orgs/dotfiles/events",
     "members_url": "https://api.github.com/orgs/dotfiles/members{/member}",
     "public_members_url": "https://api.github.com/orgs/dotfiles/public_members{/member}",
     "avatar_url": "https://0.gravatar.com/avatar/67d30facf213f62853c119fc2a05e246?d=https%3A%2F%2Fidenticons.github.com%2Fc90a68e6ab739e81c642f0e93f88c722.png"
   }
 },
 ...
]

As always, if you have any questions or feedback, please drop us a line.

OAuth changes coming

Starting today, we are returning granted scopes as part of the access_token response. For example, if you are making a POST with the application/json mime-type you'll see an additional field for the granted scopes.

{
  "access_token":"e72e16c7e42f292c6912e7710c838347ae178b4a",
  "scope":"repo,gist",
  "token_type":"bearer"
}

Right now, these scopes will be identical to what you requested, but we are moving towards a feature set that will allow GitHub users to edit their scopes, effectively granting your application less access than you originally requested. You should be aware of this possibility and adjust your application behavior accordingly.

Some things to watch out for and keep in mind:

  • Most third party applications using GitHub OAuth to identify users have the best success in adoption by starting out with a request for the minimum access that the application can possibly get away with. Something like no scopes or just user:email is very sane.

  • It is important to handle the error cases where a user chooses to grant you less access than you originally requested. Now that we are surfacing the granted scopes on the access_token response, applications can warn or otherwise communicate with their users that they will see reduced functionality or be unable to perform some actions.

  • Applications can always send users back through the flow again to get additional permission, but don't forget that users can always say no.

An Update on the New Search API

We owe a big "Thank You!" to everyone that has taken the time to try out the new Search API. We :heart: every one of you. Just as we hoped, the preview period has allowed us to see how you want to use the new API, and it has given us a chance to improve the API before finalizing it.

In order to incorporate everything that we've learned, we're going to keep the Search API in preview mode for a little while longer. We have a few bugs to squash and a couple performance kinks to iron out. We're hard at work on those improvements now, and we expect to have more news in the coming weeks.

In the mean time, keep the suggestions coming!

Releases API

This summer we made it easier to release your software. Today, you can fully automate those releases via the Releases API.

This API is a little different due to the binary assets. We use the Accept header for content negotiation when requesting a release asset. Pass a standard API media type to get the API representation:

curl -i -H "Authorization: token TOKEN" \
    -H "Accept: application/vnd.github.manifold-preview" \
    "https://uploads.github.com/repos/hubot/singularity/releases/assets/123"

HTTP/1.1 200 OK

{
  "id": 123,
  ...
}

Pass "application/octet-stream" to download the binary content.

curl -i -H "Authorization: token TOKEN" \
    -H "Accept: application/octet-stream" \
    "https://uploads.github.com/repos/hubot/singularity/releases/assets/123"

HTTP/1.1 302 Found

Uploads are handled by a single request to a companion "uploads.github.com" service.

curl -H "Authorization: token TOKEN" \
     -H "Accept: application/vnd.github.manifold-preview" \
     -H "Content-Type: application/zip" \
     --data-binary @build/mac/package.zip \
     "https://uploads.github.com/repos/hubot/singularity/releases/123/assets?name=1.0.0-mac.zip"

Preview mode

The new API is available as a preview. This gives developers a chance to provide feedback on the direction of the API before we freeze changes. We expect to lift the preview status in 30 days.

As with the Search API, we'll take this opportunity to iterate quickly. Breaking changes will be announced on this developer blog without any advance warning. Once the preview period is over, we'll consider the Releases API unchangeable. At that point, it will be stable and suitable for production use.

The preview media type is "application/vnd.github.manifold-preview". Manifold is a member of the Avengers, with the ability to teleport through time and space. He's the one in the middle holding the spear.

Manifold teleporting the Avengers to a terraformed Mars surface

Two-Factor Authentication and the API

As announced earlier today, GitHub.com now supports two-factor authentication (2FA) for increased security. For users with this feature enabled, GitHub.com will prompt for a 2FA code in addition to a username and password during authentication. We've also rolled out some improvements to the API to ensure that 2FA requirements in the API are consistent with GitHub.com.

Authenticating with the API

For users without 2FA enabled, and for applications using the OAuth web flow for authentication, everything is business as usual. You'll continue to authenticate with the API just as you always have. (That was easy.)

If you enable 2FA and use Basic Authentication to access the API, we're providing multiple options to make the flow simple and easy.

Basic Authentication and 2FA

Personal Access Tokens

Personal access tokens provide the simplest option for using 2FA with Basic Authentication. You can create these tokens via the tokens settings page on GitHub.com, and you can revoke them at any time. For more information about authenticating to the API with personal access tokens, be sure to check out our help article on the topic.

Tightly-integrated 2FA

For developers wishing to integrate GitHub 2FA directly into their application, the API's Basic Authentication now supports the ability to send the user's 2FA code, in addition to the username and password.

We're here to help

We think GitHub users are going to love the additional security provided by two-factor authentication. As always, if you have any questions or feedback, let us know. We're here to help!

Improvements to the Search API

Today we're shipping two improvements to the new Search API.

More Text Match Metadata

When searching for code, the API previously provided text match metadata (i.e., "highlights") for file content. Now, you can also get this metadata for matches that occur within the file path.

For example, when searching for files that have "client" in their path, the results include this match for lib/octokit/client/commits.rb:

{:.json} { "name": "commits.rb", "path": "lib/octokit/client/commits.rb", "text_matches": [ { "object_url": "https://api.github.com/repositories/417862/contents/lib/octokit/client/commits.rb?ref=8d487ab06ccef463aa9f5412a56f1a2f1fa4dc88", "object_type": "FileContent", "property": "path", "fragment": "lib/octokit/client/commits.rb", "matches": [ { "text": "client", "indices": [ 12, 18 ] } ] } ] // ... }

Better Text Match Metadata

Before today, the API applied HTML entity encoding to all fragment data. For example, imagine your search returns an issue like rails/rails#11889:

Example Issue Title

The response would include a text_matches array with the following object:

{:.json} { "fragment": "undefined method 'except' for #<Array:XXX>", // ... }

Inside the fragment value, we see HTML-encoded entities (e.g., &lt;). Since we're returning JSON (not HTML), API clients might not expect any HTML-encoded text. As of today, the API returns these fragments without this extraneous encoding.

{:.json} { "fragment": "undefined method 'except' for #Array:XXX", // ... }

Preview Period

We're about halfway through the preview period for the new Search API. We appreciate everyone that has provided feedback so far. Please keep it coming!

Preview the New Search API

Today we're excited to announce a brand new Search API. Whether you're searching for code, repositories, issues, or users, all the query abilities of github.com are now available via the API as well.

Maybe you want to find popular Tetris implementations written in Assembly. We've got you covered. Or perhaps you're looking for new gems that are using Octokit.rb. No problem. The possibilities are endless.

Highlights

On github.com, we enjoy the context provided by code snippets and highlights in search results.

code-snippet-highlighting

We want API consumers to have access to that information as well. So, API requests can opt to receive those text fragments in the response. Each fragment is accompanied by numeric offsets identifying the exact location of each matching search term.

Preview period

We're making this new API available today for developers to preview. We think developers are going to love it, but we want to get your feedback before we declare the Search API "final" and "unchangeable." We expect the preview period to last for roughly 60 days.

As we discover opportunities to improve this new API during the preview period, we may ship changes that break clients using the preview version of the API. We want to iterate quickly. To do so, we will announce any changes here (on the developer blog), but we will not provide any advance notice.

At the end of preview period, the Search API will become an official component of GitHub API v3. At that point, the new Search API will be stable and suitable for production use.

What about the old search API?

The legacy search API is still available. Many existing clients depend on it, and it is not changing in any way. While the new API offers much more functionality, the legacy search endpoints remain an official part of GitHub API v3.

Take it for a spin

We hope you'll kick the tires and send us your feedback. Happy searching finding!

When Does My Rate Limit Reset?

Have you ever wondered when your rate limit will reset back to its maximum value? That information is now available in the new X-RateLimit-Reset response header.

curl -I https://api.github.com/orgs/octokit

HTTP/1.1 200 OK
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1372700873
...

The X-RateLimit-Reset header provides a Unix UTC timestamp, letting you know the exact time that your fresh new rate limit kicks in.

The reset timestamp is also available as part of the /rate_limit resource.

curl https://api.github.com/rate_limit

{
  "rate": {
    "limit": 60,
    "remaining": 42,
    "reset": 1372700873
  }
}

For more information on rate limits, be sure to check out the docs.

If you have any questions or feedback, please drop us a line.

Feeds API

Today we're releasing a new Feeds API, an easy way to list all the Atom resources available to the authenticated user.

curl -u defunkt https://api.github.com/feeds

{
  "timeline_url": "https://github.com/timeline",
  "user_url": "https://github.com/{user}",
  "current_user_public_url": "https://github.com/defunkt",
  "current_user_url": "https://github.com/defunkt.private?token=abc123",
  "current_user_actor_url": "https://github.com/defunkt.private.actor?token=abc123",
  "current_user_organization_url": "https://github.com/organizations/{org}/defunkt.private.atom?token=abc123",
  "_links": {
   "timeline": {
     "href": "https://github.com/timeline",
     "type": "application/atom+xml"
   },
   "user": {
     "href": "https://github.com/{user}",
     "type": "application/atom+xml"
   },
   "current_user_public": {
     "href": "https://github.com/defunkt",
     "type": "application/atom+xml"
   },
   "current_user": {
     "href": "https://github.com/defunkt.private?token=abc123",
     "type": "application/atom+xml"
   },
   "current_user_actor": {
     "href": "https://github.com/defunkt.private.actor?token=abc123",
     "type": "application/atom+xml"
   },
   "current_user_organization": {
     "href": "https://github.com/organizations/{org}/defunkt.private.atom?token=abc123",
     "type": "application/atom+xml"
   }
  }
}

If you have any questions or feedback, please drop us a line.