Docs
Code Review

Which PRs get reviewed

One filename, and the repository holding it decides what it governs. Each review then covers only the commits since the last one.

Two rules decide what gets reviewed. There is one pipeline filename, code_review.yaml, and which repository you commit it to decides whether it governs that repository or the whole organization. And every review covers only the commits since the last review, never the whole pull request again.

Where the file lives is what it governs

There is one filename. Position decides its meaning:

Where you commit code_review.yamlGoverns
Any repository, at its rootThat repository.
Your .ellipsis repository, at its rootEvery repository in the organization.

.ellipsis/code_review.yaml works too, in any repository, and governs that repository. When a repository holds both, the root file wins.

splitshift-hq/.ellipsis
  code_review.yaml           governs every repository

splitshift-hq/splitshift-web       no file of their own, so the
splitshift-hq/splitshift-mobile    organization-wide file governs

splitshift-hq/splitshift-api
  code_review.yaml           governs splitshift-api, in place of
                             the organization-wide file

Ellipsis takes the first one that exists: the repository's own file, else the organization-wide one, else the built-in pipeline. Consequences worth knowing:

  • The .ellipsis repository is never itself reviewed. It is pure configuration. That is what keeps one filename unambiguous: its copy can only mean organization-wide, and a copy anywhere else can only mean that repository.
  • A repository's own file replaces the organization-wide one, rather than layering onto it. A repository file that sets only pull_requests: runs the built-in reviewers, not the ones your organization-wide file declares, so copy across whatever you meant to keep.
  • The organization-wide file's filters scope its own reach, not the organization's. They are consulted only when they are the file that governs, so a repository the organization-wide file excludes reviews itself the moment it commits its own code_review.yaml. A team can always opt in.
  • Whichever file wins is still an overlay onto the built-in pipeline. Fields it leaves unset inherit the built-in ones. See the pipeline file.
  • A pipeline at any other path is a configuration error, reported on the file when it syncs. A file that governs nothing would otherwise sit there looking live and reviewing nothing.
  • If Ellipsis cannot read your .ellipsis repository (the App was removed from it, or it was renamed or deleted), the organization-wide rung drops and reviews continue on the built-in pipeline. Reviews never run on a policy Ellipsis can no longer read.

Nothing else is consulted, so two files can never compete for one pull request.

Four ways to control scope

Everything about which pull requests get reviewed comes down to these four:

  • Off everywhere: turn the switch off on the Settings tab of /reviews. It is off by default and gates every path in, including the API.
  • Only some repositories: set pull_requests.repositories in the organization-wide file.
  • Different behavior in one repository: commit code_review.yaml there, with its own reviewers, prompts, models, filters, or budget.
  • Off in one repository: commit a code_review.yaml there whose pull_requests: matches nothing. Its filters are authoritative for that repository, so nothing falls through to the organization-wide file or the built-in pipeline.
# code_review.yaml: no reviews in this repository
ellipsis:
  version: v1
  kind: code_review
  name: Review disabled

pull_requests:
  for:
    users: false
    bots: false

ellipsis.enabled: false is not the way to do that. It marks the file itself inactive, so Ellipsis reads it as "no policy here" and continues down the ladder to the organization-wide file, then the built-in pipeline. Use it to park a file you are drafting, not to opt a repository out.

New commits only

The first review of a pull request covers everything on it, including commits pushed before it was opened. Every review after that covers only the commits since the last review that posted:

base ── A ── B ── C ── D ── E        commits on the PR branch

        └─────────┘
      review 1: base...C             the PR is opened at C; the first
                                     review reads the whole PR (A, B, C)

                       └────┘
      review 2: C...E                D and E are pushed later; the next
                                     review reads only D and E, and never
                                     re-comments on A, B, or C

When priya-shah opens "Expire pending trades on account deactivation" and pushes twice more over the afternoon, the pipeline runs three times, each time reading only what that push added. If the third push drops a rest-period check, the comment lands on that line, with no repetition of the earlier reviews.

Two properties follow:

  • A review that fails does not count as coverage. If a review errors or hits its budget, the commits it was reading stay unreviewed, and the next review covers everything since the last one that actually posted. Nothing is silently skipped.
  • A clean review still counts. A review that finds nothing posts a one-line summary rather than invented nitpicks, and the commits it read are covered.
  • Coverage belongs to the pull request, not to the pipeline that reviewed it. Adopting a pipeline file, editing one, or deleting one mid-pull-request picks up where the last review left off rather than re-reading the whole pull request.

Each review anchors to the exact commit it reviewed, so its comments stay attached to the right lines as the pull request grows. The reviewed range is visible on every review: the dashboard shows the commits each review covered, and the API reports it as scope.watermark (the last reviewed commit) and scope.head.

Repo scoping

Two independent fields name repositories, and they answer different questions:

  • pull_requests.repositories is the watch set: which repositories' pull requests this file reviews. Valid only in the .ellipsis repository's copy, since a repository's own file is already scoped by where it sits. Declaring it anywhere else is a configuration error, not a silently ignored key.
  • environment.repositories is the context set: extra repositories cloned into the reviewer's sandbox. The pull request's own repository is always cloned, so it never needs listing. Valid in either file.
# code_review.yaml in your .ellipsis repository
ellipsis:
  version: v1
  kind: code_review
  name: API review with web context

pull_requests:
  repositories: [splitshift-api]

environment:
  repositories:
    - name: splitshift-web

Only splitshift-api pull requests are reviewed, and splitshift-web is on disk while reviewing, so a change to an API response shape is judged against the client that consumes it.

The watch set is an include-minus-exclude filter. A bare list is shorthand for include; an empty or omitted include means every repository of the installation, so an exclude-only filter reviews everything except the listed repositories, and stays correct as repositories are added to the organization later:

pull_requests:
  repositories:
    exclude: [sandbox-experiments, splitshift-docs]

The watch set is authoritative for this file. Scoping the organization-wide file to splitshift-api means "and nothing else": a pull request in splitshift-mobile gets no review, rather than falling back to the built-in pipeline. Ellipsis never reviews, or bills you for, something your file excluded. It is not an organization-wide ceiling, though: a repository left out this way opts back in by committing its own code_review.yaml, which replaces the organization-wide file for that repository, filters included.

Pull request filters

Beside repositories, the pull_requests: block filters on branch, draft status, labels, changed paths, and author. Those five are valid in either file. All declared filters must pass; an omitted filter passes everything.

ellipsis:
  version: v1
  kind: code_review
  name: API release review

pull_requests:
  repositories: [splitshift-api]
  base: [main, "release/*"]
  draft: false
  paths: [src/**, sql/**]
  for:
    users: true
    bots: false

Against that file:

  • priya-shah's "Reject overlapping shift trades" into main, touching src/trades.py: reviewed.
  • The same change opened as a draft: not reviewed (draft: false), until it is marked ready and the next push lands.
  • A Dependabot bump into main: not reviewed, because this file set bots: false. The built-in pipeline reviews bots; declaring for: is what turns them off.
  • marcus-lee's README-only change: not reviewed (no changed file matches paths).

Field by field:

  • base and head match the target and source branch. Each entry is an exact name, a prefix glob like release/*, or the literal default for the repository's default branch whatever it is named.
  • draft unset reviews drafts and non-drafts alike; true or false requires that state.
  • labels passes a pull request carrying at least one listed label, matched exactly including case. Labels are read when the review triggers, on the open or a push; adding a label to an open pull request does not trigger a review by itself.
  • paths are include-only globs over the pull request's full changed-file set; any matching file passes. A pattern with a slash is repository-root-relative, ** crosses directories, a slashless pattern matches at any depth, and a leading / anchors to the root. Negated patterns (!foo) are a validation error.
  • for gates on the pull request's author, the account that opened it, for the pull request's whole life. Each side takes true, a list of logins, false, or an include-minus-exclude mapping, and an omitted side is false. So bots: ["renovate[bot]"] reviews Renovate's pull requests and no other bot's, and a commit someone else pushes onto an excluded author's pull request is still not reviewed. Declaring a pull_requests: block at all sets the audience to humans only; write for: {users: true, bots: true} in that block to keep reviewing bots.

Every field, type, and default is in the configuration reference.

Per-reviewer filters

The pipeline's pull_requests: decides which pull requests get reviewed at all. Each reviewer can carry its own pull_requests: block deciding which of those it bothers reading, with paths, base, and head only:

ellipsis:
  version: v1
  kind: code_review
  name: Backend review with a migration specialist

pull_requests:
  repositories: [splitshift-api]

review:
  # Runs on every pull request the pipeline matches.
  - name: correctness
    claude:
      model: claude-opus-5
      system: |
        Review this diff for defects a reviewer would ask to have fixed
        before merge: logic that does not do what the change intends,
        broken error handling, and regressions in behavior callers
        depend on. Skip style and formatting.
  # Runs only when a migration changes.
  - name: migration-safety
    claude:
      system: |
        Review database migrations for production safety. Check for
        locking that blocks writes on large tables, missing backfills
        for new non-null columns, and rollout ordering that breaks if
        the migration and the code deploy out of order.
    pull_requests:
      paths: ["**/migrations/**"]

On a pull request that touches a migration, both correctness and migration-safety run. On any other pull request, only correctness runs, and the specialist costs nothing: no sandbox starts, no model call is made, and its share of the run budget goes to the reviewers that did run.

A reviewer's block takes paths, base, and head only; repositories belongs to the pipeline, which owns the watch set. To turn a reviewer off entirely, remove it from review:; leave the whole review: key out and the built-in reviewer runs instead. See Reviewers.

When reviews are skipped

A push produces no review when:

  • Code review is off. The organization-wide setting gates every path in; see the quick start.
  • The governing pipeline does not match the pull request. Its filters are authoritative, so a non-matching pull request gets nothing rather than the built-in pipeline.
  • There is nothing new. A push whose head was already covered by the last posted review is skipped before anything starts or costs anything. Re-delivered or out-of-order events are absorbed the same way.
  • The pipeline is over its trailing budget. budget.day and budget.week are checked before a review starts, so a push storm cannot turn into a bill; reviews resume when the window rolls. See budgets.
  • A review of the pull request is already running. The push does not start a second review. When the running one finishes, Ellipsis re-checks the pull request and immediately reviews anything that landed in the meantime, so the commits are covered by the next review rather than lost.
  • Every reviewer's own filters excluded the pull request. A pipeline whose only reviewer is scoped to sql/** reviews nothing on a pull request that touches no SQL. That shows up as a cancelled review on the reviews dashboard, the commits stay uncovered, and a later push that does touch sql/** shows the specialist the whole accumulated range.

In every case above the incremental rule holds: commits nothing reviewed stay unreviewed, and the next review that runs covers all of them. The one deliberate exception is a pre_review agent you configured cancelling a review: that is a judgment that the range needs no review, so it counts as coverage and the next review starts after it.

Reviewing on demand

POST /v1/reviews reviews an open pull request without waiting for a push. Which pipeline runs is not a parameter: it resolves from the repository's files exactly as above, so an on-demand review can never disagree with the automatic one. What it adds is a pinned scope (review a historical range by naming its watermark and head) and post: false, which runs the review and returns the findings without writing anything to GitHub. See Reviews.