GitHub App PR Analysis

Analyze pull request changed files with AST, security, dependency, license, Score v2, PR comments, and commit statuses

GitHub App PR Analysis

ReformCode GitHub App PR analysis turns pull request webhooks into a proof-bearing quality gate. Instead of posting a cosmetic score, the webhook fetches changed files from GitHub, compares head and base snapshots, runs ReformCode analysis, and writes the result back to the pull request.

What Runs

For supported changed files, ReformCode runs:

  • AST code quality analysis for supported source languages.
  • Code security audit for risky patterns and secrets.
  • Dependency vulnerability scanning for changed manifests and lockfiles.
  • License compliance scanning for changed manifests and license evidence.
  • Score v2 composition for the changed-file snapshot.
  • Rule evaluation for merge-blocking policy context.

The PR comment includes the current score, score diff, security findings, license status, and block reasons. The webhook also sets a ReformCode commit status on the pull request head SHA.

GitHub App Requirements

Set these environment variables:

GITHUB_APP_ID="123456"
GITHUB_APP_INSTALLATION_ID="12345678"
GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----"
GITHUB_WEBHOOK_SECRET="replace-with-a-long-random-webhook-secret"

The app needs these repository permissions:

  • Contents: read.
  • Metadata: read.
  • Pull requests: read.
  • Issues: read/write for PR comments.
  • Commit statuses: read/write for the ReformCode status.

Subscribe the webhook to Pull request and Installation events and point it at:

https://your-domain.com/api/webhooks/github

Operational Notes

PR analysis is bounded to avoid runaway webhook work. The default cap is 50 changed files and 200 KB per fetched file. Unsupported or oversized files are skipped rather than downloaded into the analyzer.

If GitHub token exchange, comment upsert, or commit status writes fail, the webhook returns a failure instead of silently claiming analysis succeeded. This is intentional: PR quality gates should be truthful, not decorative.

Verification

Focused coverage lives in:

  • __tests__/lib/github-app/pr-analysis.test.ts
  • __tests__/lib/github-app/webhook-handler-flow.test.ts
  • __tests__/lib/github-app/webhook-handler.test.ts
  • __tests__/lib/github-app/pr-comment.test.ts

Run:

npx vitest run __tests__/lib/github-app/pr-analysis.test.ts __tests__/lib/github-app/webhook-handler-flow.test.ts __tests__/lib/github-app/webhook-handler.test.ts __tests__/lib/github-app/pr-comment.test.ts