What this covers
A Level 2 support summary of issues found and fixes shipped while validating HarperBot's webhook deployment (Vercel) against a test repository.
- Required GitHub App permissions + webhook events
- Slash command behavior:
/analyze(try/analyze --force-reviewfor a fresh review),/apply,/merge,/squash,/rebase - Operational controls:
/pause,/resume,/status - Why reviews/inline suggestions sometimes don't appear
- Optional multi-line inline review comments ("comment on lines ...")
- Common log patterns and how to interpret them
/apply is a privileged command. It requires authoring to be enabled and the commenter to have write or admin access on the repository.
Baseline permissions
Minimum repository permissions for review + merge + apply:
Metadata: read (required)
Pull requests: read & write
Issues: read & write
Code/Contents: read & write
Everything else: No access (Actions/Checks/Secrets/Administration/etc.).
Comment-level access is not enough for write operations. A non-collaborator may be able to comment on a PR but should still be denied for /apply and merge commands.
Webhook events
Pull request
Issue comment
Pull request review comment
The third event enables commands posted as inline comments on the "Files changed" tab.
Fixes shipped (L2 summary)
| Symptom | Root cause | Resolution |
|---|---|---|
/apply returned 500 with pr unbound |
Code path referenced pr when there were no suggestions |
Always fetch repo/pr before branching; added tests |
/analyze skipped due to "analysis already exists for SHA ..." |
Dedup logic applied to manual reruns | Manual /analyze reruns; use /analyze --force-review to repost inline suggestions |
| PR comment posted but no PR review / inline suggestions | Review creation only happened when inline suggestions existed; brittle diff-position math | Always create a review; use line-based inline comments with a fallback |
Inline /analyze comment in "Files changed" was ignored |
Webhook handler only processed issue comments | Added handling for pull_request_review_comment "created" events |
/apply said "No code suggestions found" even when diff blocks were present |
Parser expected --- a/ headers; deletions weren't supported |
Support simplified diff blocks + deletion operations; apply deletes safely |
/apply should not run for external contributors or when authoring is disabled |
Write-capable command needed explicit config and permission gating | Require enable_authoring: true and write/admin access; treat collaborator lookup failures as deny |
| Wanted "comment on lines +X to +Y" style inline reviews | Inline reviews were posted as single-line comments only | Added optional range comments (multi-line) gated by HARPERBOT_ENABLE_RANGE_COMMENTS |
| Need to temporarily stop auto reviews on a noisy PR | No per-PR control for auto triggers | Introduce /pause and /resume using the harperbot:paused label |
Tip: When triaging "ignored" events, remember the handler processes PR actions (opened,
reopened, synchronize) plus comment events. Review "submitted" events are expected
to be ignored.
Validation checklist
- Create a PR with a small code change
- Comment
/analyzein Conversation (and optionally as an inline file comment) - Verify a PR review appears (“View reviewed changes”)
- If authoring is enabled and the commenter has write/admin access, comment
/applyand confirm a bot-authored commit lands on the PR branch - Optionally test merges:
/merge,/squash,/rebase - Pause auto runs:
/pause; re-enable:/resume
Quota cooldown
When Gemini returns a quota/rate-limit error, HarperBot posts a notice and temporarily skips auto-analysis on new commits for a short cooldown window.
Config (Vercel → Project → Settings → Environment Variables):
HARPERBOT_QUOTA_COOLDOWN_SECONDS=1800
Examples: 300 (5 min), 1800 (30 min, default), 0 (disable cooldown).
Multi-line inline reviews
By default, HarperBot posts inline review comments anchored to a single line. If you want GitHub’s “comment on lines +X to +Y” style UI for block feedback, enable range comments.
Config (Vercel → Project → Settings → Environment Variables):
HARPERBOT_ENABLE_RANGE_COMMENTS=1
Notes: keep this off if you only want one-line ```suggestion``` fixes. Range comments are best
for block-level guidance; /apply continues to work for suggestions.
Log signals
Useful strings to filter in Vercel logs:
Processing /analyze
Error handling apply comment:
Posted ... inline suggestions as a review
Posted a review without inline suggestions
Quota exceeded for PR
Ignored webhook event: