What this covers
A Level 2 support summary of issues found and fixes shipped while validating HarperBot's webhook deployment (Vercel) against a test repository.
- Required GitHub App permissions + webhook events
- Slash command behavior:
/analyze(try/analyze --force-reviewfor a fresh review),/apply,/merge,/squash,/rebase - Operational controls:
/pause,/resume,/status - Why reviews/inline suggestions sometimes don't appear
- Optional multi-line inline review comments ("comment on lines ...")
- Common log patterns and how to interpret them
Baseline permissions
Minimum repository permissions for review + merge + apply:
Metadata: read (required)
Pull requests: read & write
Issues: read & write
Code/Contents: read & write
Everything else: No access (Actions/Checks/Secrets/Administration/etc.).
Webhook events
Pull request
Issue comment
Pull request review comment
The third event enables commands posted as inline comments on the "Files changed" tab.
Fixes shipped (L2 summary)
| Symptom | Root cause | Resolution |
|---|---|---|
/apply returned 500 with pr unbound |
Code path referenced pr when there were no suggestions |
Always fetch repo/pr before branching; added tests |
/analyze skipped due to "analysis already exists for SHA ..." |
Dedup logic applied to manual reruns | Manual /analyze reruns; use /analyze --force-review to repost inline suggestions |
| PR comment posted but no PR review / inline suggestions | Review creation only happened when inline suggestions existed; brittle diff-position math | Always create a review; use line-based inline comments with a fallback |
Inline /analyze comment in "Files changed" was ignored |
Webhook handler only processed issue comments | Added handling for pull_request_review_comment "created" events |
/apply said "No code suggestions found" even when diff blocks were present |
Parser expected --- a/ headers; deletions weren't supported |
Support simplified diff blocks + deletion operations; apply deletes safely |
| Wanted "comment on lines +X to +Y" style inline reviews | Inline reviews were posted as single-line comments only | Added optional range comments (multi-line) gated by HARPERBOT_ENABLE_RANGE_COMMENTS |
| Need to temporarily stop auto reviews on a noisy PR | No per-PR control for auto triggers | Introduce /pause and /resume using the harperbot:paused label |
Tip: When triaging "ignored" events, remember the handler processes PR actions (opened,
reopened, synchronize) plus comment events. Review "submitted" events are expected
to be ignored.
Validation checklist
- Create a PR with a small code change
- Comment
/analyzein Conversation (and optionally as an inline file comment) - Verify a PR review appears (“View reviewed changes”)
- Comment
/applyand confirm a bot-authored commit lands on the PR branch - Optionally test merges:
/merge,/squash,/rebase - Pause auto runs:
/pause; re-enable:/resume
Quota cooldown
When Gemini returns a quota/rate-limit error, HarperBot posts a notice and temporarily skips auto-analysis on new commits for a short cooldown window.
Config (Vercel → Project → Settings → Environment Variables):
HARPERBOT_QUOTA_COOLDOWN_SECONDS=1800
Examples: 300 (5 min), 1800 (30 min, default), 0 (disable cooldown).
Multi-line inline reviews
By default, HarperBot posts inline review comments anchored to a single line. If you want GitHub’s “comment on lines +X to +Y” style UI for block feedback, enable range comments.
Config (Vercel → Project → Settings → Environment Variables):
HARPERBOT_ENABLE_RANGE_COMMENTS=1
Notes: keep this off if you only want one-line ```suggestion``` fixes. Range comments are best
for block-level guidance; /apply continues to work for suggestions.
Log signals
Useful strings to filter in Vercel logs:
Processing /analyze
Error handling apply comment:
Posted ... inline suggestions as a review
Posted a review without inline suggestions
Quota exceeded for PR
Ignored webhook event: