How to summarize pull requests with AI

This article goes over how to use AI (LLMs) to summarize pull requests (PRs) with GitHub Actions.

Photo by Nahrizul Kadri on Unsplash

Action

The easiest method is to use the action pull-request-summary:

# .github/workflows/pull-request-summary.yml
on: pull_request
jobs:
  pull-request-summary:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: ai-action/pull-request-summary@v1

The action reviews the code changes with a large language model (LLM) and edits the pull request description with the response.

See example.

Custom

To create a custom workflow, set up Ollama and checkout the repository:

- uses: ai-action/setup-ollama@v1
- uses: actions/checkout@v4

Then run a prompt with the PR diff against codellama and edit the PR description with the response:

- run: |
    PROMPT=$(printf 'Summarize the following code diff:\n%s' $(gh pr diff $PR_NUMBER))
    LLM='codellama'
    PR_SUMMARY=$(ollama run $LLM "$PROMPT")
    gh pr edit $PR_NUMBER --body "$PR_SUMMARY"
  env:
    GITHUB_TOKEN: ${{ github.token }}
    PR_NUMBER: ${{ github.event.pull_request.number }}

Here’s the full workflow:

# .github/workflows/pull-request-summary.yml
on: pull_request

jobs:
  pull-request-summary:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: ai-action/setup-ollama@v1
      - uses: actions/checkout@v4
      - run: |
          PROMPT=$(printf 'Summarize code diff below:\n%s' $(gh pr diff $PR_NUMBER))
          LLM='codellama'
          PR_SUMMARY=$(ollama run $LLM "$PROMPT")
          gh pr edit $PR_NUMBER --body "$PR_SUMMARY"
        env:
          GITHUB_TOKEN: ${{ github.token }}
          PR_NUMBER: ${{ github.event.pull_request.number }}

See example.

This article was originally published on remarkablemark.org on March 2, 2025.