Using LLM Agents in Sync to Build PR reviews
October 26, 2024
RBRDCK: AI-Powered Pull Request Reviewer
RBRDCK is an AI-powered pull request reviewer that leverages a local Large Language Model (LLM) to analyze code changes and provide feedback. The primary goal is to integrate this tool into continuous integration/continuous deployment (CI/CD) pipelines and GitHub Actions to automatically comment on and review pull requests.
Overview
RBRDCK stands for Reubber Duck, and it's designed to enhance the code review process by automating the analysis of pull requests. By using a local LLM, it ensures that sensitive code remains within your infrastructure, providing both cost efficiency and security.
Features
- Automated Code Reviews: Analyzes pull requests and provides feedback on code quality, documentation, test coverage, and dependencies.
- Local LLM Integration: Utilizes a locally hosted LLM, eliminating the need for external API calls and reducing operational costs.
- Modular Agents: Employs specialized agents for different aspects of code review, each focusing on a specific area.
- CLI Interface: Offers a command-line interface for easy integration into existing workflows.
Example images of project in ACTION!
Folder Structure
The project is organized as follows:
github_repo/
├── main.py
├── cli.py
├── config.py
├── requirements.txt
├── README.md
├── llm/
│ ├── __init__.py
│ └── ollama_llm.py
├── agents/
│ ├── __init__.py
│ ├── base_review_agent.py
│ ├── code_quality_agent.py
│ ├── documentation_review_agent.py
│ ├── test_coverage_agent.py
│ └── dependency_review_agent.py
├── utils/
│ ├── __init__.py
│ ├── diff_parser.py
│ └── github_helper.py
├── prompts/
│ ├── __init__.py
│ └── prompt_templates.py
└── tests/
└── ... (testing scripts)
How It Works
1. LLM Integration
At the core of RBRDCK is the OllamaLLM class, which interfaces with a locally hosted LLM:
# llm/ollama_llm.py
class OllamaLLM:
def call(self, prompt: str) -> str:
# Sends the prompt to the local LLM and returns the response pass
2. Agents
RBRDCK uses specialized agents to perform different aspects of the code review:
CodeQualityAgent: Analyzes code for style, complexity, and potential issues. DocumentationReviewAgent: Ensures that code changes are properly documented. TestCoverageAgent: Checks if the code changes are adequately tested. DependencyReviewAgent: Reviews changes in dependencies for security and compatibility issues.
Each agent inherits from BaseReviewAgent and implements its specific review logic.
3. Review Orchestration
The ReviewOrchestrator coordinates the agents:
# agents/review_orchestrator.py
class ReviewOrchestrator:
def conduct_collaborative_review(self, context: ReviewContext) -> str:
# Orchestrates the review process among agents
pass
4. CLI Interface
The cli.py script provides a command-line interface to run the review process:
python cli.py username repo-name --pr 123
Getting Started
Prerequisites
Python 3.7+ A local LLM server running (e.g., Ollama) As of right now, will work with OPENAI implementation soon... GitHub access token with appropriate permissions
Installation
Clone the Repository
git clone https://github.com/cravinos/rbrdck.git
Install Dependencies
pip install -r requirements.txt
Configure Environment Variables
Create a .env file and set your GitHub token:
GITHUB_TOKEN=your_github_token_here
Usage
Run the CLI to review a specific pull request:
python cli.py username repo-name --pr 123
Future Development Plans
I am looking to develop this project further by integrating it with OpenAI SWARM and AutoGen v0.4.
OpenAI SWARM
OpenAI SWARM is a work-in-progress framework aimed at orchestrating multiple AI agents to perform complex tasks collaboratively. Although it currently has limited functionalities, it shows promise for managing interactions between different AI components.
AutoGen v0.4
AutoGen v0.4 is another framework that facilitates the creation of autonomous AI agents. It shares similarities with SWARM but is currently undergoing significant changes and has limited features.
Integration Goals
Enhanced Collaboration: Use SWARM or AutoGen to improve how agents communicate and collaborate during the review process. Scalability: Leverage these frameworks to handle larger projects and more complex reviews efficiently. Advanced Capabilities: Incorporate advanced reasoning and problem-solving features provided by these frameworks.