CodeScout — AI Codebase Intelligence & RAG
Index any public GitHub repository into Neon pgvector and ask plain-English questions backed by exact file and line-range citations.
Built and deployed a Retrieval-Augmented Generation (RAG) web application for developers that streams public GitHub repository tarballs, chunks source files into 40-line overlapping windows with 1-based line numbers, indexes 768-D embeddings in Neon pgvector, and answers questions strictly from retrieved code with exact file:startLine-endLine citations.
Operational Challenge
Developers onboarding onto unfamiliar open-source repositories or auditing team codebases waste hours manually grepping through folders to trace entry points, APIs, and data flows.
Intended Audience & Use Case
Software engineers, technical leads, and contributors who need fast, verifiable answers about unfamiliar GitHub repositories without reading every file manually.
System Design & Data Flow
Next.js and TypeScript RAG application that streams public GitHub repository tarballs in memory (skipping node_modules, lockfiles, images, and binaries), splits source files into 40-line overlapping windows with 1-based line numbers, stores 768-D embeddings in Neon PostgreSQL (pgvector), and retrieves top chunks by cosine similarity for grounded, line-cited answers.
Important Technical Decisions
- —Preserved exact 1-based startLine and endLine metadata on every 40-line overlapping code chunk so answers link directly to clickable path:start-end excerpts.
- —Streamed GitHub repository tarballs in memory and filtered out node_modules, lockfiles, images, and binaries to keep indexing fast and focused on source code.
- —Enforced strict grounded guardrails so the assistant refuses to speculate when the answer is not present in the indexed repository.
Features Implemented
- Streaming in-memory GitHub tarball ingestion skipping lockfiles, binaries, and node_modules
- 40-line overlapping vector chunks with 1-based line numbers in Neon pgvector (768-D)
- Zero-hallucination answers with clickable path:start-end citations and strict refusal guardrails
Testing & Engineering Notes
- —TypeScript strict typing across tarball ingestion, vector chunking, and RAG query APIs.
- —Cosine-similarity retrieval over 768-D pgvector embeddings with explicit refusal when no relevant code chunks match.