Case study / RAG + Evaluation

Retrieval that can explain
where it failed.

I built a TypeScript retrieval-augmented generation system that separates retrieval quality from answer quality, then exposes the same pipeline through the Ask Bucky portfolio assistant.

Try live demo Selected work

01Documents

02Chunk

03Embed

04Vector store

05Retrieve

06Generate

07Evaluate

Engineering / 01

What I built.

01Chunking: Configurable chunk size, overlap, sentence boundaries, and metadata.
02Retrieval: One query embedding, cosine similarity, Top-K ranking, and a relevance threshold.
03Grounding: Only selected profile evidence is assembled into the generation context.
04Evaluation: Precision@K and Recall@K for retrieval; Faithfulness and Answer Relevance for generation.

Evaluation / 02

Diagnose the layer, not just the answer.

Retrieval Quality

Precision@K reveals noisy evidence. Recall@K reveals relevant evidence the retriever missed.

Answer Quality

Faithfulness catches unsupported claims. Answer Relevance checks whether the response addresses the question directly.

Bad answer
Check retrieval
Measure Precision / Recall
Fix retrieval or inspect Faithfulness / Relevance
Fix generation

Live system / 03

Try the retrieval loop.

Ask about Bucky's experience, projects, skills, or education. Successful answers show the profile sources used.

Retrieval system / Profile corpus

Ask Bucky AI

Public profile context

Ask about

Experience
AI Projects
Technical Skills
Education