BuildMyRig AI

Picking PC parts means juggling CPU sockets, RAM generations, power supply wattage and how to split a budget between them. Users just want to say “$1,500, Cyberpunk at 1440p, black case” and get something they can buy.

My first version did what most AI builders do: one prompt with the parts catalog in context. It mostly worked, but sometimes it returned part IDs that didn't exist, paired an AM5 CPU with an LGA1700 motherboard, or went over budget. Each of those failures cost an OpenAI call and, worse, the user's trust.

The business depends on that trust: every recommended part carries an Amazon affiliate link, so a build that doesn't work is cost with no revenue.

The catalog already fit in the prompt, so missing knowledge wasn't the bottleneck. The model had no ground truth for combinatorial rules. So I made Postgres the source of truth and put three independent checks between the model and the user:

Retrieval has a narrower job: grounding the model's reasoning about games and performance, not picking parts. The knowledge base is about 120 chunks, one per game description, benchmark row or component spec, because each record is already a single, self-contained fact.

I used pgvector instead of a separate vector database: no new infrastructure, deleting a game removes its chunks in the same transaction, and a filtered similarity search (“top 5 benchmarks for Cyberpunk 2077”) is a single SQL query.

Every OpenAI call is traced to the database with tokens, USD cost, latency, retries and validation results, and an admin dashboard shows p50/p95 latency, daily cost and error rate per endpoint. Tracing can never take the app down: if writing a trace fails, the request still succeeds.

My first retrieval eval scored a perfect 100%, and that was the problem: its answer key had been generated from the retriever's own top results, so it could never fail. I rebuilt it with 10 hand-written queries whose correct answers came straight from SQL. The new score was lower, 95% MRR and 100% hit rate in the top 5, and that drop meant the eval could finally catch regressions.

It also surfaced a real failure: “physics puzzle game with portals and GLaDOS” ranked Portal above Portal 2. The fix is query expansion with release year and studio, then a re-ranker at larger scale.

On any saved build, users can chat to adjust it (“make it $200 cheaper without losing Cyberpunk performance”). It's a single agent with four tools that wrap code I already trusted: swap a component, check compatibility, estimate FPS and explain a choice. Tools never throw; errors come back as data the model can reason about.

It runs on hard limits: 5 tool calls, 20k tokens, 30 seconds and a $0.50 cost cap per conversation, with one history-compaction step when the context grows. I chose one agent over several because the workflow is linear; extra agents would add cost and fragility without adding anything.

I'd build the evals first. Adding them after the generator worked meant I was inferring quality from smoke tests instead of measuring it while tuning prompts. Next up: evals for recommendation quality (right parts, but is it a good build?), query expansion for ambiguous titles, and streaming responses for the chat.