Skip to work
Dempsey Zhang

Full Stack AI Product Engineer, Toronto

Dempsey
Zhang

I build AI products people actually use, from the database schema to the agent prompt to the deploy pipeline.

Since January 2025 I have been building Addie, an AI college counseling platform used by 2,000+ students at 20+ schools. I merged 560+ pull requests there and built AI features end to end: a voice agent students talk to, guided AI conversations checked by an LLM judge, and meeting transcription. I start from evals, so I can show when an agent is actually better and not just better in a demo.

Download resumeEmail meGitHubLinkedIn
Portrait of Dempsey Zhang

Selected work at Addie

Four AI features I built at Addie, an AI college counseling platform. Each had a way to fail that a demo would not show, so I measured or guarded against it first.

Project lead, built end to end

Experiences: a voice agent students talk to

Students start a short voice session to log a campus visit or a college fair. The agent stays on goal, wraps up on time and saves structured notes to their profile.

The problem

Voice sessions lived inside counselor assigned conversations. A session a student starts on their own could not leak into chat lists, reminders or counselor dashboards, and speakerphone made the agent hear itself.

What I built

  • Its own session type. A purpose field picks the prompt and the extraction schema, and sessions live in their own table, so experiences stay out of reminder sweeps and conversation lists.
  • Paced by design. The agent wraps up at 5 minutes and stops at 10, and prompts can be edited without a deploy.
  • Feeds matching for free. Notes reach college matching as text on the student's card, so the matching engine needed no change.
  • Real WebRTC. As owner of browser voice, I moved it from a WebSocket relay to a real peer connection so echo cancellation works.

Result

Speakerphone calls went from the agent answering itself to clean turns, verified on a real iPhone, and the WebRTC move removed about 345 backend lines.

Built with OpenAI Realtime API, WebRTC, Next.js, FastAPI, Postgres

  1. Student's browser or phone

    Microphone with echo cancellation on

    WebRTC peer connection, audio both ways

  2. OpenAI Realtime

    Wraps up at 5 minutes, stops at 10

    Tool call

  3. Addie API

    • Guarded tool calls
    • Extraction scoped to the session's purpose
    Structured notes

  4. Notes on the student's profile

    Feeds Addie's context in later conversations

How a voice session runs. Simplified; no real student data.

Built the conversations, journeys and reminders; project lead for the evals

Guided AI conversations, checked by an LLM judge

Counselors design conversations that Addie runs with students over web, SMS, WhatsApp and voice. 13,000+ have been assigned.

The problem

Counselors needed Addie to follow their plan with each student, step by step and on time. Later, users reported conversations that repeated themselves, never closed, or claimed things students never said, and the only way to check was copying transcripts into other tools.

What I built

  • Builder and journeys. Structured and open questions, with journeys that unlock each conversation after the last one or on a set date.
  • Reminders with a kill switch. SMS, WhatsApp and email, fanned out in background jobs, with a global pause and a per school kill switch.
  • LLM as a judge. Every real conversation is scored for repetition, closure and made up claims, with the evidence quoted.
  • Calibrated before it runs. Admins hand label conversations, rubrics are rewritten until the judge agrees within one point, and auto scoring turns on only after calibration passes.

Result

Instead of copy and paste spot checks, every real conversation can be scored with evidence, and admins work from a Needs Review queue.

Built with OpenAI, Celery, Twilio, WhatsApp, Resend, Next.js, Evals

  1. Counselor designs a journey

    Conversations with structured and open questions

    Each step unlocks after the last, or on a date

  2. Addie runs the conversation

    Web, SMS, WhatsApp or voice

    Student goes quiet

  3. Reminders

    SMS, WhatsApp and email, with a global pause and per school kill switch

    Conversation ends

  4. LLM judge

    • Scores repetition, closure and made up claims
    • Quotes its evidence
    • Calibrated against admin labels before auto scoring
A conversation from plan to score. Simplified; no real student data.

I build the retrieval side, in progress

A counselor assistant that finds the facts

Counselors ask about any student and get sourced answers from meetings, notes and files.

The problem

The current assistant answers from a fixed dump of the student's record, about 40k tokens a turn. The plan was an agent that fetches only what it needs. Before shipping it I built a 70 question recall test with facts planted in synthetic students. The first retrieval design found almost none of them.

What I built

  • Layered context. Short records inline in the prompt, conversation digests, and originals fetched on demand.
  • Scoped tools. A LangGraph agent with 9 retrieval tools. The server binds whose data a tool can read; the model never names a student.
  • Chosen by a harness. The agent loop was picked by a live run of 3 designs, 12 scenarios and 3 samples each.
  • Every turn recorded. Context and tool calls are stored with each answer, so the eval track can score it.

Result

The redesign lifted recall from 3% to 43% of planted facts using a third of the tokens, and answers start in 1.8 seconds instead of 5.3. It stays behind a flag while the next step closes the gap to the current assistant's 67%.

Built with LangGraph, Python, FastAPI, SSE, Postgres, Evals

Planted facts found

Same 70 questions, synthetic students, real model

Retrieval agent, first design
3%
Retrieval agent, redesigned
43%
Current assistant
67%

Tokens per turn

Lower is cheaper

Retrieval agent, first design
18.6k
Retrieval agent, redesigned
12.1k
Current assistant
39.6k
Recall test I wrote before building the agent: 70 questions, each with a fact planted in a synthetic student's record.

Project lead, built end to end

Meeting transcription counselors can trust

Counselors record or upload an in person meeting and get a speaker labeled transcript and a summary that becomes part of the student's record.

The problem

Recording students has legal weight. Washington and California require every person in the room to consent, so the feature could not roll out to more schools without a consent flow built into it.

What I built

  • Vendor test first. AssemblyAI transcribed 115 minutes of audio in 97 seconds at 99.4% accuracy, for about half the cost.
  • Consent before the microphone. The counselor names who is in the room, reads a consent script, and every person's consent is recorded.
  • Background pipeline. Diarization runs in a background job with retries; counselors label speakers, then an LLM writes the summary from the labeled transcript.
  • Part of the record. Summaries flow into the student's notes and into the counselor chat's context.

Result

Live at 5 schools with 44 meetings transcribed so far, and full pipeline speaker accuracy of 99.8%.

Built with AssemblyAI, Celery, FastAPI, Next.js, Prisma, GCS

  1. Counselor starts a meeting

    Records in the browser or uploads a file

    Before the microphone opens

  2. Consent

    • Everyone in the room is named
    • Consent script read aloud
    • Each person's consent recorded with the notice version
    Audio to a private bucket

  3. Background transcription

    AssemblyAI diarization, retried on failure

    Counselor labels the speakers

  4. Summary from the labeled transcript

    Saved to the student's notes and the counselor chat's context

How a meeting becomes part of the record. Simplified; no real student data.

Also at Addie

Safer school reports

School admins ask a reporting agent questions in plain English. Before it opened to them I found its metrics only limit lived only in the prompt, so I added a hard column guardrail that keeps transcripts out, school scope taken from the signed in admin, and an audit entry for every query.

Deploys that tell the truth

Found deploys reporting success while running 3 day old code. Rebuilt the pipeline with verification, real rollback and alerts, and cut the slowest CI build from 13.7 to 6.6 minutes.

Security and student privacy

Fixed a critical admin login bypass, enforced school level data isolation (FERPA) with audit logs, and kept student PII out of error tracking.

Projects

OfferRing

A job search copilot I build on my own time and use every day.

A browser side panel and a dashboard for my own job search. It never submits an application.

Autofill
Fills standard fields on Greenhouse, Lever, Ashby, Workday and more, live field by field.
Grounded drafts
Drafts open question answers only from my fact bank, with numbers, employers and dates checked in code.
Ringo
An agentic RAG assistant over my postings, sent answers and facts, with every source cited.
Answer memory
Remembers every answer I submit and reuses it when the same question comes back.
Duplicate warning
Warns me before I apply to the same posting twice.
Daily ranking
Scores new postings from companies I watch and public job feeds from 1 to 5 against my profile.

How it stays honest

  • Hybrid retrieval. Ringo searches by meaning and by words (pgvector embeddings and Postgres full text, merged by reciprocal rank fusion) and uses structured tools for dates, counts and statuses. The agent picks the tool and rewrites its query when a search misses.
  • Sensitive answers stay mine. Work authorization, sponsorship and EEO questions are never decided by a model; they come only from stored answers.
  • No scraping. Job discovery uses public ATS APIs and official search APIs, and nothing is automated on LinkedIn.

Built with WXT, React, Next.js, Prisma, Postgres, pgvector, Voyage embeddings, Claude Haiku, Evals

Private deployment for my own job search; the source is not public yet.

Answer quality, first eval run and nowFictional postings and questions, scored in code and by a calibrated judge
MetricFirstNow
Groundedness per claim82.2%95.5%
Relevance to the posting (1 to 5)3.874.10
Form questions answered right40%100%
The first answer eval failed three thresholds and showed why: drafts credited one fact's result to another. Fixing the prompt and the checks brought every threshold to pass. Form answers compare rules alone with rules plus the model, on 15 fictional questions, with zero wrong answers in both. Ringo's eval answers 22 of 22 questions with the right facts and citations, including paraphrases and a Chinese question over English text, at under a tenth of a cent each.

Recorded with a fictional applicant and demo data.

Autofill. One click fills a real Greenhouse application, live field by field. Work authorization and sponsorship come only from stored answers, and questions it cannot answer truthfully are left for me. It never submits.

More projects

  • TabTidy

    Chrome extension to search, group, save and restore tabs. Published on the Chrome Web Store.

    Chrome Web StoreSource

  • The Wild Oasis

    Cabin booking site with Google sign in and a date range picker. Next.js, NextAuth, Supabase.

    LiveSource

  • Forkify

    Recipe search with saved favourites, in plain JavaScript.

    LiveSource

Experience

  1. Jan 2025 to now

    Addie

    Full Stack Developer (AI Product), Remote, Toronto

    AI college counseling platform for 2,000+ students and 70+ active counselors at 20+ schools. 560+ merged pull requests across a Python backend and three Next.js apps. Project lead for meeting transcription, Experiences, the conversation quality eval system, the student and counselor UI revamps and the Turborepo monorepo migration. Top contributor to the repo from July to October 2026.

  2. Sep 2024 to Jan 2025

    Vosyn.ai

    Full Stack Engineer Intern, Remote, Toronto

    Led the homepage in React. Built Redux search filtering (25% faster) and moved search results to Next.js server rendering, cutting page load time by 30%.

  3. Four years

    Before engineering

    Tech recruiter

    Hired engineers for Huawei's IC Lab and for clients like ByteDance and Amazon. It taught me to start from what people actually need.

Education

  • 2018 to 2020

    University of Toronto, Master of Industrial Relations and Human Resources

  • 2014 to 2018

    University of British Columbia, Bachelor of Management

Skills

AI and LLM
Agents and tool calling, LangGraph, LangChain, OpenAI Realtime, WebRTC voice, Text to SQL, RAG with Qdrant, Evals and LLM as a judge, Guardrails, Tracing
Front end
TypeScript, React, Next.js App Router, Server Actions, Tailwind CSS, shadcn/ui, Radix UI, Redux
Back end
Python, FastAPI, Celery, Redis, Node.js, Twilio, REST
Data and infra
PostgreSQL, Prisma, Qdrant, Docker Swarm, GitHub Actions, Turborepo, GCP, Sentry, Vercel
Testing
Jest, React Testing Library, Pytest, Playwright
How I work
Claude Code, Coding agent skills for the team, Feature flags, Evals before launch

Get in touch

I am glad to talk about AI products, evals, or a role you are hiring for.

zsg258852@gmail.com

LinkedInGitHubResume (PDF)