Highlights6
- Multi-tenant RAG assistant: led end-to-end development of an assistant used by 1,000+ faculty members to author courses reaching 60,000+ students.
- Knowledge graph: combined Neo4j with LLM-generated Cypher queries, making transcript analysis 16× faster by cutting it from 4 hours to 15 minutes.
- Retrieval quality: Prompt Flow evaluations for groundedness, relevance and coherence, hybrid vector plus keyword retrieval, and a citation on every answer.
- Tenant isolation: enforced at retrieval rather than in the prompt, with declarative per-tenant config, so onboarding a new college was a config change rather than a deployment.
- Assessment platform: REST APIs and SQL-backed admin surfaces for quiz platforms and question banks across ASU Online, with schema-level invariants for assessment integrity.
- Frontend engagement: responsive React and Material UI interfaces lifted engagement 35% and cut bounce 20%.
More detail
All 9 points
Retrieval quality
- Built the Prompt Flow evaluation harness that scores groundedness, relevance and coherence on a fixed question set, so a prompt change shipped on evidence rather than on how the first three answers felt.
- Measured chunking instead of assuming it: fixed-size splitting against section-aware splitting on real course documents, because a policy paragraph cut in half answers half a question.
- Ran hybrid retrieval, vector plus keyword, since course codes and policy numbers are exactly the tokens an embedding blurs and exactly what faculty search for.
- Surfaced a citation with every answer, linking the source document and section, so a faculty member can check a claim rather than trust it.
Multi-tenancy
- Enforced tenant isolation at retrieval rather than in the prompt, so one college's material cannot surface in another's answer even when the instruction is ignored.
- Kept per-tenant configuration declarative (corpus scope, model, prompt version), so onboarding a new college was a config change rather than a deployment.
The knowledge graph
- Constrained generated Cypher to a schema allow-list, so a model-written query cannot traverse outside the sanctioned subgraph or return a node nobody meant to expose.
Assessment platform
- Shipped the admin surfaces the instructional designers run on: bulk import, question reuse across banks, and a diff view before a bank is republished.
Automation and UI
- Authored Google Apps Script automation generating Drive folder/doc hierarchies from Sheets metadata, eliminating manual course-provisioning toil.
Tools
- Python
- LangChain
- OpenAI
- Prompt Flow
- Semantic Kernel
- Neo4j
- JavaScript
- Google Apps Script
- SQL
- Pandas