Agents & evaluation
Did the agent follow the procedure?
Making an agent’s procedural obligations explicit, so plausible answers cannot hide skipped checks or incorrect execution paths.
ContractEval ↓Preprint
Praphul Singh
Senior Principal Applied Scientist · Oracle Health AI
How can AI systems improve through experience while remaining understandable and reliable?
I study language models and AI agents through this question, working across agent evolution, reinforcement learning, recurrent and looped models, interpretability, and model editing.
At Oracle Health AI, my current focus is Evolve, a governed platform for the automatic evolution of clinical AI agents. My broader work connects research with clinical workflows and data infrastructure operating at the scale of millions of notes. Across Oracle, I have filed 19 U.S. patent applications.
Outside research, you’ll find me writing, playing cricket, singing, or at the gym. I also love being out in nature or simply spending a quiet evening gazing at the moon.
B.Tech. in Electrical Engineering, IIT Kanpur. Based in the San Francisco Bay Area.
Research highlights
All selected papers ↓Three perspectives on building AI systems we can understand and trust.
Model understanding
What changes when a model specializes?
Examining what a specialist model’s parameter update explains and what aggregate benchmark gains leave unresolved.
Beyond Endpoint Gains ↓Findings of EMNLP 2026
Speech & privacy
Can speech stay useful while protecting privacy?
Separating voice identity and spoken identifiers to examine the tradeoffs between speech privacy and clinical preservation.
AegisVox ↓ML4H 2026
News
- AegisVox accepted to Machine Learning for Health (ML4H) 2026.
- Promoted to Senior Principal Applied Scientist at Oracle Health AI.
- Beyond Endpoint Gains accepted to Findings of EMNLP 2026.
- New preprint: ContractEval, on procedural instruction conformance in AI agents.
- New preprints: OG-Rank and JEDA, on adaptive ranking and query-free retrieval.
- New preprint: Optimizing Long-Form Clinical Text Generation with Claim-Based Rewards.
- RedactOR accepted to the ACL 2025 Industry Track.
Selected papers
Full list on Google Scholar ↗Recent work and earlier research. My name is in bold.
2026
AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech
Separating voice identity and spoken identifiers to examine the tradeoffs between speech privacy and clinical preservation.
Paper ↗PDF ↗
Read summary
A factorized framework uses consistent surrogate voices and identifier replacements. Matched renderer comparisons examine replacement realization, speaker linkage, and clinical preservation separately, showing why stronger performance on one measure need not improve the others.
ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance
Making an agent’s procedural obligations explicit, so plausible answers cannot hide skipped checks or incorrect execution paths.
Beyond Endpoint Gains: A Weight-Delta Audit of Medical Specialization
Examining what a specialist model’s parameter update explains and what aggregate benchmark gains leave unresolved.
2025
OG-Rank: Learning to Rank Fast and Slow with Uncertainty and Reward-Trend Guided Adaptive Exploration
A fast scoring path with selective generation for ambiguous cases, allocating inference effort where it can help ranking.
JEDA: Query-Free Clinical Order Search from Ambient Dialogues
Learning embeddings that connect implicit conversational intent directly to canonical clinical orders.
Optimizing Long-Form Clinical Text Generation with Claim-Based Rewards
Connecting claim-level evaluation to reinforcement learning for more grounded and complete clinical text.
RedactOR: An LLM-Powered Framework for Automatic Clinical Data De-Identification
A multimodal de-identification system that combines routing and consistent substitutions to preserve the utility of clinical data.
2021
Mitigating Uncertainty of Classifier for Unsupervised Domain Adaptation
Studying feature distributions, prediction uncertainty, and activation mappings for adaptation across domains.
Selected writing
More on the blog ↗Notes on AI, building things, and being human.
AI & experience
How to Make Your Agents Dream
Turning an agent’s past work into useful memory through evidence, consolidation, and evaluation.
Research & practice
Stop Quoting Papers, Start Building Models
What working with real systems teaches us beyond reading and discussing papers.
Life & reflection
The Paradox of Becoming
Ambition, attachment, discipline, and learning to live with our own contradictions.
From research to deployed systems
My industry work brings questions of learning, evaluation, and reliability into clinical and enterprise systems.
Clinical data infrastructure
I lead Oracle Health’s data de-identification and secure storage platform, supporting millions of clinical notes, alongside clinical audio, across thousands of patients for AI training, analytics, and evaluation.
Clinical AI workflows
I’ve designed and built workflows spanning order entry, diagnosis association, search, and note generation. My current focus is how clinical agents can improve through governed evolution.
Enterprise search & language models
My work on Oracle OneSearch spanned enterprise search, custom language models, retrieval-augmented generation, document question answering, and SQL analysis.
Background
2019–present
Oracle
Senior Principal Applied Scientist (since October 2026)
Previously: applied science, data science, and engineering roles spanning OneSearch, My Oracle Support, and Oracle Health AI.
2015–2019
Indian Institute of Technology Kanpur
B.Tech., Electrical Engineering
Honors & academic ranks
- InventOR award · Oracle
- All India Rank 555 · JEE Advanced
- All India Rank 1,388 · JEE Main
- State Rank 15 · UPSEE