Edward Profio
AI/ML Product Leader & Technologist | Harvard University ’27 | Carnegie Mellon University ’25
AI/ML Product Leader & Technologist. At UW Health I launched Epic-integrated GenAI assistants that returned an estimated 42,000+ clinician hours in the first year of a 500-clinician pilot, then scaled organization-wide (2023–25). I’m now building Aurifex, an AI platform for hospital discharge planning, designed to run inside Epic. Earlier: two 0-to-1 founding product roles, at Gallify and CODAmarket (CODAworx).
Case studies
All projectsSide projects
How I build AI responsibly
Most of my AI work has been in healthcare, but I hold every AI and agentic product to the same three habits, from clinical agents in Epic to marketplace recommenders and a study app.
AI drafts, and a person decides
I design the controls on the same ticket as the feature, not after it. An agent that touches sensitive data gets only the access it needs, runs under its own identity, logs every tool call and can be paused without a code change. Nothing it drafts reaches a patient, a chart or a customer until a named person has reviewed it, and the person can always start from a blank page instead.
- UW Health: Azure OpenAI GPT-4-based agents in Epic drafted In Basket replies to patient messages, nurse shift summaries and routine documentation, and clinicians chose to start from the draft or from a blank reply before signing. Each agent ran under a least-privilege identity with audit logs, and HIPAA compliance and AI guardrails were worked through with the legal and compliance team.
- Aurifex Healthcare: GPT-4o agents draft inpatient rehab referrals and PM&R consults with a cited source for every statement. A case manager approves each referral and the attending signs each consult, so nothing is sent on the agent's say-so.
Every change passes an evaluation gate
Before I build, I write down what good looks like: the right tool, the right data, a clear stop and escalate when the agent is unsure, and no claims it can't support. I build the evaluation set from real cases, test for prompt injection and data leakage, compare results across user groups, and block any release that misses the bar. After launch I keep the same checks running, so drift shows up before users notice it.
- Aurifex Healthcare: Every change must pass a 1,200-case evaluation suite before it merges, including 40 of 40 prompt-injection tests and a ≤3-point accuracy gap across patient groups. In blinded review, 85% of drafted consults were rated acceptable, with a 1.5% hallucination rate per drafted statement.
- UW Health: Clinicians judged 93% of sampled drafts as doing the task, with a 2.5% hallucination rate per reviewed draft, and prompt-injection, leakage and subgroup tests ran before launch.
- Veritas: No question or feedback type ships until it passes a review set of 400+ real chapters across 12 subjects, and the product cleared every launch gate at 97% source-grounded answers and 99% answer-key accuracy.
I measure what changes for people, and report it to the people accountable
Before launch I pick the outcome that matters to the people using the product and to the business, put the analytics in place first, and name who owns the model afterward. Engagement on its own doesn't count as success. Each test has one primary metric and guardrails, and the results go to the people who have to act on them, in a form they can use.
- UW Health: A Databricks pipeline gave the AI steering committee governance and compliance metrics, such as how often clinicians sent a generated draft without editing it, and gave the product team performance metrics, such as how prompt changes affected usage, cutting iteration from months to days.
- CODAmarket: I set up analytics, funnels, cohort reports and NPS before changing anything, then ran 20+ A/B tests a year, each with one primary metric plus guardrails, which is how self-service onboarding was shown to double 90-day user retention.
Selected publications
Earlier life-science research.
