An agentic proposal engine with its own evaluation layer
Built a multi-stage LLM system that drafts proposals from structured evidence and scores its own output, and validated the grader before trusting it.
- rubric dimensions in the self-score
- 8
- strictness control on the validated autograder
- 0–10
- Organization
- Frontier Design LLC
- My role
- Senior Director of Artificial Intelligence
- Period · Focus
- Jan 2026 – Jun 2026 · Platforms, Governance
In brief
Proposal quality at Frontier depended on people remembering past performance. I built a structured past-performance knowledge base and a Claude-based engine that parses it, deep-reads source documents, researches the target, runs a realism check, drafts in the firm's voice, and self-scores against an eight-dimension rubric. I designed and validated the LLM autograder behind that score.
Context and challenge
Proposals, concept notes, and two-pagers drew on past projects that lived mostly in people's memory. Generated drafts needed a way to be judged consistently before a human reviewed them.
My mandate and scope
- Architected and built the proposal and whitepaper engine on Claude.
- Built the past-performance knowledge base from scratch: project records in XML with richness scoring and structured narratives.
- Designed an LLM rubric autograder with confidence scoring, a 0 to 10 strictness control, length normalization, and negative constraints, and ran a validation study published as an IEEE-format analytic report.
- Led architecture and build of an AWS project database that replaced spreadsheet workflows, and stood up Salesforce with automated workflows.
- Standardized the firm's AI service offerings into productized packages with a repeatable structure for scoping and delivery.
Key decisions and why
- 1
Structure the evidence before generating anything
Encoding past performance as scored, structured records gave every draft a queryable evidence layer instead of relying on whatever a person remembered.
- 2
Check realism before drafting and score after
A realism check against the target and a rubric self-score put two gates around generation, so weak drafts were caught before review.
- 3
Validate the grader before trusting it
An autograder is itself a model. The validation study correlated scores across rubric categories before its scores were relied on.
What changed
- Proposal drafting moved from memory-dependent writing to a pipeline with an evidence layer and automated quality gates.
Outcome and evidence
Outcomes here are delivered milestones and operational results. No public metrics are available.
- Engine used to produce federal and commercial proposals, concept notes, and two-pagers.
- Validation study of the autograder published as an IEEE-format analytic report.
What to expect from me
When I put language models into a workflow, I build the evaluation layer alongside them and test the evaluator too.
Technical perspective
- Multi-stage pipeline: parse knowledge base, deep-read sources, research the target entity, realism check, draft, self-score.
- Autograder controls: confidence scoring, 0 to 10 strictness, length normalization, negative constraints.
Artifacts
Confidentiality: Client proposals are not published.