Problem and solution
01/Problem
02/Solution
03/How it works
The system, end to end.
- 01next →
Render
PDF → page images at consistent DPI.
- 02next →
Detect
Layout regions + reading order.
- 03next →
Extract
VLM per region (text, tables, figures).
- 04next →
Reconcile
Stitch into schema, preserve links.
- 05loop
Export
JSON / Markdown / Postgres rows.
04/Architecture
Components and data flow.
How the moving parts fit together. Solid arrows are synchronous calls; dashed arrows are queued or batched.
- 01
Upload pipeline
- 02
PDF rasterizer
- 03
Layout detector
- 04
Region router
- 05
Vision-LLM pool
- 06
Reconciler
- 07
Schema validator
- 08
Export sinks
05/Technology
The stack.
- AI / ML
- GPT-4V / Claude visionLayoutLMv3Donut
- Frontend
- Reacttldraw viewer
- Backend
- PythonFastAPICelery
- Database
- PostgresS3
- Infrastructure
- ModalGitHub Actions
- APIs
- OpenAIAnthropic
06/Engineering decisions
What we picked, and why.
- 01
Region-by-region VLM calls
Choice
One VLM call per detected region
Alternatives
- Whole-page VLM
- Two-stage OCR + VLM
Why
Whole-page lost table structure; OCR+VLM doubled failure modes. Region routing gave the best recall on tables.
07/Evaluation
What we measured.
- 01
92%
Table accuracy
- 02
96%
Section recall
- 03
3.1s
p95 / page
08/Challenges
What almost broke us.
- 01
Multi-column layouts
What happened
Reading order broke on two-column papers.
What we did
Added a layout-aware ordering pass with explicit column boundaries.
09/Results
Replaced a 4-step manual pipeline for two analyst teams. ~6× throughput on document-heavy briefs.
10/Future improvements
- ›Diagram-level structural extraction
- ›OCR-free Chinese / Japanese
Have an interesting AI problem?
Let's build something.
If you're working on something that needs real AI engineering — not just a demo — let's talk.