fyllyoWe tested 8 OCR readers on tax documents. One made up 73 numbers.
When software fills a tax form from a scanned statement or a phone photo of a pay stub, everything depends on reading the numbers right. A missed digit is bad. A number the reader made up is worse, because it looks real. We tested 8 readers that can run on a Mac to see which ones you can trust.
The results
Each reader read the same pages three ways: as a clean page, as a scan and as a phone photo. We counted how many of 578 known amounts, dates and ID numbers it got exactly right, and how many numbers it wrote that weren’t on the page at all.
| Reader | Kind | Clean | Scan | Photo | Made-up numbers | Seconds a page |
|---|---|---|---|---|---|---|
| LightOnOCR-2 (1B) | OCR model, open source | 100% | 100% | 100% | 0 | 12–17 |
| Nemotron Parse | OCR model, open source | 99.5% | 99% | 98.6% | 1 | 16–31 |
| GLM-OCR | OCR model, open source | 99.1% | 98.4% | 97.9% | 0 | 11–15 |
| Apple document reader | Built into macOS | 96.2% | 99% | 99.3% | 4 | 0.4 |
| Apple Vision, in strips | Built into macOS | 98.3% | 99.3% | 99.1% | 25 | 1–3 |
| Apple Vision, whole page | Built into macOS | 89.8% | 97.2% | 95.3% | 3 | 0.2 |
| PaddleOCR | OCR toolkit, open source | 86.7% | 66.8% | 77.3% | 1 | 8–10 |
| Gemma 4 E4B | General AI model reading images | 78.2% | 81.8% | 71.8% | 73 | 13–19 |
Made-up numbers are totalled across the three conditions. Times are on an Apple silicon Mac (M4 Max); the built-in Apple readers use the Mac’s own hardware and are far faster.
How we tested
- The pages: 17 pages of the paperwork tax forms are filled from: brokerage and bank statements, 1099s, a W-2c, a tax transcript, a prior return and letters. Some were real statements, which aren’t shown here; the rest were made for test clients.
- Three conditions: each page as a clean image, as a simulated scan (a slight tilt, blur, grain and heavy JPEG compression) and as a simulated phone photo (an angled page, uneven light and blur).
- The answer key: 578 amounts, dates and ID numbers per condition, checked by hand. A number counts only when every digit is right.
- Made-up numbers: any amount in a reader’s output that doesn’t appear on the page.
Made-up numbers are the real danger
A general AI model reading images, Gemma 4 E4B, wrote 73 numbers that weren’t on the pages, and missed up to 28% of the real ones. They looked like the real thing: plausible amounts in the right places. That’s exactly the error you won’t catch when you check a filled form quickly.
The dedicated OCR models behaved differently. LightOnOCR-2 read every one of the 578 numbers right in all three conditions and made up none. GLM-OCR made up none either, but skipped some blocks of text entirely. Apple’s Vision, run on strips of the page, read more numbers than run on the whole page, but made up 25.
What we learned
- Use an OCR model, not a chat model, to read numbers. Ask a general AI model to read a statement and it can make up numbers. A dedicated OCR model rarely does.
- Small can be enough. The most accurate reader was a 1-billion-parameter model, about 1.1 GB, small enough to run on a laptop.
- Photos are where readers split. On clean pages most readers were fine. On phone photos, the spread ran from 71.8% to 100%.
- Two readers rarely agree on a wrong number. When two different readers produce the same number, it’s almost certainly right. When only one sees it, it’s worth a look.
You’re welcome to cite these results with a link to this page.
What Fyllyo uses now
Since Fyllyo 0.8.0, Fyllyo reads each page from its text layer when that holds the numbers, and from its image with LightOnOCR-2 when it doesn’t, as on scans and phone photos. Apple’s text recognition reads each image a second time, and an amount only one of them saw is pointed out for you to check. Both run on your Mac, so your documents are never uploaded.
That’s what lets Fyllyo fill a form from a photo of a pay stub or a scanned 1099. See how the AI form filler works.