On-device OCR reviewed — PaddleOCR vs Tesseract vs transformer OCR

Optical character recognition has quietly become something you can run without uploading a single byte. The models are small enough, the browser runtimes are mature enough, and the privacy argument is compelling enough that “send the scan to a server” is no longer the default for many jobs. This review compares the engines that matter for on-device work — the two classic pipelines, Tesseract and PaddleOCR, against the newer transformer-based approaches such as TrOCR and docTR — and tries to be honest about where each one earns its place.

The framing question throughout is not “which is most accurate” in the abstract. It is “which is most accurate per megabyte downloaded and per second of phone battery,” because that is the constraint that actually governs a local tool.

The two classic pipelines

Tesseract is the elder. Its current line (version 5) is built around an LSTM recognizer — a recurrent network that reads a normalised text line left to right and emits characters — and it retains a legacy pattern-matching engine for compatibility. You select between them with the OCR Engine Mode flag, where mode 1 is “neural nets LSTM only” and mode 3 is the default. Its real strength is breadth: official trained models cover more than 100 languages and 35-plus scripts, and the data files are independent, so you ship only the languages you need. The weakness is equally well known. Tesseract expects reasonably clean, deskewed, document-like input. It does not cope with curved text or strong perspective, and its accuracy on handwriting is poor — peer-reviewed comparisons routinely put it below 40 percent on cursive samples, against well above 90 percent on clean printed English.

PaddleOCR takes the modern detection-then-recognition route. The pipeline first locates text regions with a DBNet-style detector (Differentiable Binarization), optionally corrects orientation, then hands each crop to a recognition head. Across versions the recogniser has moved from a CRNN to a transformer-flavoured design (the SVTR family), and the current PP-OCRv5 release folds Simplified Chinese, Traditional Chinese, English, Japanese, and Pinyin into a single model, with multilingual coverage extending to roughly a hundred languages. The PaddleOCR 3.0 technical report puts PP-OCRv5’s overall weighted accuracy at 80.1 percent across a thirteen-category internal benchmark spanning handwriting, vertical text, and artistic fonts — up from 53 percent for the previous version. That benchmark is the vendor’s own, so read it as direction rather than gospel, but the architecture genuinely handles scene text and CJK far better than Tesseract does.

The structural difference matters for a browser. Tesseract is one engine doing line recognition; you give it a page and it does its own layout analysis. PaddleOCR is two (or three) models chained together, which gives you cleaner bounding boxes and better dense-document recall, at the cost of loading and orchestrating more pieces.

The transformer challengers

TrOCR, from Microsoft, is the purest expression of the “throw a transformer at it” idea. It is an encoder-decoder model: a Vision Transformer encoder (initialised from BEiT) reads image patches, and a text-transformer decoder (initialised from RoBERTa) generates characters autoregressively. The paper’s headline is that a standard, convolution-free architecture reaches state-of-the-art results without bespoke OCR machinery. In practice TrOCR is a recognition model, not a detector — it reads a single cropped line at a time. You still need something to find the lines first. Its decoder is also generative, which is where the cost lives: producing text token by token is slower than a CTC head that emits a whole line in one pass, and the base checkpoints are hundreds of megabytes before quantisation.

docTR, from Mindee, is the pragmatic middle path. It ships a model zoo rather than a single net — DBNet or LinkNet for detection, and a choice of CRNN, SAR, MASTER, or ViT-based recognisers (ViTSTR, PARSeq) — and lets you mix backbones to trade speed against accuracy. It is explicitly tuned for document images: scanned PDFs, forms, photographs of printed pages. Its closest cousin for local deployment is OnnxTR, which wraps the same models for ONNX Runtime, which is the form you actually want in a browser.

It is worth separating two claims that marketing tends to blur. Transformer OCR is not uniformly more accurate than the classic pipelines; it is more accurate on the hard cases — handwriting, unusual fonts, degraded scans — and often slower and heavier on the easy ones. For a crisp 300-DPI invoice in English, a CRNN will match a ViT decoder and finish in a fraction of the time.

How they actually run in the browser

This is the part that decides what ships. None of these engines run natively in a page; they run through one of two substrates.

The WebAssembly route compiles the engine to WASM and executes it on the CPU. Tesseract.js is the canonical example — a port that downloads a WASM core plus per-language traineddata and caches both in IndexedDB, running the work inside a Web Worker so the UI thread stays responsive. Robert Knight’s tesseract-wasm is a leaner build of the same idea, with SIMD acceleration where the browser supports it (Chrome 91+, Firefox 90+, Safari 16.4+). WASM is the compatibility floor: it works almost everywhere, single-threaded by default, and multi-threaded only when the page is cross-origin isolated via the COOP and COEP headers.

The ONNX Runtime Web route is how the neural pipelines and transformers reach the browser. Models are exported to ONNX and executed by ONNX Runtime Web, which can target WASM (broad compatibility) or WebGPU (GPU-accelerated, much faster, still experimental in places). PaddleOCR has both an official browser SDK and several community ports — ppu-paddle-ocr and paddleocr.js among them — that run the PP-OCRv5 family on ONNX Runtime and fall back from WebGPU to WASM silently if the GPU path is unavailable. Transformers.js takes the same approach for TrOCR: load the model, set device: 'webgpu', and inference moves to the GPU; otherwise it runs on WASM with an 8-bit quantised model by default. Quantisation is not optional housekeeping here — it is the difference between a download a phone will tolerate and one it will not.

Two practical frictions recur. WebGPU support is real but uneven, so any serious tool needs the WASM fallback wired up, not bolted on. And cross-origin isolation — the header dance that unlocks multi-threaded WASM and shared memory — is easy to forget and quietly halves your throughput when you do.

The verdict

There is no single winner, which is the honest answer and also the useful one.

For maximum language and script coverage on clean documents, with the smallest per-language footprint and the widest browser support, Tesseract (via tesseract-wasm or Tesseract.js) remains the sensible default. It is unmatched for “read this printed page in one of a hundred languages” and it degrades gracefully to ancient hardware.

For scene text, photographs, dense layouts, and especially CJK, PaddleOCR is the stronger pipeline. Its detector earns its keep on anything that is not a flat scan, and PP-OCRv5’s multilingual model is a genuine step up. Pay for it in orchestration complexity and a larger model bundle.

For handwriting and degraded or unusual text where the classic engines simply fail, the transformer recognisers — TrOCR for single-line recognition, docTR/OnnxTR for a configurable document pipeline — are worth the weight. Reserve them for the cases that justify a slower, heavier model and ideally a WebGPU device; do not reach for them to read a clean receipt.

My own bias, after staring at more misread serial numbers than I care to admit, is to start with the lightest engine that clears the accuracy bar and only escalate when the input genuinely demands it. A 500-megabyte transformer that reads a barcode label perfectly is still the wrong tool if a 10-megabyte CRNN reads it correctly in a tenth of the time. On-device OCR has reached the point where the interesting engineering is no longer “can it run locally” but “which of these four do I actually need” — and that question has a different answer for almost every job.