README

A pure PHP OCR engine for clean rendered text bitmaps, such as the images of Blu-ray and DVD subtitles. It is a port of the nOCR engine from Subtitle Edit by Nikolaj Olsson.

The engine cuts an image into lines and glyphs and compares each glyph with a database of known glyphs. It reads text that a computer drew, in a font that the database knows. It does not read photos, scans or handwriting.

Install

Needs PHP 8.2 or later with ext-zlib. ext-gd is optional and only needed for Image::fromGd().

composer require yama6a/php-glyph-ocr

Usage

use GlyphOcr\GlyphDatabase;
use GlyphOcr\Image;
use GlyphOcr\Recognizer;

$recognizer = new Recognizer(GlyphDatabase::subtitleFonts());
$result = $recognizer->recognize(Image::fromPng(file_get_contents('subtitle.png')));

$result->text();                    // all lines, joined with "\n"
$result->confidence();              // mean confidence of all glyphs, from 0 to 1
$result->lines[0]->text;            // the first line
$result->lines[0]->confidence;      // mean confidence of the glyphs on the first line
$result->lines[0]->chars[0];        // RecognizedChar: text, confidence, italic, glyph, sample

• Images: Image::fromPng($bytes) decodes every PNG colour type without GD. Image::fromGd($gdImage) copies a GD image. Image::fromRgba($pixels, $width, $height) takes raw RGBA bytes or a list of integers.
• Confidence: the share of the database glyph's line points that agree with the image, from 0 to 1. An unknown glyph reads as * with confidence 0.
• Ink: a pixel is ink when the sum of its premultiplied red, green and blue is at least inkThreshold (200). So white or yellow text counts and a black outline does not.
• State: a recognizer learns the glyph heights from the images it reads and uses them to split the next images. Use one recognizer per subtitle stream, or call reset().

Databases and training

GlyphDatabase reads and writes the .nocr files of Subtitle Edit, version 1 and 2. The package ships two databases:

• subtitleFonts(): glyphs trained on DejaVu Sans, Liberation Sans and Noto Sans, upright and italic, at 28 and 48 px, followed by the Latin database for other fonts. Liberation Sans has the metrics of Arial. Use this database for subtitles.
• latin(): the Latin database of Subtitle Edit, unchanged. Use it to get the same result as Subtitle Edit.
• Other scripts and other fonts need their own database.

tests/fixtures/generator/train.php writes resources/SubtitleFonts.nocr again. The fonts and their licenses are in tests/fixtures/generator/fonts. The database holds line segments measured on rendered glyphs, and no font outlines.

Train a glyph from a sample that a person confirmed. The new glyph goes first, so it wins over older glyphs that match equally well:

use GlyphOcr\Trainer;

$database = GlyphDatabase::fromFile('my-font.nocr');
$trainer = new Trainer();
foreach ($recognizer->recognize($image)->unknownChars() as $char) {
    $database->add($trainer->train($char->sample, askAPerson($char->sample->toAscii())));
}
$database->save('my-font.nocr');

• Recognizer::split($image) returns the glyph samples of each line without matching them, with null for a space.
• GlyphSample::merge([$left, $right]) joins the parts of a character that the splitter cuts apart, such as " or %.
• The trainer draws random line segments with a seeded generator, so the same sample and seed give the same glyph.

Accuracy

The golden images in tests/fixtures are white or yellow subtitles, 20 to 60 pixels, 1 or 2 lines. tests/fixtures/SOURCES.md describes them. One recognizer reads each set in order.

Character accuracy, with the lines read without error in brackets:

• Character accuracy is 1 minus the edit distance divided by the length of the expected text.
• The Latin database has no glyphs of these fonts. The numbers show how it does on fonts it has not seen. The subtitle fonts database has no glyphs of Open Sans.
• "After training" adds to the Latin database one trained glyph per character for each font, size and style: the alphabet, digits and .,!?'-:, drawn on a separate image. A new recognizer reads each image.
• With lineContext: false, the Latin database reads 80.3% of the characters. Then 37 of the 269 capital I and lower case l in the golden texts come out as the other letter. The subtitle fonts database reads 91.2%, and 63 come out as the other letter. With lineContext: true, none do.
• With italicSlant: 0.2, the italic set reads 96.0% of the characters.
• With the same database and lineContext: false, the port reads every golden image exactly as Subtitle Edit does. SubtitleEditParityTest checks this.

Speed

Mean time per golden image, one image of 1 or 2 lines, after the database is loaded:

A full 1920x1080 frame with the same 2 lines takes about 1 second and 103 MB, so crop the image to the text where you can. A glyph that matches nothing is the slow case, because it runs through every match pass. So the subtitle fonts database is faster on the fonts it knows. Measured on one core of an x86_64 machine, without JIT.

Limits

• One text colour on a transparent or dark background. The ink threshold removes the outline, so text with a light outline or a light background does not work.
• The matcher compares shapes. Most sans-serif fonts draw I and l as the same bar, and l is about 5% taller. The recognizer picks the letter from the tops of the capitals, the ascenders and the other bars on the line. A line without them falls back to the heights of earlier lines, then to the neighbouring letters.
• No dictionary and no language model. Subtitle Edit fixes common OCR errors in a separate step, which this package does not port.
• Italic text needs italic glyphs in the database, or italicSlant.

Exceptions

Every exception implements GlyphOcr\Exceptions\GlyphOcrException: InvalidArgumentException, InvalidImageException for a PNG that does not decode, and InvalidDatabaseException for a .nocr file that does not load.

Attribution

This package is a port of the nOCR engine from Subtitle Edit by Nikolaj Olsson, MIT license, at commit b8da12a. LICENSE keeps both copyright notices.

resources/SubtitleFonts.nocr is trained on renderings of these fonts: