Goals
- Accessibility
— a digital edition of the codex, encoded in TEI and released under a
permissive MIT license, with a quick download link and this online
viewer.
- Transparency — the generation process is documented, with special regard to how AI was used.
- Reproducibility — the pipeline that produced the edition is published and configurable.
- Community — an unaffiliated, community-driven endeavor. See the contributing guide.
Method
The
edition is encoded with the Text Encoding Initiative (TEI), which
provides the framework for describing the manuscript in a
machine-readable way using XML. Recent advances in AI, combined with
existing open data, make this work feasible at scale: multi-modal models
draft the columnar layout of each folio side, which is then
cross-checked and corrected.
Each folio
side is one TEI document carrying the main biblical text only — the
ketiv consonantal text as written — with verse milestones marking where
each verse begins and samekh/pe paragraph-division markers inline.
Per-word biblical references are derived at index time, not stored per
word.
Data
The
biblical text in the edition was not transcribed directly from the
images. The text used is the Unicode/XML Leningrad Codex (UXLC)
maintained by Christopher V. Kimball and accessible on the tanach.us site. For this reason, the text in the edition may differ from the images in the manuscript. That is an intentional choice.
The text determined to be on each folio was derived from Ben Denckla's index of the Leningrad Codex.
The verse zones that mark where each verse sits on the facsimile image come from the Masoretica handwritten text recognition (HTR) of the Leningrad Codex by Idan Dershowitz.
Each folio page ends with a Credits section. It lists the people and data sources named in the respStmt elements of that folio's TEI header, including the contributors who encoded it.
Transparency & AI use
AI tools were integral to this project, used for:
- Generating the code in the repository
- Aligning the biblical text to the columns and lines of a folio
The alignment work is fully documented per folio:
- The facsimile image used (URL and sha256) is recorded in the TEI header
- The model and parameters are recorded in the TEI header
- The prompt is linked to a commit sha
The
pipeline and prompts are published in the repository, and the full
history is visible in git. Contributors were allowed to use any model.
Each folio page indicates which model was used for alignment.
Reproducibility
The
pipeline that generated the edition is published so future researchers
can re-run it and configure it to different models and inference
strategies — with the caveat that AI outputs are not always
reproducible. See thepipeline docs in the repository.
Reuse
The edition is MIT licensed. Browse it folio by folio, fetch raw TEI per folio at /edition/Fxxx[A|B].xml, read the manifest at/edition/index.xml, fetch an IIIF manifest with the verse zones per folio at /iiif/Fxxx[A|B].json, or grab the whole-edition zip. Agents should prefer the XML over scraping this site — see /llms.txt.
The Author
I'm
Charles Loder, the initial author of this project. You can find more
about my work and contact me through my social media profiles:
Posts
Stumble It!
