Manga109-v2026's Modern Update for a Classic Comic Understanding Dataset
Introduction
In this post, I review recent work from the University of Tokyo Aizawa-Yamasaki-Matusi Lab on updated annotations for their famous Manga109 dataset. Manga109 is one of the most cited works in the comic understanding literature and quite possibly the most cited single work in the space. Baek et al. return to the dataset with an introspective investigation of annotation problems in the original data. Addressing the issue, the authors present an updated version of the Manga109 dataset with roughly 29,000 updated annotations, comprising approximately 19.6% of all text annotations, that better align with modern OCR and multimodal understanding systems.
Baek et al. write that manga understanding systems have progressed quickly in recent years and now include translation, transcription, understanding and even manga-oriented MLLMs. Whereas the previous annotations were tailored to the Optical Character Recognition (OCR) systems and analysis apparatus circa 2017 and revised in 2020, modern systems are adversely affected by annotations in the previous Manga109 version. The annotations potentially obfuscate meaningful stylistic components in the manga pages or outright confound models' ability to draw inferences or learn specific tasks on the manga pages. Manga OCR and Machine Translation (MT) are severely affected in particular. OCR, a preliminary task in image-to-text manga translation pipelines, interpretation of speech balloons (also known as speech bubbles) is affected by under-segmented speech balloons while inaccurate dialogue transcriptions in the original Manga109 annotations directly affect MT models' ability to learn. Experiments confirm improvements in alignment through a 14.4-point increase in OCR performance.
The new dataset has been dubbed Manga109-v2026, and a preprint of the corresponding paper appeared on ArXiv in May 2026. On the Manga109 website, Baek et al. report that the paper was accepted to the Culture x AI workshop at the Forty-Third International Conference on Machine Learning (ICML26) in Seoul, South Korea.
Issue Detection
The paper begins with and centers on detection of issues in the annotations present in the previous version of Manga109. These problems drive forward the authors' contribution, and inferences drawn during the detection procedures serve as data on which to build new sets of annotations.
Mantra Inc. provides the authors with a proprietary OCR system whose details Baek et al. are not at liberty to discuss. It was developed for manga translation applications, but Baek et al. find that it better reflects how modern models interpret manga. Rather than treating its OCR results as new ground truth immediately, Baek et al. utilize discrepancies between the proprietary system's results and existing Manga109 annotations as a basis for determining how existing Manga109 annotations may confound downstream applications and evaluations.
They manually inspect discrepancies between proprietary OCR and existing annotations, arriving at a five-category taxonomy of annotation issues in the original Manga109 dataset.
Issue Types and Revision Procedures
From the Issue Detection procedure, Baek et al. identify five broad types of annotation issues. The table above provides the name, a short description and the issue, and the approximate number of times it is detected in the original Manga109 data, and the number of annotations changed in the new Manga109-v2026 to address the issue type.
| Type | Description | Approx. Instances |
|---|---|---|
| Under-Segmented Speech Balloons | Multiple segments of text within speech balloons are included in a single annotation. | 14900 |
| Inaccurate Transcriptions | Japanese characters in the original annotation do not match what is on the page. | 9200 |
| Overlapping Text and Onomatopoeia | Onomatopoeia are annotated as text and onomatopoeia, resulting in overlapping annotations for the same content. | 4300 |
| Missing Text Annotations | Text in the manga is missing annotation. | 800 |
| Overly Large Bounding Boxes | A bounding box encompasses multiple text instances and/or irrelevant content. | 50 |
We review in the subsections below the notes Baek et al. include in the paper about the issues they detect:
Under-Segmented Speech Balloons
The final issue type refers to cases in which multiple speech balloons are connected into a longer single, composite piece of dialogue and annotated with a single bounding box. Baek et al. characterize this as acceptable practice in previous stages of manga understanding research, but they say it is not acceptable according to modern OCR evaluation settings. Thus, modern OCR models detect the constituent texts as separate instances rather than pieces of a whole, composite text box.
To resolve issues of this type, Baek et al. again compare the proprietary OCR results to annotations in the original Manga109 dataset. Here, the authors look for instances where multiple OCR bounding boxes overlap with a single annotation bounding box. They manually verify each case and divide verified cases of Under-Segmented Speech Balloons into multiple regions corresponding to constituent speech balloons.
Inaccurate Transcriptions
These are cases where the Japanese characters in OCR output did not agree with existing text annotations. Two models, GPT-5 and Gemini 3 Flash, are called upon judge whether the proprietary OCR or original annotations were more appropriate for the text sample. In cases where both judges pick the same text, that text was chosen as the new annotation, but for 2,051 cases in which the models disagree, Baek et al. manually examine and verify the annotation. In total, 9,207 annotations are revised in this manner.
Overlapping Text and Onomatopoeia
Baek et al. (2022) update Manga109 with annotations for onomatopoeia, but these new annotations sometimes overlap with dialogue regions. The authors identify such overlap as a potential source of error in downstream tasks and name translation as a specific example. They caution that the onomatopoeia may be incorrectly treated as regular text thanks to these annotation issues despite their completely different stylistic and narrative roles. The issue is identified via the spatial coordinates of onomatopoeia and dialogue bounding boxes and resolved by removing the overlapping segments of dialogue boxes. The onomatopoeia text boxes are retained.
Missing Text Annotations
Whereas the Inaccurate Transcriptions type captures disagreement between proprietary OCR results and the original Manga109 annotations, Missing Text Annotation cases occur whenever the OCR outputs capture text in the manga pages that is not captured by existing annotations. Many of these cases are short expressions, but Baek et al. nonetheless acknowledge their effect on manga understanding systems. These are resolved through manual verification and addition of bounding boxes and annotation texts.
Overly Large Bounding Boxes
This rare type groups 50 cases in which text bounding boxes in the original Manga109 annotations overlap with other text bounding boxes. These cases are resolved through direct manual inspection, and the bounding boxes are divided into separate boxes for constituent text.
Manga109-v2026: Manga109 Updated for Modern AI
Now, we review the changes in Manga109-v2026 as compared to the original dataset and the positive impact of these changes that Baek et al. measure on OCR performance.
Changed Annotations
Baek et al. report that around 29,000 annotations from the original Manga109 dataset are revised in Manga109-v2026, including ~9,200 corrections to Inaccurate Transcription issues and, ostensibly, corrections to all identified instances of other issue types.
Experimental Verification
Baek et al. utilize the evaluation procedure established for the previous MangaOCR benchmark and evaluate the same OCR outputs using both the original Manga109 and new Manga109-v2026 annotations. The end-to-end H-mean score of the OCR model increases from 48.5 to 62.9 percent. The authors present this improvement in H-mean scores as evidence that the new annotations better align with modern OCR systems and show the importance of updating culturally grounded datasets.
Limitations
While we cannot argue that issues of the types Baek et al. identify should be ignored, there are limitations to the methods of identifying and addressing instances of these issue types.
Particularly, the authors rely on LLM-as-a-judge evaluation to identify instances of the Text Transcription type. If the issue is truly manifest, then the annotation text for a particular sample from a Manga109 page is inaccurate, but the authors ask two models whether the proprietary OCR output or the original data is a better annotation and go with the judges' response if they agree. Only when there is disagreement among the judges do Baek et al. manually verify updated annotations for this type of issue. We observe that 7,156 of the 9,207 revised Text Transcription annotations, roughly 77.7% of all revised annotations of this type and 24.7% of revised annotations overall, appear in Manga109-v2026 without human re-verification.
The utilization of OCR results from a proprietary method is also troublesome. Baek et al. are not able to disclose many particulars of the Mantra Inc. OCR system they use to derive OCR data from which their analysis of all issue types stems. This limitation affects the authors' analysis of all issue types and all changes in Manga109 annotations downstream. We cannot be sure from what Baek et al. are at liberty to say what kind of OCR architecture is at play or whether its training data are in-domain for detecting the annotation issues. Overall, we trust the judgement of the Aizawa-Yamasaki-Matsui Lab, but the broader research community is unable to reproduce the results and cannot see much of the information that must have justified the authors' reliance on the Mantra Inc. OCR system for their method.
While annotation changes for most types fall back on manual verification rather than the OCR output from that model, most of the new annotations responding to Text Transcription issues come from the proprietary outputs. New inaccuracies in the proprietary OCR may have been carried over to the new dataset during mitigation but only for Text Transcription issues. However, there is no guarantee that the Mantra Inc. system captured all or even an overwhelming majority of instances of each issue type. We should therefore view the new Manga109-v2026 dataset as a definite improvement over its successor but must move forward with the caveat that any of the issues identified by Baek et al. in the current work may still be present, albeit in reduced numbers.
Finally, the authors perform a short benchmarking analysis comparing performance on the new and old annotations, comparing Manga109-v2026 to the original Manga109, and use improvement with the new data as evidence that Manga109-v2026 is better aligned with new OCR models. While this could be the case, we would like to see a more thorough investigation eliminating potential confounding factors or alternative explanations to isolate the dataset used and alignment as the independent and dependent variables, respectively.
Future Work For the UF Data Studio
An update to Manga109 represents a monumental change to the data landscape of comic understanding research. On the one hand, we have the introduction of newly curated annotations for manga understanding tasks; but on the other, we perceive that much of the previous research in that direction has been called into question thanks to the issues Baek et al. find in the data.
Despite the limitations we identify, new research should put Manga109-v2026 into use immediately; and the UF Data Studio will follow its own advice on the matter. Even if the link between the new dataset and alignment with modern OCR systems warrants further verification, we can be sure the new dataset corrects at least 2500 transcription errors in the original dataset and substantially improves the quality of bounding box data to mitigate other issue types thanks to manual verification and adjustments from Baek et al. Rationally, there is every reason to choose Manga109-v2026 over the original version for new work with manga in machine learning, although we believe there is much room for additional experimental validation to more accurately characterize its improvement over its predecessor.
While the authors produce evidence from a benchmarking experiment, we have already shown that the procedure for mitigating Text Transcription issues leaves roughly 7,000 new annotations unverified by human annotators. As a result, an annotation study could address this gap in supervision by manually reviewing all of or this cohort of transcription problems. Other types are mitigated with explicit manual verification, so once these annotations are addressed, we perceive that all new annotations will have been manually verified by annotators.
Another, more longitudinal annotator study could see volunteer annotators review each of the updated annotations in Manga109-v2026 and compare them to the original annotations and determine which is more appropriate from their perspective. If the new dataset wins over the old one consistently, then we have evidence that the new annotations align better with human judgment.
A wealth of existing research cites the original dataset, totalling more than 2000 citations across various Manga109 papers, per the Manga109 website and as verified by this author through Google Scholar. We must ask what the effect of switching to the new dataset will be on previous methods for numerous downstream tasks that are trained on the original Manga109 dataset. Baek et al. take a step in this direction using the MangaOCR benchmark as a pilot experiment for the new annotations in the updated dataset. They show that performance increases when benchmarking with the new data, but we believe the true test of updated Manga109-v2026's alignment will be whether methods and pipelines learn tasks better when conditioned on the new data versus the original Manga109 data as measured by performance on the same out-of-domain test datasets.
Reviewing the manga understanding literature, we perceive that most of Manga109's utility for its users, comic understanding researchers, is in conditioning methods and pipelines for manga data. While the experiment in Baek et al. (2026) is a positive sign, researchers will be interested in whether training methods and pipelines with the new data will propagate better performance in downstream tasks on unseen data. The proposed experiment will provide researchers with exactly that signal.
