Evaluating BERT's Encoding of Intrinsic Semantic Features of OCR'd Digital Library Collections

Ji, Meng; Hu, Yuerong; Worthey, Glen; Dubnicek, Ryan; Underwood, Ted; Downie, J. Stephen

doi:10.1109/jcdl52503.2021.00045

Cited by 5 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These state-of-the-art French pretrained language models have been fine-tuned on various data for text classification tasks, such as tweets classification [19] or clinical notes classification [4]. Finally, studies on automatic speech transcriptions [14] and digitized texts with optical character recognition (OCR) [18] analyzed the impact of noisy inputs on contextualized word embeddings. Indeed, erroneous data, such as ungrammatical sentences found in many exercise statements, may cause a decline in performance due to model resilience.…”

Section: Related Workmentioning

confidence: 99%

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

Lincker,

Guinaudeau,

Pons

et al. 2023

20th International Conference on Content-Based Multimedia Indexing

View full text Add to dashboard Cite

In order to foster inclusive education, automatic systems that can adapt textbooks to make them accessible to children with Developmental Coordination Disorder (DCD) are necessary. In this context, we propose a task to classify exercises according to their DCD adaptation type. We introduce a challenging exercise dataset extracted from French textbooks, with two major difficulties: limited and unbalanced, noisy data. To set a baseline on the dataset, we use state-of-the-art models combined through early and late fusion techniques to take advantage of text and vision/layout modalities. Our approach achieves an overall accuracy of 0.802. However, the experiments show the difficulty of the task, especially for minority classes, where the accuracy drops to 0.583. CCS CONCEPTS• Applied computing → Education; Interactive learning environments; • Computing methodologies → Natural language processing; Neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

Lincker,

Guinaudeau,

Pons

et al. 2023

20th International Conference on Content-Based Multimedia Indexing

View full text Add to dashboard Cite

show abstract

“…On the other hand, NLP is a branch of artificial intelligence that focuses on the interaction between computers and human language. It empowers computers to understand, interpret, and generate human language in a way that is both meaningful and contextually relevant [7]. In addition to facilitating human-computer interaction, NLP is essential for data analysis because it enables computers to interpret and extract information from massive amounts of unstructured text data.…”

Section: Introductionmentioning

confidence: 99%

VisFormers—Combining Vision and Transformers for Enhanced Complex Document Classification

Dutta,

Adhikary,

Dwivedi

2024

MAKE

View full text Add to dashboard Cite

Complex documents have text, figures, tables, and other elements. The classification of scanned copies of different categories of complex documents like memos, newspapers, letters, and more is essential for rapid digitization. However, this task is very challenging as most scanned complex documents look similar. This is because all documents have similar colors of the page and letters, similar textures for all papers, and very few contrasting features. Several attempts have been made in the state of the art to classify complex documents; however, only a few of these works have addressed the classification of complex documents with similar features, and among these, the performances could be more satisfactory. To overcome this, this paper presents a method to use an optical character reader to extract the texts. It proposes a multi-headed model to combine vision-based transfer learning and natural-language-based Transformers within the same network for simultaneous training for different inputs and optimizers in specific parts of the network. A subset of the Ryers Vision Lab Complex Document Information Processing dataset containing 16 different document classes was used to evaluate the performances. The proposed multi-headed VisFormers network classified the documents with up to 94.2% accuracy, while a regular natural-language-processing-based Transformer network achieved 83%, and vision-based VGG19 transfer learning could achieve only up to 90% accuracy. The model deployment can help sort the scanned copies of various documents into different categories.

show abstract

Context-Relevant Denoising for Unsupervised Domain-Adapted Sentence Embeddings

Lowe,

Prusa,

Leevy

et al. 2024

2024 IEEE International Conference on Information Reuse and Integration for Data Science (IRI)

View full text Add to dashboard Cite

Evaluating BERT's Encoding of Intrinsic Semantic Features of OCR'd Digital Library Collections

Cited by 5 publications

References 3 publications

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

VisFormers—Combining Vision and Transformers for Enhanced Complex Document Classification

Context-Relevant Denoising for Unsupervised Domain-Adapted Sentence Embeddings

Contact Info

Product

Resources

About