METHOD AND SYSTEM FOR PROVIDING NON-VISUAL ACCESS TO GRAPHICAL ARTIFACTS AVAILABLE IN DIGITAL CONTENT

A method for providing non-visual access to graphical artifacts available in digital content includes classifying a graphical artifact into known and/or unknown categories using a deep neural network. The method further includes identifying semantically connected visual and textual components of the...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Palani, Hari Prasath, Thompson, Owen, Mukherjee, Joyeeta Mitra
Format: Patent
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A method for providing non-visual access to graphical artifacts available in digital content includes classifying a graphical artifact into known and/or unknown categories using a deep neural network. The method further includes identifying semantically connected visual and textual components of the graphical artifact, using a deep learning-based object detection model. Furthermore, the method includes extracting the visual and the textual components in a unified framework with predefined semantics associated with each component, using a pre-trained large multi-modal model fine-tuned to extract both the visual and the textual components from an image in the graphical artifact. The method further includes filtering out the predefined semantics through extraction and converting the predefined semantics into accessible representations. Also, the method includes delivering the accessible representations in conformance with requirements of a delivery system.