Paper Title
Multimodal Document Intelligence System using Vision Transformers and Large Language Models

Abstract
Organizations across industries generate a large volume of documents such as invoices, medical records, legal agreements, and financial reports. Extracting meaningful insights from these documents remains a challenging task due to their unstructured format, complex layouts, and varying content types. Traditional document processing systems rely heavily on rule-based approaches and simple Optical Character Recognition (OCR), which often fail to capture contextual relationships between textual and visual elements. This research proposes a Multimodal Document Intelligence System that integrates Vision Transformers, Optical Character Recognition (OCR), Natural Language Processing (NLP), and Large Language Models (LLMs) to automatically extract, analyze, and interpret information from document images and PDFs. The proposed system combines document layout understanding using LayoutLMv3, text extraction using PaddleOCR, semantic representation using Sentence Transformers, and knowledge retrieval using a vector database. A Retrieval-Augmented Generation (RAG) approach enables the system to answer user queries based on the extracted document content. Experimental evaluation demonstrates that the proposed system achieves high accuracy in document understanding tasks, improving information extraction and question-answering capabilities across multiple document datasets. Keywords - Document Intelligence, Vision Transformers, OCR, LayoutLM, Large Language Models, Retrieval-Augmented Generation