Exploring Patents Visually: An Interactive Search System for Multimodal Patent Image Search and Interpretation

Abstract

Most patent retrieval systems are text-based, which under utilizes the multimodal nature of patent documents. Although a few multimodal patent retrieval systems exist, they fall short in providing efficient and informative visualizations, and facilitating the interpretation of retrieval results. To address these shortcomings, this paper presents iPatent, a novel web-based multimodal patent image retrieval system. Unlike previous solutions, iPatent integrates state-of-the-art deep learning models for fine-grained unimodal, cross-modal, and multimodal patent image retrieval. Additionally, it employs both traditional machine learning techniques and modern generative methods for interactive visual exploration and insightful interpretation of retrieval results. iPatent leverages modern web technologies to provide an interactive interface that enables users to explore large patent databases efficiently and in a visually informative way.

Cite

@inproceedings{awale2025ipatent,
  author = {Awale, Sushil and M{\"u}ller-Budack, Eric and Delaviz, Rahim and Ewerth, Ralph},
  title = {Exploring Patents Visually: An Interactive Search System for Multimodal Patent Image Search and Interpretation},
  booktitle = {Patent Text Mining and Semantic Technologies co-located with the {ACM} {SIGIR} Conference on Research and Development in Information Retrieval, {PatentSemTech}@{SIGIR} 2025, Padua, Italy, July 17, 2025},
  pages = {76--83},
  publisher = {CEUR-WS.org},
  year = {2025},
  url = {https://ceur-ws.org/Vol-4062/paper8.pdf},
  month = jul,
}