AICoE Project

A Hierarchically Collaborative Autonomous AI Agent Framework for Multimodal Intelligent Pathology Analysis

Project Name

A Hierarchically Collaborative Autonomous AI Agent Framework for Multimodal Intelligent Pathology Analysis

Project Goal

The overall goal of this project is to support the sustainable development of digital pathology and to establish a widely acceptable AI solution for computational pathology. Building on multimodal large language models with autonomous inference capabilities, the project will develop a Hierarchically Collaborative Autonomous AI Agent Framework for multimodal intelligent pathology analysis.  Within this framework, a central control agent will decompose pathology analysis into specialized sub-tasks and assign them to dedicated sub-agents responsible for extracting pathological features from key regions. These AI agents will learn from pathologists’ diagnostic behaviors and expertise. The central agent will then integrate multimodal information to deliver autonomous pathology analysis that aligns with clinical knowledge and diagnostic practices, thereby enhancing both the accuracy and trustworthiness of AI-assisted pathology. To further improve accessibility and scalability, the project will incorporate edge computing at microscope endpoints, reducing the infrastructure costs associated with digital pathology systems in medical institutions. In parallel, clinically driven diagnostic toolkits will be continuously developed, alongside comprehensive system security mechanisms to ensure safe and reliable operation. The project will also conduct field validation of the AI agent system in partner hospitals to promote the clinical implementation of digital pathology analysis systems and the promotion of precision medicine.


Project Description

Because pathological tissue slide images enable direct observation of human tissues at the cellular level, pathological diagnosis often serves as the final stage in clinical disease diagnosis. However, pathology images have enormous resolutions, and manual examination of lesion regions by pathologists is not only time-consuming but also heavily relies on their experience and subjective judgment. Furthermore, many quantitative indicators, such as cell count and tumor size, are difficult to calculate accurately manually. Although many pathology AI models have been developed in recent years, most existing pathological AI systems is passively activated by pathologists, and cannot provide the analysis results immediately. In addition, pathological scanners are expensive, and digital pathology images require enormous storage space. It is extremely costly to build a digital pathology system, and most physicians still rely on microscopes for examination and diagnosis. These two issues lead to the lack of fluency in current pathological AI systems, and hinder clinical adoption of pathological AI systems. To address these issues, this project proposes "A Hierarchically Collaborative Autonomous AI Agent Framework for Multimodal Intelligent Pathology Analysis." We propose to utilize the reasoning ability of multimodal large language models to integrate information from pathology images and reports, and build a highly resilient and autonomous AI agent system, endowing it with the ability to autonomously initiate and analyze pathology images. We will also develop model compression techniques to integrate the pathology AI models into microscopes with edge computing. This project aims to reduce hardware construction costs of pathology AI system as well as endow it with autonomous inference capabilities, in order to achieve sustainable development of computational pathology.