Multimodal Knowledge Systems: Construction and Reasoning

This book focuses on advancing the integration of multimodal data (text, images, and structured knowledge) to enable precise knowledge extraction and human-like reasoning. The book’s primary objective is to address critical challenges such as modality gaps, semantic misalignment, dataset biases, and static reasoning paradigms. By introducing novel frameworks that unify graph-based learning, hierarchical representation, bias mitigation, and iterative refinement, this book provides systematic solutions to build robust, interpretable, and scalable AI systems. This book addresses gaps caused by incomplete textual semantics, spurious correlations across modalities, and inflexible reasoning pipelines by offering three pivotal contributions. First, the authors offer theoretical innovations in graph alignment techniques, hierarchical learning paradigms, and multi-agent reasoning frameworks. Then, the book goes on to offer practical tools including benchmark datasets, reproducible methodologies, and applications validated on state-of-the-art tasks. Finally, the book offers a broader impact through solutions tailored for low-resource settings, ethical considerations in AI deployment, and integration with emerging technologies like large foundation models. By bridging the divide between theoretical advancements and real-world applicability, the book serves as an essential resource for researchers and practitioners aiming to leverage multimodal data effectively, ethically, and at scale.

• Focuses on the integration of multimodal data to enable precise knowledge extraction and human-like reasoning;
• Covers challenges presented by graph-based alignment, hierarchical learning, and multi-agent debate frameworks;
• Offers tools such as bias-mitigated datasets and ethical deployment guidelines for scalable, interpretable AI.

Dr. Changmeng Zheng received his Bachelor’s and Master’s degree from South China University of Technology (Guangzhou China), and the PhD degree from the Hong Kong Polytechnic University (Hong Kong SAR). He is currently a Research Assistant Professor with the department of computing, the Hong Kong Polytechnic University, Hong Kong SAR. His research work has been published in refereed journals and conferences such as IEEE Transactions on Multimedia, IEEE Transactions on Circuits and Systems for Video Technology, Neural Networks, ACL, EMNLP, COLING, ACM MM. His research interests are in the areas of multimodal learning and social media analytics, especially the knowledge graph and large language models.

Prof. Qing Li received his BEng. Degree from Hunan University (Changsha, China), MSc and PhD degrees from the University of Southern California (Los Angeles, USA), all in computer science. He is currently a Chair Professor at the Hong Kong Polytechnic University, a visiting professor of the Zhejiang University, a guest professor of the University of Science and Technology of China, and an adjunct professor of the Hunan University. His research interests include multi-modal data modeling, multimedia retrieval and management, and e-learning systems. Dr. Li has published over 500 papers in technical journals and international conferences in these areas, and is actively involved in the research community by serving as a journal reviewer, programme committee chair/co-chair, and as an organizer/co-organizer of several international conferences. Currently he serves as the Chairman of the Hong Kong Web Society, a councillor of the Database Society of Chinese Computer Federation, and a Steering Committee member of the international WISE Society. He is a Fellow of IEEE, AAIA, and IET.