
Analyzing X-ray images is increasingly supported by ML technologies. Additionally, the quality and the Level of Detail (LOD) of technologies such as X-rays have increased significantly since their discovery in 1895. Doctors’ letters are written in digital form and available for automated processing. Modern technologies, e.g., Machine Learning (ML) or big data, are employed to detect, e.g., tumors or regions of interest automatically, which can produce highly annotated multimedia assets. Medical image (and video) processing is important in modern healthcare. Viewer applications such as the Digital Imaging and COmmunications in Medicine (DICOM) Viewer provide a 3D experience for viewing medical images and extracting their relevant medical features, and hence offer innovative and accurate diagnostics.


The invention of Magnet Resonance Imaging (MRI) and Computer Tomography (CT) scans in the 1970s directly led to the production of digital images of a physical structure such as the human body. Medical image processing has been of interest since Wilhelm Röntgen discovered X-rays in 1895. Thus, in this paper, we show how Graph Codes contribute new querying options for diagnosis and how Explainable Graph Codes can help to readily understand medical multimedia formats. Finally, the presentation of these new facilities in the form of explainability is introduced and demonstrated. On this basis, various metrics for the calculation of similarity, recommendations, and automated inferencing and reasoning can be applied supporting the field of diagnostics. These Graph Codes are transformed to feature relevant Graph Codes by employing a modified Term Frequency Inverse Document Frequency (TFIDF) algorithm, which further supports value ranges and Boolean operations required in the medical context. In this paper, we demonstrate how typical multimedia processing techniques can be extended and adapted to medical applications and how these applications benefit from employing a Multimedia Feature Graph (MMFG) and specialized, efficient indexing structures in the form of Graph Codes.

Multimedia Information Retrieval (MMIR) frameworks, like the Generic Multimedia Analysis Framework (GMAF), can contribute to solving this problem, when adapted to special requirements and modalities of medical applications.

In particular, the extraction of features from various different formats and their representation in a homogeneous way are areas of interest in medical applications. Due to the exponential growth of medical information in the form of, e.g., text, images, Electrocardiograms (ECGs), X-rays, and multimedia, the management of a patient’s data has become a huge challenge.
