Towards Automated Tissue Characterization Using Parallel Bag-of-Features Experts Dealing with Two-Photon Excitation Fluorescence and Second Harmonic Generation Microscopy Datasets

Label-free tissue imaging with Multiphoton Microscopy (MPM) techniques such as Two-Photon Excitation Fluorescence Microscopy (TPEF) or Second Harmonic Generation Microscopy (SHG) can provide cues of similar pathologic relevance to the information collected for characterization / confirmation purposes with traditional histopathology protocols based on fixation and staining. To date, various approaches for the automated characterization of MPM datasets have been proposed, but usually these address the outputs of a single technique, while MPM imaging sessions can simultaneously yield multiple information categories associated to distinct modalities. We discuss here the main concepts of the Bag-of-Features (BoF) paradigm used for automated image classification and retrieval, and a series of past experiments that dealt with BoF based tissue characterization. Further on, we present some of our connected efforts placed on the problem of human epithelial tissue characterization with TPEF & SHG and introduce a concept for MPM data classification based on multiple BoF experts that utilize in a parallel manner complementary information categories.