Cross-modal retrieval has recently drawn much attention due to the widespread existence of multimodal data. It takes one type of data as the query to retrieve relevant data objects of another type, and generally involves two basic problems: the measure of relevance and coupled feature selection. Most previous methods just focus on solving the first problem. In this paper, we aim to deal with both problems in a novel joint learning framework. To address the first problem, we learn projection matrices to map multimodal data into a common subspace, in which the similarity between different modalities of data can be measured. In the learning procedure, the l21-norm penalties are imposed on the projection matrices separately to solve the second problem, which selects relevant and discriminative features from different feature spaces simultaneously. A multimodal graph regularization term is further imposed on the projected data,which preserves the inter-modality and intra-modality similarity relationships.An iterative algorithm is presented to solve the proposed joint learning problem, along with its convergence analysis. Experimental results on cross-modal retrieval tasks demonstrate that the proposed method outperforms the state-of-the-art subspace approaches.

No abstract

In recent years, cross-modal retrieval has drawn much attention due to the rapid growth of multimodal data. It takes one type of data as the query to retrieve relevant data of another type. For example, a user can use a text to retrieve relevant pictures or videos. Since the query and its retrieved results can be of different modalities, how to measure the content similarity between different modalities of data remains a challenge. Various methods have been proposed to deal with such a problem. In this paper, we first review a number of representative methods for cross-modal retrieval and classify them into two main groups: 1) real-valued representation learning, and 2) binary representation learning. Real-valued representation learning methods aim to learn real-valued common representations for different modalities of data. To speed up the cross-modal retrieval, a number of binary representation learning methods are proposed to map different modalities of data into a common Hamming space. Then, we introduce several multimodal datasets in the community, and show the experimental results on two commonly used multimodal datasets. The comparison reveals the characteristic of different kinds of cross-modal retrieval methods, which is expected to benefit both practical applications and future research. Finally, we discuss open problems and future research directions.

Blocking energy metabolism of cancer cells and simultaneously stimulating the immune system to perform immune attack are significant for cancer treatment. However, how to potently deliver different drugs with these functions remains a challenge. Herein, we synthesized a nanoprodrug formed by a F127-coated drug dimer to inhibit glycolysis of cancer cells and alleviate the immunosuppressive microenvironment. The dimer was delicately constructed to connect lonidamine (LND) and NLG919 by a disulfide bond which can be cleaved by excess GSH to release two drugs. LND can decrease the expression of hexokinase II and destroy mitochondria to restrain glycolysis for energy supply. NLG919 can reduce the accumulation of kynurenine and the number of regulatory T cells, thus alleviating the immunosuppressive microenvironment. Notably, the consumption of GSH by disulfide bond increased the intracellular oxidative stress and triggered immunogenic cell death of cancer cells. This strategy can offer more possibilities to explore dimeric prodrugs for synergistic cancer therapy.