We review a broad range of phenomena in diffraction in the context of hadron-hadron, hadron-nucleus collisions and deep inelastic lepton-proton/nucleus scattering focusing on the interplay between the perturbative QCD and nonperturbative models. We discuss inclusive diffraction in DIS, phenomenology of dipole models, resummation and parton saturation at low x, hard diffractive production of vector mesons, inelastic diffraction in hadron-hadron scattering, formalism of color fluctuations, inclusive coherent and incoherent diffraction as well as soft and hard diffraction phenomena in hadron-hadron/nucleus and photon-nucleus collisions. For each topic we review key results from the past and present experiments including HERA and the LHC. Finally, we identify the remaining open questions, which could be addressed in the continuing experiments, in particular in photon-induced reactions at the LHC and the future Electron-Ion Collider (EIC) in the US, Large Hadron electron Collider (LHeC) and Future Circular Collider (FCC) at CERN.