In recent years, deep learning (DL) is becoming an increasingly important tool for solving inverse scattering problems (ISPs). This paper reviews methods, promises, and pitfalls of deep learning as applied to ISPs. More specifically, we review several state-of-the-art methods of solving ISPs with DL, and we also offer some insights on how to combine neural networks with the knowledge of the underlying physics as well as traditional non-learning techniques. Despite the successes, DL also has its own challenges and limitations in solving ISPs. These fundamental questions are discussed, and possible suitable future research directions and countermeasures will be suggested.