We review the formation and evolution of compact binary stars consisting of white dwarfs (WDs), neutron stars (NSs), and black holes (BHs). Mergings of compact-star binaries are expected to be the most important sources for forthcoming gravitational-wave (GW) astronomy. In the first part of the review, we discuss observational manifestations of close binaries with NS and/or BH components and their merger rate, crucial points in the formation and evolution of compact stars in binary systems, including the treatment of the natal kicks, which NSs and BHs acquire during the core collapse of massive stars and the common envelope phase of binary evolution, which are most relevant to the merging rates of NS-NS, NS-BH and BH-BH binaries. The second part of the review is devoted mainly to the formation and evolution of binary WDs and their observational manifestations, including their role as progenitors of cosmologically-important thermonuclear SN Ia. We also consider AM CVn-stars, which are thought to be the best verification binary GW sources for future low-frequency GW space interferometers.