In many fields of physics the discovery of an effective field theory represents a milestone of theoretical progress and sometimes allows analytical answers where otherwise only the brutal power of computers can find structure-less numerical hints. Let us mention nonlinear sigma models where nonlinearly realized symmetries [1] play a central role. Applications to slow dynamics in magnets, to the localizing effects of disorder in Anderson localization and, more recently, as a model for non-equilibrium dynamics of spin glasses [2] are well-known examples.In the field of spin glasses, Talagrand [3] proved recently that Parisi replica symmetry breaking [4] represents, as advocated since three decades, the exact structure of the basic SK model solution. Explicit solutions, which are necessary for applications in solid state physics at low temperatures, were however not yet available except in the vicinity of the critical temperature.Numerical methods combined with the renormalization group of De Dominicis [5], aiming at a relation between Parisi-and droplet-picture [6], appears to support the Parisi picture even for finite range interaction, and including low temperatures.Phenomenological theories which study the interplay between frustrated magnetism with other degrees of freedom, for example with electronic transport, exist [7], but systematic approaches were limited to the region of the spin glass freezing temperature, or neglected the details of Parisi RSB. As the literature shows, most applications were, probably due to the complication of the Parisi solution and the way how to include it, concerned with precursor effects in the nonmagnetic region above the critical temperature [8]. In the case of quantum phase transitions (driven by a non-thermal parameter) the nonmagnetic region can extend down to T = 0. Since the specific features of the frustrated order are not involved (only * e-mail: opperman@physik.uni-wuerzburg. This article reviews recent years' progress in the low temperature analysis of standard models of spin glass order such as the Sherrington-Kirkpatrick (SK) model. Applications to CdTe/CdMnTe layered systems and explanation of glassy antiferromagnetic order at lowest temperatures stimulated us to study in detail the beautifully complex physical effects of replica symmetry breaking (RSB). We discuss analytical ideas based on highly precise numerical data which lead to the construction of relatively simple effective field theories for the SK model and help to understand the mysterious features of its exact solution. The goal is to find construction principles for the theory of interplay between frustrated magnetic order and various relevant physical degrees of freedom. The emphasis in this article is on the role of Parisi's RSB, which surprisingly creates critical phenomena in the low temperature limit despite the absence of a standard phase transition.