We study the ground states of cold atoms in the tight-binding bands built from p orbitals on a two dimensional honeycomb optical lattice. The band structure includes two completely flat bands. Exact many-body ground states with on-site repulsion can be found at low particle densities, for both fermions and bosons. We find crystalline order at n=1/6 with a sqrt[3] x sqrt[3] structure breaking a number of discrete lattice symmetries. In fermionic systems, if the repulsion is strong enough, we find the bonding strength becomes dimerized at n=1/2. Experimental signatures of crystalline order can be detected through the noise correlations in time of flight experiments.

We investigated the electron-pairing mechanism in an iron-based superconductor, iron selenide (FeSe), using scanning tunneling microscopy and spectroscopy. Tunneling conductance spectra of stoichiometric FeSe crystalline films in their superconducting state revealed evidence for a gap function with nodal lines. Electron pairing with twofold symmetry was demonstrated by direct imaging of quasiparticle excitations in the vicinity of magnetic vortex cores, Fe adatoms, and Se vacancies. The twofold pairing symmetry was further supported by the observation of striped electronic nanostructures in the slightly Se-doped samples. The anisotropy can be explained in terms of the orbital-dependent reconstruction of electronic structure in FeSe.

Following the recent observation of the quantum spin Hall (QSH) effect in HgTe quantum wells, an important issue is to understand the effect of impurities on transport in the QSH regime. Using linear response and renormalization group methods, we calculate the edge conductance of a QSH insulator as a function of temperature in the presence of a magnetic impurity. At high temperatures, Kondo and/or two-particle scattering give rise to a logarithmic temperature dependence. At low temperatures, for weak Coulomb interactions in the edge liquid, the conductance is restored to unitarity with unusual power laws characteristic of a "local helical liquid," while for strong interactions, transport proceeds by weak tunneling through the impurity where only half an electron charge is transferred in each tunneling event.

We study "frustrated" hopping models, in which at least one energy band, at the maximum or minimum of the spectrum, is dispersionless. The states of the flat band(s) can be represented in a basis which is fully localized, having support on a vanishing fraction of the system in the thermodynamic limit. In the majority of examples, a dispersive band touches the flat band(s) at a number of discrete points in momentum space. We demonstrate that this band touching is related to states which exhibit non-trivial topology in real space. Specifically, these states have support on one-dimensional loops which wind around the entire system (with periodic boundary conditions). A counting argument is given that determines, in each case, whether there is band touching or not, in precise correspondence to the result of straightforward diagonalization. When they are present, the topological structure protects the band touchings in the sense that they can only be removed by perturbations which also split the degeneracy of the flat band.

According to the "no-node" theorem, many-body ground state wavefunctions of conventional Bose-Einstein condensations (BEC) are positive-definite, thus time-reversal symmetry cannot be spontaneously broken. We find that multi-component bosons with spin-orbit coupling provide an unconventional type of BECs beyond this paradigm. We focus on the subtle case of isotropic Rashba spin-orbit coupling and the spin-independent interaction. In the limit of the weak confining potential, the condensate wavefunctions are frustrated at the Hartree-Fock level due to the degeneracy of the Rashba ring. Quantum zero-point energy selects the spin-spiral type condensate through the "orderfrom-disorder" mechanism. In a strong harmonic confining trap, the condensate spontaneously generates a half-quantum vortex combined with the skyrmion type of spin texture. In both cases, time-reversal symmetry is spontaneously broken. These phenomena can be realized in both cold atom systems with artificial spin-orbit couplings generated from atom-laser interactions and exciton condensates in semi-conductor systems.

We study the ground state properties of the interacting spinless fermions in the px,y-orbital bands in the two dimensional honeycomb optical lattice, which exhibit different novel features from those in the pz-orbital system of graphene. In addition to two dispersive bands with Dirac cones, the tightbinding band structure exhibits another two completely flat bands over the entire Brillouin zone. With the realistic sinusoidal optical potential, the flat bands acquire a finite but much smaller band width compared to the dispersive bands. The band flatness dramatically enhanced interaction effects giving rise to various charge and bond ordered states at commensurate fillings of n = i 6 (i = 1 ∼ 6). At n = 1 6 , the many-body ground states can be exactly solved as the close packed hexagon states which can be stabilized even in the weakly interacting regime. The dimerization of bonding strength occurs at both n = 1 2 and 5 6 , and the latter case is accompanied with the charge density wave of holes. The trimerization of bonding strength and charge inhomogeneity appear at n = 1 3 , 2 3 . These crystalline orders exhibit themselves in the noise correlations of the time of flight spectra.

We study the Fermi surface instabilities of the Pomeranchuk type in the spin triplet channel with high orbital partial waves (F a l (l > 0)). The ordered phases are classified into two classes, dubbed the α and β-phases by analogy to the superfluid 3 He-A and B-phases. The Fermi surfaces in the α-phases exhibit spontaneous anisotropic distortions, while those in the β-phases remain circular or spherical with topologically non-trivial spin configurations in momentum space. In the α-phase, the Goldstone modes in the density channel exhibit anisotropic overdamping. The Goldstone modes in the spin channel have nearly isotropic underdamped dispersion relation at small propagating wavevectors. Due to the coupling to the Goldstone modes, the spin wave spectrum develops resonance peaks in both the α and β-phases, which can be detected in inelastic neutron scattering experiments. In the p-wave channel β-phase, a chiral ground state inhomogeneity is spontaneously generated due to a Lifshitz-like instability in the originally nonchiral systems. Possible experiments to detect these phases are discussed.

We investigate the general structure of orbital exchange physics in Mott-insulating states of p-orbital systems in optical lattices. Orbital orders occur in both the triangular and kagome lattices. In contrast, orbital exchange in the honeycomb lattice is frustrated as described by a novel quantum 120 degrees model. Its classical ground states are mapped into configurations of the fully packed loop model with an extra U(1) rotation degree of freedom. Quantum orbital fluctuations select a six-site plaquette ground state ordering pattern in the semiclassical limit from the "order from disorder" mechanism. This effect arises from the appearance of a zero energy flat band of orbital excitations.