In Kondo insulator samarium hexaboride SmB6, strong correlation and band hybridization lead to an insulating gap and a diverging resistance at low temperature. The resistance divergence ends at about 5 Kelvin, a behavior recently demonstrated to arise from the surface conductance. However, questions remain whether and where a topological surface state exists. Quantum oscillations have not been observed to map the Fermi surface. We solve the problem by resolving the Landau Level quantization and Fermi surface topology using torque magnetometry. The observed Fermi surface suggests a two dimensional surface state on the (101) plane. Furthermore, the tracking of the Landau Levels in the infinite magnetic field limit points to -1/2, which indicates a 2D Dirac electronic state.The recent development of topological insulators is a triumph of single electron band theory [1][2][3][4][5][6][7][8] . It is interesting to understand whether similar exotic states of matter can arise once strong electronic interaction comes into play. Kondo insulators, a strongly-correlated heavyfermion system, offer a good playground for the exploration of this question. In a Kondo insulator 9,10 , the hybridization between itinerant electrons and localized orbitals opens a gap and makes the material insulating. Once the sample temperature is cold enough, the electronic structure in the strongly correlated system can be mapped to a rather simple electronic state that resembles a normal topological insulator 11 . As a result, in the ground state of the Kondo insulator there exists a bulk insulating state and a conductive surface state. In samarium hexaboride (SmB 6 ), the existence of the surface state has been suggested by recent experimental observations of the surface conductance as well as a map of the hybridization gap 12-14 . However, a direct observation of the Fermi surface has not yet been achieved by transport measurements in Kondo insulators. In this letter we report the observation of quantum oscillations in Kondo insulator SmB 6 using torque magnetometry. The observed Fermi surface is shown to be two-dimensional (2D) and arises from the crystalline (101) surface, and the Landau Level index plot shows a Berry phase contributed -1/2 factor in the infinite field limit, which indicates that this Fermi surface encloses Dirac points, a characteristic property of topological insulators.The direct observation of quantum oscillations is an essential step in understanding the electronic state of the bulk and surfaces of Kondo insulator. Wolgast et al. have argued strongly that the great robustness and certain other properties of the low T surface conductivity of SmB 6 are best understood as a consequence of having TI surface states 12 . Nonetheless there is yet no direct evidence for this interpretation of the surface conduction. Such evidence should come from microscopic measurements of the electronic structure, as has been accomplished for the weakly correlated TI materials, such as Bi 2 Se 3 , Bi 2 Te 3 , and graphene [15][16][17][18][19...

The search for unconventional superconductivity has been focused on materials with strong spin-orbit coupling and unique crystal lattices. Doped bismuth selenide (Bi 2 Se 3 ) is a strong candidate, given the topological insulator nature of the parent compound and its triangular lattice. The coupling between the physical properties in the superconducting state and its underlying crystal symmetry is a crucial test for unconventional superconductivity. In this paper, we report direct evidence that the superconducting magnetic response couples strongly to the underlying trigonal crystal symmetry in the recently discovered superconductor with trigonal crystal structure, niobium (Nb)-doped Bi 2 Se 3 . As a result, the in-plane magnetic torque signal vanishes every 60°. More importantly, the superconducting hysteresis loop amplitude is enhanced along one preferred direction, spontaneously breaking the rotational symmetry. This observation indicates the presence of nematic order in the superconducting ground state of Nb-doped Bi 2 Se 3 .

In metals, orbital motions of conduction electrons on the Fermi surface are quantized in magnetic fields, which is manifested by quantum oscillations in electrical resistivity. This Landau quantization is generally absent in insulators. Here, we report a notable exception in an insulator-ytterbium dodecaboride (YbB). The resistivity of YbB, which is of a much larger magnitude than the resistivity in metals, exhibits distinct quantum oscillations. These unconventional oscillations arise from the insulating bulk, even though the temperature dependence of the oscillation amplitude follows the conventional Fermi liquid theory of metals with a large effective mass. Quantum oscillations in the magnetic torque are also observed, albeit with a lighter effective mass.

The recent conjecture of a topologically-protected surface state in SmB 6 and the verification of robust surface conduction below 4 K have prompted a large effort to understand the surface states.Conventional Hall transport measurements allow current to flow on all surfaces of a topological insulator, so such measurements are influenced by contributions from multiple surfaces of varying transport character. Instead, we study magnetotransport of SmB 6 using a Corbino geometry, which can directly measure the conductivity of a single, independent surface. Both (011) and (001) crystal surfaces show a strong negative magnetoresistance at all magnetic field angles measured.The (011) surface has a carrier mobility of 122 cm 2 /V·sec with a carrier density of 2.5×10 13 cm −2 , which are significantly smaller than indicated by Hall transport studies. This mobility value can explain a failure so far to observe Shubnikov-de Haas oscillations. Analysis of the angle-dependence of conductivity on the (011) surface suggests a combination of a field-dependent enhancement of the carrier density and a suppression of Kondo scattering from native oxide layer magnetic moments as the likely origin of the negative magnetoresistance. Our results also reveal a hysteretic behavior whose magnitude depends on the magnetic field sweep rate and temperature. Although this feature becomes smaller when the field sweep is slower, does not disappear or saturate during our slowest sweep-rate measurements, which is much slower than a typical magnetotransport trace.These observations cannot be explained by quantum interference corrections such as weak antilocalization, but are more likely due to an extrinsic magnetic effect such as the magnetocaloric effect or glassy ordering.

Topological insulator Bi2Se3 has shown a number of interesting physical properties. Doping Bi2Se3 with copper (Cu) or strontium (Sr) has been demonstrated to make the material superconducting and potentially even a topological superconductor. The recent discovery of superconducting niobium (Nb) doped Bi2Se3 reveals an exciting new physical phenomenon, the coexistence of superconductivity and magnetic ordering, as well as signatures of an odd-parity p-wave superconducting order. To understand this new phenomenon, a detailed knowledge of the electronic structure is needed. To date, the quantum oscillations needed to reveal this structure have not been reported in Nb-doped Bi2Se3. In this rapid communication, we present the first observation of quantum oscillations in the magnetization (the de Haas-van Alphen effect) of Nb-doped Bi2Se3. In the fully superconducting crystal, two distinct orbits are observed, in sharp contrast to Bi2Se3, Cu-doped Bi2Se3, and Sr-doped Bi2Se3. The multiple frequencies observed in our quantum oscillations, combined with our electrical transport studies, indicate the multi-orbit nature of the electronic state of Nb-doped Bi2Se3.