We report direct imaging of standing waves of the nontrivial surface states of topological insulator Bi2Te3 using a scanning tunneling microscope. The interference fringes are caused by the scattering of the topological states off Ag impurities and step edges on the Bi2Te3(111) surface. By studying the voltage-dependent standing wave patterns, we determine the energy dispersion E(k), which confirms the Dirac cone structure of the topological states. We further show that, very different from the conventional surface states, backscattering of the topological states by nonmagnetic impurities is completely suppressed. The absence of backscattering is a spectacular manifestation of the time-reversal symmetry, which offers a direct proof of the topological nature of the surface states.

We investigate the interplay between the strong correlation and the spin-orbit coupling in the Kane-Mele-Hubbard model and obtain the qualitative phase diagram via the variational cluster approach. We identify, through an increase of the Hubbard U, the transition from the topological band insulator to either the spin liquid phase or the easy-plane antiferromagnetic insulating phase, depending on the strength of the spin-orbit coupling. A nontrivial evolution of the bulk bands in the topological quantum phase transition is also demonstrated.

We show that a one-dimensional tight-binding electron moving in a slowly varying potential, V, ='icos(an"), where n is the site index and 0 & v & 1, has a mobility edge in its spectrum provided that 2k is smaller than the total unperturbed bandwidth of the system. We study the nature of the localized and extended eigenstates of this system as a function of k and v. PACS numbers: 71.50.+t, 03.65.6e, 71.30.+h, 71.55.Jv V"=X cos(tran '), (2) where k, a, and v are positive numbers which completely define the tight-binding problem. For a rational and v an integer, we get back the periodic Bloch model, whereas it has been shown that for a irrational and v~2 , the pseudorandom tight-binding model defined by Eqs. (1) and (2) becomes equivalent to a corresponding random Anderson model with all the states localized and the localization length equal to that of the corresponding random model. For v=1 and a irrational, Eqs. (1) and(2) define Harper's equation with an incommensurate potential which has been studied extensively in the last few years. ' For this situation (v=1, a irrational) the model has either all extended or all localized states depending on whether X, is smaller or larger than 2, the k=2 case being the self-dual point discovered by Aubry ' where all the states are critical.We have studied the model defined by Eqs.(1) and(2) for arbitrary values of v in the range 0 & v & 2 and for arbitrary irrational a. In particular, we find that forThe motion of an electron in a one-dimensional lattice as described by the nearest-neighbor tight-binding equation &n+1+ &n -1+~n &n E&n is a paradigm in condensed-matter physics. In Eq. (1), u" is the amplitude of the wave function at the nth site, V" is the on-site diagonal potential, and E is the energy. For periodic V"Bloch's theorem gives extended band states. Specifically for V"=O, Eq. (1) is trivially solved with u"=uoe'", giving rise to the one-dimensional tight-binding energy band E=2cos8 with 0~0~tr. For random V", Eq. (1) is the Anderson model' with localized states as the only allowed solutions. It is well known that in one-dimensional systems with any diagonal randomness all states are localized and there is no mobility edge separating localized and extended states. There is a class of pseudorandom and incommensurate potentials lying in between the random Anderson inodel and the periodic Bloch model which have attracted a lot of recent attention. ' A very simple model to study such potentials is given by ' 0 & v & 1 there exists a mobility edge in this onedimensional problem provided that l & 2. Thus for 0& v& 1 (and for 1I. & 2) we find extended states in the middle of the band (~E~& 2 -X) and localized states at the band edge (2+X &~E~& 2 -A, ), with the mobility edge at F. , =+ (2 -1). For X & 2 all states are found to be localized. For v& 1 our results are consistent withThouless's recent finding that all states away from the exact band center are localized even though the localization length could be very large at the band center; in particular, Thouless showed...

We report the direct observation of Landau quantization in Bi2Se3 thin films by using a low-temperature scanning tunneling microscope. In particular, we discovered the zeroth Landau level, which is predicted to give rise to the half-quantized Hall effect for the topological surface states. The existence of the discrete Landau levels (LLs) and the suppression of LLs by surface impurities strongly support the 2D nature of the topological states. These observations may eventually lead to the realization of quantum Hall effect in topological insulators.