The silicon-based planar CMOS technology is expected to face fundamental limits in the near future, and therefore, new types of nanoscale devices are being investigated aggressively. To understand the device physics and assess their performance limits, new types of simulation techniques considering quantum mechanical phenomena, carrier's ballistic transport and atomistic effects are also needed. In this paper, we present our recent approaches toward the quantum and atomistic transport modeling, and make a performance projection of emerging nano-MOS transistors employing new device structures and new channel materials.