The unique structures and properties of carbon nanotubes (CNTs) have attracted extensive investigations for many applications, such as those in the field of biomedical materials and devices, biosensors, drug delivery, and tissue engineering. Anticipated large-scale productions for numerous diversified applications of CNTs might adversely affect the environment and human health. For successful applications in the biomedical field, the issue of interfacing between CNTs and mammalian cells in vitro needs to be addressed before in vivo studies can be carried out systematically. We review the important studies pertaining to the internalization of CNTs into the cells and the culturing of cells on the CNT-based scaffold or support materials. The review will focus on the description of a variety of factors affecting CNT cytotoxicity: type of CNTs, impurities, lengths of CNTs, aspect ratios, dispersion, chemical modification, and assaying methods of cytotoxicity.