We examine the feasibility of phytoremediation as an alternative strategy to limit the exposure of asbestos in site with asbestos-containing materials. We collected soils from four locations from two sites—one with naturally occurring asbestos, and another, a superfund site, where asbestos containing materials were disposed over decades—and performed ecotoxicology tests. We also performed two experiments with crop cultivar and two grasses from serpentine ecotype and cultivar to determined best choice for phytoremediation. Asbestos concentrations in different size fractions of soils varied by orders of magnitude. However, different asbestos concentrations had little effect on germination and root growth. Presence of co-contaminants such as heavy metals and lack of nutrients affected plant growth to different extents, indicating several of these limiting factors should be considered instead of the primary contaminant of concern. Crop cultivar survived on asbestos contaminated soil. Grasses from serpentine ecotype did not show higher biomass than the cultivar. Overall, these results showed that soil conditions play a critical role in screening different crop species for phytoremediation, and that asbestos concentration has limited to no effect on plant growth. Our study provided a framework for phytoremediation of asbestos-contaminated sites to limit long-term asbestos exposure.