Cover crop residue left on the soil surface as organic mulch in no-tillage crop production provides several environmental benefits, including weed suppression. Thus, many farmers who use cover crops attempt to reduce the use of agricultural inputs, especially herbicides. Therefore, our objectives were to study the potential of different cover crop species to suppress weeds and produce an in situ organic mulch, and evaluate the effect of the organic mulch with and without spraying glyphosate on weed suppression for vegetable (tomato (Solanum lycopersicum L. and broccoli (Brassica oleracea L. var. botrytis) growth and yield. Five cover crop treatments (sunn hemp (Crotalaria juncea L.), jack bean [Canavalia ensiformis (L.) DC.], pearl millet [Pennisetum glaucum (L.) R. Br.], grain sorghum [Sorghum bicolor (L.) Moench ssp. bicolor] and a no-cover crop (control)) were used in the main plots; and spraying or no spraying glyphosate on the flattened cover crop in the sub plots of split-plot experimental design. Organic mulch from pearl millet, sorghum and sunn hemp resulted in lower weed biomass during the early season of both tomato and broccoli than jack bean and no-cover crop (control). Spraying glyphosate after roller crimping reduced weed biomass by 103 g m−2 and 20 g m−2 by 45 and 60 days after transplanting (DAT) of tomato, respectively and resulted in a better tomato yield compared to non spraying. Glyphosate reduced weed biomass by 110 g m−2 in the early season of broccoli (30 DAT), but did not affect yield. Terminating high biomass cover crops with a roller crimper is a promising technique for weed management in vegetable crops, which has the potential to reduce or even eliminate the need for herbicide.