Drowsy driving has a strong influence on the road traffic safety. Relying on improvements of sensorial technologies, a multimodal approach can provide features that can be more effective in detecting the level of alertness of the drivers.In this paper, we analyze a multimodal alertness dataset that contains physiological, environmental, and vehicular features provided by Ford to determine the effect of following a multimodal approach compared to relying on single modalities. Moreover, we propose a cascaded system that uses sequential feature selection, time-series feature extraction, and decision fusion to capture discriminative patterns in the data. Our experimental results confirm the effectiveness of our system in improving alertness detection rates and provide guidelines of the specific modalities and approaches that can be used for improved alertness detection.