Intelligent Transportation Systems (ITS) have a significant and promising role in the overall improvement of transport performances. However, in the absence of a set of key performance indicators (KPI), it is very difficult to evaluate ITS and the effects of their application objectively. Therefore, the first aim of the paper is to develop an appropriate KPIs for assessment of railway ITS and the monitoring of their impacts. Based on that, a tool for the evaluation of the importance of these indicators is proposed. For that purpose a set of twenty-four indicators was grouped by topics under the economic, social, and environmental dimensions of sustainability. The paper thoroughly describes their development and classification. An evaluation of indicators based on expert opinions via the Group Analytic Hierarchy Process (GAHP) method is presented. Results obtained by the GAHP method illustrate the importance or significance of indicators for railway ITS.