To improve ITS services and increase the success rate of their deployment, regular evaluation of these services is important. Based on the evaluation results more robust business models and effective policy incentives can be developed. The use of clear and quantifiable key performance indicators (KPIs) can support this process. Both deployment and benefit KPIs are relevant in this respect. The former measures the extent by which a service is successfully implemented, while the latter measures the extent by which the targeted objective of the service (e.g. increased transport safety) is realised. Which KPIs should be best used to maximise the effectiveness of the evaluation of ITS services? This has been studied by NEWBITS.
Based on a systematic literature review and an on-line stakeholder survey, the most relevant KPIs for measuring the success of ITS deployment have been assessed. With respect to deployment KPIs, an important finding of the NEWBITS study was that no universal list of KPIs that can be applied for each ITS project can be identified. Instead, the KPIs to be used should be chosen in line with the specific characterises of the ITS service at hand. However, a large overlap in relevant KPIs exists for ITS services within the same market segment. For example, as is shown in the figure below, for Cooperative Vehicle Systems (CVS) the number of vehicles using the specific system is regarded the most relevant KPI, followed by the number of vehicles featuring ITS technology in general and the number of end-users of the ITS service.
As for the benefit KPIs, it was found that these should be best defined in line with the primary objective (i.e. safety, transport efficiency, environmental performance or comfort) of the service. For example, in case the primary objective of the service is to improve the environmental performance of the traffic, the level of emissions may be a relevant KPI, while in case of services targeted on transport safety the (change in) number of traffic accidents is more appropriate. It was also found that KPIs direct measuring the intended impacts are preferred to more indirect measures. For example, for ITS services which aim at improving the environmental performance of transport, the ‘level of emissions’ was regarded the most relevant KPI, while ‘change in transport volume’ was only mentioned scarcely.
Although these recommendations seems to be straightforward, a review of the KPIs used in actually implemented/piloted ITS services show that KPIs are often not applied at all. In case they are used, indirect measures are often preferred over more direct measures, probably because the former are more easily measureable and/or cover several impacts. Therefore, it can be concluded that there is sufficient room for improvement in this field.
For more information, see NEWBITS D2.2