Key performance indicators (KPIs)
The OCs are required to provide certain information to indicate how they are performing in complying with the provisions of the contract.
This KPI information is used solely to monitor performance and has no contractual significance.
5.8 Key performance indicators
The contracts contain descriptions for each KPI, including the source of the data and the format of the values to be provided. However, issues regarding the consistency of measurements to be used were raised by NW and SW during 2006/07. These were substantially resolved following discussions between Transport Scotland, PAG and the OCs.
The agreement required some adjustment to the OCs’ methods of calculating KPIs and it was agreed these will be introduced progressively to the end of 2007/08 and fully implemented during 2008/09.
PAG assessment for reasonableness
Some KPI information is reported by the OCs each month. PAG collates this, compares OC performance, identifies trends and includes a summary in its briefing notes for the progress meetings with all the OCs.
NE - BEAR
A PAG audit in March 2008 examined samples of results provided for twelve of the KPIs. This found that most of the results were verifiable, although some calculations were based on incomplete information.
The most recent information provided by BEAR at the end of 2007/08 corrected around 26% of the values provided during the year. Some of this was due to adjustments in the method of measurement, but the lack of robust values during the year is a concern.
PAG will monitor the results and revisions provided during 2008/09.
SE – BEAR
The PAG audit of twelve KPIs in April 2008 examined the results provided during 2007/08. The results for four of the twelve KPIs could not be verified and BEAR was required to improve its systems of recording and reporting KPI data in
these cases.
Results provided by BEAR at the end of 2007/08 amended over 40% of the results provided during the year. While some of the changes are minor, or as a result of adjustments in the method of measurement, concern remains that the values provided during the year were not a true indication of the OC’s performance.
PAG will monitor the results provided during 2008/09.
NW - Scotland TranServ
In March 2008, PAG audited samples of results provided during 2007/08 for twelve KPIs. The audit found that Scotland TranServ had developed effective processes to produce verifiable KPI results. Where declining performance was indicated by the KPIs, NCRs were raised to generate improvement.
The OC revised or included previously unreported results for around 10% of the KPI results at the end of the year.
SW – Amey
The PAG audit of KPI results in SW recorded that some could not be verified. This problem had been recognised by Amey and NCRs had been raised to correct the problems identified.
Only approximately 6% of the results provided during 2007/08 were revised in the most recent KPI information provided at the end of the year.
