1. Foreword

2. Purpose of paper

3. Overview

4. Scope

5. Scoring mechanism and weighting

6. Results for 1996–97

7. Interpretation

Appendices

1. Output measures for assessing comparative performance

2. Tables of scores and results

1. FOREWORD

The assessment of comparative efficiency made for the 1994 Periodic Review included comparisons of both costs and service. For the 1999 review I intend to make separate assessments of companies' performance in reducing costs and their success in improving service to customers.

This paper sets out the approach I am minded to use in assessing the relative performance of companies, taking into account compliance with environmental standards as well as service delivery to customers. It has not yet proved possible to include an assessment of drinking water quality but the Chief Drinking Water Inspector has agreed to work on comparative data that could, in future, be incorporated.

As indicated in Setting price limits (February 1998) I am minded to make an explicit adjustment in price limits to reflect the overall service customers have experienced during the current period. Such an adjustment would give companies further incentive to continue improving the service they offer.

I invite comment on the methodology used to assess company performance, both the measures used and the way they are combined. I would be grateful for any suggestions for improvement.

When I have received the views of interested parties, the methodology will be refined and more recent data incorporated — largely from this year's July Returns, from the other regulators and from the Customer Service Committees (CSCs). At the same time I will be considering how best the results may be reflected in future price limits. This work will form part of the consultation paper, Prospects for prices, strategic options and issues, to be issued in October this year.

I C R BYATT

Director General of Water Services

2. PURPOSE OF PAPER

This paper proposes a methodology for assessing how well companies perform overall in providing services to their customers. The results are illustrated using current information on company performance.

The need for an assessment is twofold:

• to enable service quality to be taken into account in setting prices that will reinforce the incentive to companies to maintain and improve services; and more generally
• to inform customers about the performance of their supplier.

3. OVERVIEW

The main points made in the paper are as follows:

• The overall service to customers provided by different companies can be compared by combining assessments of performance across a range of services important to customers.
• A methodology is described which, it is believed, reflects tangible differences in company performance.
• The described approach allows those companies performing outside the normal range for the industry to be identified. However, it is less useful in differentiating between companies in the middle of the range.

4. SCOPE

The Director General of Water Services (the Director) has decided that price limits should be adjusted for those companies offering particularly good or poor service to their customers. This was suggested in the consultation paper The proposed framework and approach to the 1999 Periodic Review (June 1997) and confirmed in Setting price limits for water and sewerage services (February 1998).

This technical paper:

• explains one method of assessing overall performance;
• sets out the results using this method; and
• highlights the issues which need to be considered in developing a methodology for assessing performance.

There will be more than one way of making an assessment. Ofwat has approached the issue with the aim of including the areas it believes are most important to customers and measuring performance in as objective a way as possible. The approach described in this paper is based on what Ofwat considers to be robust data, ie data which can be tracked to source, and uses simple and transparent processes to produce the results.

Most of the performance data used has been published in Ofwat's annual Levels of Service (LOS) report or in other previously published Ofwat reports.

Some nine years after privatisation it would seem reasonable to expect companies to have overcome any problems they might have inherited. Price limits set in 1989, and again in 1994, allowed companies to invest to remedy problems with inherited assets and systems. Nevertheless, to accommodate some companies' arguments that current performance reflects the condition of the assets they inherited in 1989, rather than performance since 1989, the paper also examines the improvement in performance during the past five years, ie the base year for setting price limits in 1994.

Results are produced for two groups — all companies, and the water and sewerage companies (WaSCs) — assessing their improvement since 1992 and their performance at the end of the year 1996–97.

4.1 What should be included in the assessment?

The assessment focuses on the delivery of services to customers, which is the companies' primary function. As with any other industry, these can be considered under the headings of product, distribution and availability and customer services; broadly equivalent to the wholesale, distribution and retail functions. There are three key areas for the assessment of the water only companies and the water supply elements of the WaSCs and two further categories for the sewerage and sewage treatment services. These are:

• water supply;
• water distribution;
• sewerage service (WaSC only);
• customer service; and
• environmental impact (WaSC only)

• services included should be of real importance to customers and, where possible, based on those used in the published reports;
• measures should be meaningful to companies and customers;
• data should be supported by high quality information; and
• objectively assessed measures are preferred where possible.

4.2 Key assessment areas

The specific measures that Ofwat proposes to include are discussed below and are described further in the tables in Appendix 1.

4.2.1 Reliability of water supply

The likelihood of receiving a restricted water supply is clearly important to customers. Simple comparative measures of the reliability of essential supplies are not available and no company has needed to restrict essential supplies since 1976. However, there are differences in the impact on customers of hosepipe and sprinkler bans in different company areas. While the imposition of hosepipe bans clearly reflects a difference in the level of service to customers, it may also reflect reasonable and necessary action by a company to preserve essential supplies. Weather and water resource positions will also vary. However, the data available suggests there is variation in performance under similar circumstances. By comparing total performance over a long timescale, annual variability can be taken into account.

Companies' progress towards their economic levels of leakage may also be considered to reflect their performance in managing their water resources on behalf of their customers. Good leakage performance will help companies secure essential supplies and to minimise the environmental impact of water abstraction.

Since company submissions on the economics of leakage will not be available until June 1998, it has been necessary to develop a proxy measure. For this paper we have used the matrix showing leakage levels against resource positions published in Ofwat's Report on leakage and water efficiency (October 1997). To chart performance over time, the model also examines companies progress since 1992–93.

Whilst leakage is clearly an issue which causes customers concern, it is recognised that the effect of high leakage levels on customers is an indirect one. It may be considered that leakage is adequately dealt with elsewhere in the regulatory system, through companies' supply demand submissions and the setting of leakage targets. However, on balance, we think it appropriate to include a measure of leakage of water from company pipes as one element in the overall assessment of current performance.

4.2.2 Water distribution and quality

Two important aspects of company performance in supplying water are pressure, and interruptions to supply. Performance in these areas is reviewed annually against defined standards — DG2 for pressure and DG3 for interruptions — and results are published in the LOS report. The safety and quality of water supplied is also very important to customers. The Drinking Water Inspectorate (DWI) monitors compliance with water quality regulations.

DG2 measures the total number of properties at risk of receiving water below a prescribed rate of flow and pressure. The data is derived from a company assessment of risk and allows exclusions for abnormal demand. Performance improvement is measured by the total number of DG2 pressure problems solved through company action since 1992(this excludes properties added to or removed from the 'at risk' categories because of better information alone). The methodologies associated with company risk assessment are now generally sound and the data is considered suitable for comparative purposes.

Companies provide data in their July Returns on planned and unplanned interruptions to supply (DG3). The LOS report concentrates on the latter and uses a scoring system to reflect the number and duration of interruptions in order to produce comparative performance assessments. These results have been used in this analysis. Planned interruptions have not been included because of the difficulty in taking account of the impact of different maintenance techniques used by the companies.

Performance improvement in the area of interruptions to supply is based on a comparison of the rolling average figure for interruptions in excess of 12 hours (the only data available with a history since 1992) for 1992–95 and that for 1994–97.

The safety of drinking water and its quality in terms of taste and appearance is clearly important to customers and should be included in an assessment of service to customers. The Chief Drinking Water Inspector advises that the overall level of compliance with safety standards is such that no meaningful distinctions can be drawn between companies' performance. Overall compliance reflects not only the historical situation but also a combination of a large number of parameters to which it is not sensible to attribute any weighting. Furthermore, the strong drinking water quality enforcement regime is such that the companies have no choice but to make the necessary improvements to comply with standards.

It may, however, be possible to compare performance in one area important to customers — discoloured water as reflected in failure against the standards for iron content. Accordingly, the Chief Drinking Water Inspector is working on producing comparative performance data that may be available for inclusion as part of the water supply key area in the future.

4.2.3 Sewerage service — sewer flooding

Customers take a properly functioning sewerage system for granted — the only real evidence of failure will be flooding. Market research shows that customers regard this as a serious issue. Accordingly, it is appropriate that sewer flooding should form part of this assessment.

The Director collects and publishes annual data on company performance for flooding due to inadequate sewer capacity (including an assessed risk of flooding as well as actual incidents) and flooding incidents related to the condition of sewers and associated equipment. The former results from long term problems that generally can only be resolved by capital investment; the latter is generally the result of insufficient ongoing maintenance and is more within companies' control.

The assessment in this paper of company improvement on risk of flooding is based on the reported number of problems solved since 1992 through company action (this excludes properties added to or removed from the 'at risk' categories due to better information alone). The improvement in the number of flooding incidents is expressed as the change in the rolling average figure for 1992–95 and that for 1994–97; this will minimise the impact of atypical weather conditions in a single year.

Combined sewer overflows are also part of the sewage collection system and, as such, might be expected to appear here in this part of the assessment. However, performance in this area has been included in the environmental impact section as failures will have their major effect on the receiving rivers and coastal waters.

4.2.4 Customer service

Data on current levels of customer service is available from two key sources — the companies and the CSCs. It exists in a number of forms, from the objective data provided annually by companies in the July Return and the existence and nature of specific services offered, to the partly subjective assessments of service quality made by the CSCs. Four objective and independently audited measures of customer contact are available and have been included in this overall assessment — response to billing contacts (DG6), replying to written complaints (DG7), issuing bills for metered customers (DG8) and speed of response to telephone contacts (DG9). Two of these —DG6 & DG7 — have a long enough history to be used in the analysis of performance improvement.

Customer service clearly goes wider than the speed of response to complaints and frequency of reading meters. It is therefore important to include an assessment of other aspects. To this end a further eight measures of service are included:

• debt and disconnection;
• information to customers;
• opening hours for telephone access;
• compensation policies;
• range of payment options;
• special needs services;
• meter option schemes; and
• supply pipe repair policies.

Members of the CSCs and companies have expressed concern that there are currently no measures reflecting the quality of replies to complaints or the quality of the telephone service provided. In respect of written complaints, the limited results of CSC audits are being used to update the 1996 analysis of company complaint handling procedures. This may result in suitable data for inclusion. Data on quality of response to telephone contact is not currently available but could be incorporated in future if it becomes so.

Assessing improvement in customer service is more difficult, since historical information is not available in all areas. However, the measures DG6 and DG7 are measures of customer service — albeit time related — and do seem to correlate with good service in other areas. Accordingly, improvement in these figures since 1992 has been used.

4.2.5 Environmental impact

Customers are clearly interested in understanding the environmental impact of the activities of companies, especially since this is a major driver behind the increases in customers' bills. The Environment Agency (EA) has provided the Director with information for inclusion in the last three LOS reports and its advice was sought on measures chosen to make a comparative assessment of environmental impact. It is supportive of the general approach adopted.

The three measures included to reflect the environmental impact of the sewerage activities carried out by the WaSCs are selected from those reported in the LOS report in the last two years. The LOS reports include data in seven areas. However, the number of successful prosecutions is not used in this analysis as company performance in this area is adequately reflected in the data on pollution incidents. Also, it is felt that outputs relating to marine performance (bathing waters non-compliant and unsatisfactory sea outfalls) should not be included as these do not apply to all companies. Some CSCs and the EA felt that these were important issues that should be included. However, the method of analysing scores used needs an indication of comparative performance for all companies for each issue.

The measures chosen, therefore, are:

• Sewage treatment works failing their consent. Failures considered by the EA not to be reflective of company performance have been excluded.
• Data on unsatisfactory combined sewer overflows. This reflects company progress in dealing with the problem of overflows that the EA considers are unsatisfactory.
• The total number of major and significant sewage related incidents as defined in the EA report Water Pollution Incidents in England and Wales (1996). The annual total of major and significant incidents is expressed as a percentage of the resident equivalent population served by sewage treatment works to allow for the different size of companies. (While it may have been better to use the number of outlets where an incident could take place as a denominator, this data is not available).

5. SCORING MECHANISM AND WEIGHTING

5.1 Scoring mechanism

The objective data used in the assessment originates from the companies, though it has been independently audited, and the EA. Assessment of the quality of customer service provided is made by the CSCs and by Ofwat.

The raw data is processed to produce a performance score by identifying the best and worst performances in each area (both for the all company analysis and the WaSC analysis) and awarding maximum and minimum marks respectively in a range from 5 to 50. Intermediate performances attract proportional scores. Adjustments are made for extraordinary data values — very high or low — that could unduly skew the performance scores by extending the range between the best and the worst companies. The following exceptional figures were excluded from the initial calculation of the range of performance and were awarded the highest or lowest score as appropriate.

• Bournemouth & West Hampshire for water pressure (the risk of customers receiving low water pressure is more than ten times the average and four times that of the next worse company);
• Northumbrian for improvement in unplanned interruptions to supply (freeze/thaw influenced results);
• Bournemouth & West Hampshire, Essex & Suffolk, and Tendring Hundred for water pressure improvements (poor starting positions gives unusually high problem solving results);
• Northumbrian for sewer flooding problems solved (poor reporting methodology results in apparent deterioration).

The contribution to the total current performance score by each of the five key areas is shown in Table 1 overleaf. Table 2, meanwhile, summarises the weighting of the contributing performance measures in the analysis of performance improvement.

Table 1: Score weighting (current performance)

Key area	Sub-area	Weight	Total
Water supply	Hosepipe bans	1	3
	Leakage and resource position	2
Water distribution	Water pressure	1.5	3
	Unplanned interruptions	1.5
Sewerage service	Incidents due to sewer capacity	1	3
	Incident due to blockages etc	1.5
	Risk of flooding	0.5
Customer service	Customer contact	1	3
	Assessed service	2
Environmental impact	Sewage treatment works in breach of consent	1	3
	Unsatisfactory combined sewer overflows	1
	Major and serious pollution incidents	1

Table 2: Performance improvement

Key area	Sub-area	Weight	Total
Water resources	Total leakage — improvement since 1992–93	3	3
Water distribution	Pressure problems solved by company action	1.5	3
	Unplanned interruptions — improvement since 1992	1.5
Sewerage service	Problems solved by company action	1.5	3
	Reduction in incidents due to blockages etc since 1992	1.5
Customer service	DG6 (billing contacts) improvement since 1992	1.5	3
	DG7 (written complaints) improvement since 1992	1.5
Environmental impact	Sewage treatment works in breach of consent	1	3
	Unsatisfactory combined sewer overflows	1
	Major and serious pollution incidents	1

The five key areas have been equally weighted in the model. Clearly different weightings could be applied. One obvious consideration is the relative cost in delivering the various elements of service. Another is the relative importance that customers attach to each. While these could lead to a different set of weights from those identified in Tables 1 and 2, both would involve an element of subjective judgement and opinions would vary. Small changes in relative weighting have been examined but these have, as might be expected, little material effect on the outcome. On balance, it would seem there is merit in simplicity.

Within the five key areas it is possible to make limited judgements as to the relative weightings, for example for sewer flooding incidents compared with the theoretical risk of flooding. Wherever possible alternatives were tested to prove the robustness of outcomes. This showed that, in general, changes in weighting within each key area did not have a major effect and that the model was not unduly sensitive to variations in a single performance parameter.

The results produced by the model were checked to see if they reflected differences in company performance that would be obvious to customers. Comparisons were made between companies exhibiting the highest and lowest levels of performance according to the model using 'pen-pictures' of real performance. This exercise confirmed that the model was drawing out real performance differences that would be likely to influence customers' choice of supplier, were they to have a choice.

6. RESULTS FOR 1996–97

The results for 1996–97 using the proposed model are illustrated below in Figures 1 to 4. The scale in each case is set so that the base is the minimum achievable score using the proposed methodology (45 for water and customer service, 75 when sewage service is added in), higher scores represent better performance. Companies are not identified in the figures and the letters at the bottom of each column refer to the respective tables in Appendix 2 where the composition of the total scores is detailed. Companies do not keep the same letter in each figure. On the figures companies with scores greater or less than one standard deviation from the mean score have histogram bars shaded black. The total scores only illustrate difference in absolute performance — a company with a score that is twice that of another is not necessarily performing twice as well.

6.1 Current performance

The model outputs for current performance for all companies and WaSCs are shown in Figures 1 and 2 below.

Figure 1: All companies (including water supply, water resources and customer service)

Figure 2: Water and sewerage companies (including water supply, water resources, customer service, sewerage service and environmental impact)

6.2 Performance improvement

The results of the analysis of improvement are shown Figures 3 and 4 below:

Figure 3: All companies (including water supply, water resources and customer service)

Figure 4: Water and sewerage companies (including water supply, water resources, customer service, sewerage service and environmental impact)

We have considered scoring the sewerage service separately from the water service. The difficulty with this is how to handle customer service since the performance data in this area is not separated for the two services.

7. INTERPRETATION

As mentioned above alteration, to the weighting will influence the outcome of the analysis generally, although not materially. Nevertheless, the results cannot be assumed to identify precise rankings, but rather broad bands of overall performance. For the purposes of this paper these bands have been identified based on intervals of +/- one standard deviation. These are shown on the figures. Different banding methods (lower confidence limits, percentiles, natural breaks) could have be used.

The robustness of the analysis is reliant on the quality of the data on which it is based and, of course, on the relative weightings. For this reason objective, audited data has been used wherever possible and consistency checks applied to the more subjective elements. Checks have also been carried out to ensure that the analysis reflects tangible and not artificial differences in performance. Ofwat believes that the results are reasonably robust. However, there are limits on the degree to which it is possible to make meaningful distinctions between companies when scores are not materially different.

The next step is to consider how a company's current performance may relate to its improvement score. To illustrate this a matrix approach has been used. The two matrices below show the final output for the all company and WaSC analyses. The 'A' columns and rows in matrices are the 'best' performance areas. While companies are identified for illustrative purposes in the matrices, it should be remembered that the data used to illustrate this paper will not be that used for any final assessment.

Figure 5: Water — all company results

All company analysis		Current performance
		C	B	A
Performance improvement	A	Bournemouth & West Hampshire	Folkestone & Dover , Yorkshire, North West	Southern
	B	Three Valleys, Essex & Suffolk, Mid Kent	Portsmouth, Wrexham, South Staffordshire, Wessex, Severn Trent, Chester, South East, North Surrey, Cambridge, Dwr Cymru, York, Mid Southern, Tendring Hundred, South West, Sutton & East Surrey	Anglian
	C		Northumbrian, Bristol, Thames	Hartlepool

Figure 6: WaSC results

Water and sewerage company analysis		Current performance
		C	B	A
Performance improvement	A		North West
	B	South West , Yorkshire	Severn Trent, Dwr Cymru, Thames Wessex	Southern, Anglian
	C		Northumbrian

See Erratum for Figures 5 and 6

It can be seen that although the improvement score varies, in only a small number of companies has performance improvement differed from the norm. It might be appropriate to take account of this when making adjustments to price limits to reflect the level of service provided to customers. Some might argue that lower current performance simply reflects the inadequacy of water and sewerage systems at the time of privatisation. It might, however, also be expected that such companies would have made greater efforts to improve performance. The matrix does not suggest that this has been the case. Ofwat therefore believes it appropriate to attach greater weight to the assessment of current performance using the improvement performance to modify judgements where necessary.

This paper has produced two sets of results — all companies, and water and sewerage companies. The all company analysis does not include any sewerage or environmental scores (apart from leakage) and only represents performance in the water and customer service areas. The WaSC analysis includes sewerage and environmental performance results as well as those for water and customer service results. In so far as the outcome of the WaSC performance assessment is broadly consistent with the all company analysis, it is reasonable to conclude that a company tends to perform to a similar standard across its operations. If significant differences were found it might be appropriate to take account of this in reaching judgements about any price adjustments.

APPENDIX 1. OUTPUT MEASURES FOR ASSESSING COMPARATIVE PERFORMANCE

Table 3: Output measures for assessment of current performance

Output	Source	Description	Performance range	Scoring criteria
Water pressure 1996–97	Company data	Company assessment of properties at risk of receiving low pressure	From 5.5% properties at risk (worst) to zero at risk	Percentage at risk figure scored from 5 (poorest performance) to 50 (best)
Unplanned interruptions to water supply	Company data	Properties affected by unplanned interruption to supply greater than six hours	Performance scores (combination of 6,12 and 24 hour interruptions) from 2.77 (worst) to 0.14 (best)	Comparison of interruption scores as used in the Ofwat Levels of Service report. Interruption scores scored from 5 (poorest performance) to 50 (best)
Sewer flooding incidents due to overloaded sewers	Company data	Properties flooded internally by sewage as a result of an overloaded company sewer	Percentage of connected properties flooded from 0.01 (worst) to 0.001 (best)	Sewer flooding incidents due to hydraulic incapacity excluding extreme weather events. Percentage figure scored from 5 (poorest performance) to 50 (best)
Sewer flooding incidents with causes other than capacity	Company data	Properties flooded internally by sewage — caused by blockages, sewer collapses, equipment failure etc.	Percentage of connected properties flooded from 0.035 (worst) to 0.005 (best)	Sewer flooding incidents due to hydraulic incapacity excluding extreme weather events. Percentage figure divided into ten bands and scored from 5 (poorest performance) to 50 (best)
Properties at risk of sewer flooding	Company data	Properties at risk of internal flooding from sewers more than once in ten years	Percentage of connected properties at risk from 0.244 (worst) to 0.12 (best)	Sum of properties at risk of flooding more than once in ten and twice in ten years expressed as a per cent. Percentage figure scored from 5 (poorest performance) to 50 (best)

Output	Source	Description	Performance range	Scoring criteria
Company contact score	Company data	Speed of response to billing contacts	Percentage of contacts answered within five working days from 71.3 (worst) to 100 (best)	Percentage of substantive replies dispatched within five working days. Percentage figure scored from 5 (poorest performance) to 50 (best)
		Speed of response to written complaints	Percentage of written complaints answered within ten working days from 86.9 (worst) to 100 (best)	Percentage of substantive replies dispatched within ten working days. Percent figure scored from 5 (poorest performance) to 50 (best)
		Meter reading performance	Percentage of meters read annually from 92.9 (worst) to 100 (best)	Percentage of bills based on actual meter readings during the year. Percentage figure scored from 5 (poorest performance) to 50 (best)
		Ease of telephone contact	Percentage of calls answered within 30 seconds from 48.3 (worst) to 97.8 (best)	Calls answered within 30 seconds as a per cent of all calls answered. Percentage figure scored from 5 (poorest performance) to 50 (best)
Assessed service score	Ofwat and Customer Service Committees	Payment options available to customers	Based on 1995 Ofwat study (1995) and updated by CSCs in 1997.	Five areas assessed: information on bills and in other literature, range of options, instalment options, payment locations and counter fees. Numerical assessment resulted in a three band result. This is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)
	Ofwat and CSCs	Services provided for customers with special needs	Based on criteria established in Ofwat study Services for Disabled and Elderly customers (1993) and updated by CSCs in 1998.	Nine key areas assessed and numerically scored resulting in a three band result. This is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)
		Company meter option schemes	Performance assessed by cost of company preferred option qualified by extent of promotion in annual billing literature.	Based on data contained in Ofwat's Tariffs report. Three band result. This is combined with the other assessed service scores and the total divided into four areas scored from 5 (poorest) to 50 (best)
		Compensation for poor service	Best marks for companies going beyond GSS and offering £20 payments and automatic payments. Lowest band for companies going no further than GSS	Assessment of company policies and charters. Three band scoring system — result is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)

Output	Source	Description	Performance range	Scoring criteria
Assessed service score	Ofwat	Information provided to customers with their bills		Measured bills, unmeasured bills, accompanying literature, and overall customer friendliness of material assessed resulting in a final score from 1 to 3. This is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)
	Ofwat	Company policies in respect of supply pipes	Based on data contained in Ofwat Leakage and water efficiency report. Companies assigned to five groups, from best (all external leaks repaired free, some free replacement and full reinstatement) to worst (only first external leak repaired free).	Result is modified to a range of 0.5 to 3 and then combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best).
	Ofwat and CSCs	Debt and disconnection policies	Uses criteria developed in 1996 Ofwat report on company progress in implementing debt and disconnection guidelines. Results (scale of 1 to 3) updated by CSCs in 1997 and modified by 1997 disconnection figures (no company with a disconnection rate over 10 per 10,000 customers can get the highest marks).	Result is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)
	Ofwat	Customer access to company by telephone	Uses results published in Ofwat's Levels of service reports. Three categories — best for companies offering 24 hour telephone access, lowest for those only available on a working day basis.	Result is combined with the other assessed service scores and the total scored from 5 (poorest) to 50 (best)

Output	Source	Description	Performance range	Scoring criteria
Pollution incidents	Environment Agency	Category 1 & 2 pollution incidents	Incidents per million equivalent resident population range from 13.69 (worst) to 1.15 (best).	Major and significant sewerage related incidents only expressed per million equivalent resident population. For calendar year 1996. Scored from 5 (poorest performance) to 50 (best).
Sewage treatment works breaching consent	Environment Agency	Sewage treatment works failing consent parameters	Percentage of equivalent resident population served by sewage treatment works failing consent range from 13.9 (worst) to 0.25 (best).	Percentage of equivalent populations served. Scored from 5 (poorest performance) to 50 (best)
Unsatisfactory combined sewer overflows	Company	Percentage of sewer overflows operating in an unsatisfactory manner	Percentage of total combined sewer overflows unsatisfactory — range from 48.9 (worst) to 4.99 (best).	Percentage of total unsatisfactory combined sewer overflows. Percentage figure scored from 5 (poorest performance) to 50 (best)
Total hosepipe bans since 1992–93	Company	Average person weeks as % of winter population	Range of results from 1.59% (worst) to 0% (best)	Percentage figure compared and scored from 5 (poorest performance) to 50 (best)
Leakage	Company	Leakage and resources	Using the Ofwat leakage and resource matrix, assigned scores range from 5 (very high leakage, tight resources) to 50 (low leakage, adequate resources).	Six performance bands scoring 5 (very high leakage — tight resources), 10 (high leakage- tight resources, very high leakage — marginal resources), 20 (very high leakage — adequate resources, high leakage — marginal resources, medium leakage — tight resources), 30 (low leakage — tight resources, medium leakage — marginal resources, high leakage — adequate resources), 40 (low leakage — marginal resources, medium leakage — adequate resources), 50 (low leakage — adequate resources)

Table 4: Output measures for assessment of performance improvement

Output	Source	Description	Performance range	Comments
Water pressure problems solved since 1992	Company	Properties removed by company action from risk of receiving low water pressure since 1992	Problems solved per 10,000 customers since 1992. Range from -10.87 (worst) to 4475.76 (best)	Change in at risk figure since 1992 after changes brought about by new information Percentage figure scored from 5 (poorest performance) to 50 (best)
Unplanned interruption improvement	Company	Change in level of unplanned interruptions over 12 hours	Range of 12 hour three year average from -0.842 (worst) to 0.462 (best)	Comparison of 1992-95 average with 1994-97. Percentage figure scored from 5 (poorest performance) to 50 (best)
Sewer flooding problems solved since 1992	Company	Reduction by company action in number of properties at risk	Range of problems solved from -176.62 (worst) to 60.76 (best)	Properties removed from at risk categories as % of the 1992 figure after changes brought about by new information. Percentage figure scored from 5 (poorest performance) to 50 (best)
Reduction in number of sewer flooding incidents due to blockages etc since 1992	Company	Change in the number of sewer flooding incidents caused by blockages, collapses, equipment failures, etc	Range of comparative scores from 0.63 (worst) to 2.69 (best)	Comparison of 1992-95 average with 1994-97. Figure scored from 5 (poorest performance) to 50 (best)

Output	Source	Description	Performance range	Comments
Improvement in speed of response to billing contacts	Company	Change in percent billing contacts answered within five working days since 1992	Range of comparative scores from -19.66 (worst) to 29.10 (best)	Comparative figure scored from 5 (poorest performance) to 50 (best)
Improvement in speed of response to customer complaints	Company	Change in percentage of complaints answered within ten working days since 1992	Range of comparative scores from -10.27 (worst) to 40.52 (best)	Comparative figure scored from 5 (poorest performance) to 50 (best)
Reduction in pollution incidents	Environment Agency	Category 1 & 2 pollution incidents	Results for 1994 divided by those for 1996. Range of outputs from 0.61 (worst) to 11.20 (best)	Major and significant sewerage related incidents only. For calendar year 1996. Improvement ratios scored from 5 (poorest performance) to 50 (best).
Reduction in sewage treatment works breaching consent	Environment Agency	Sewage treatment works failing consent parameters	Results for 1994 divided by those for 1996. Range of outputs from 0.001 (worst) to 11.18 (best)	Comparison of equivalent population served. Improvement ratios scored from 5 (poorest performance) to 50 (best)
Reduction in unsatisfactory combined sewer overflows	Company	Sewer overflows operating in an unsatisfactory manner	Results for 1994 divided by those for 1996. Range of outputs from 1.04 (worst) to 2.10 (best)	Comparison of number of combined sewer overflows operating unsatisfactorily. Improvement ratios scored from 5 (poorest performance) to 50 (best)
Improvement in total leakage	Company	Change in total leakage	Reduction in total leakage since 1992-93 as a percentage of 1992-93 total leakage. Range of results from -35% (worst) to 30% (best)	Comparison of leakage levels. Improvement ratios scored from 5 (poorest performance) to 50 (best)

APPENDIX 2. TABLES OF SCORES AND RESULTS

Table 5: Scores and results for current performance - all companies
1. Unweighted score

2. Weighted score

Erratum: Following publication of the subject paper an error has been found in tables 5 and 8 in Appendix 2. In table 5 the weighting factor has not been applied to lines 1 and 2; similarly, the weighting has not been applied to lines 3 and 4 in table 8.

When corrections are made the net effect is to move company C in table 5 out of the top category, and to move company B into the top category in table 8. No other changes are material to the illustrative results contained in the paper. Of course the changes will affect the matrices included in the paper, moving Anglian Water in figure 5 from the AB box to the BB box, and in figure 6, moving Anglian Water from the AB to the AA box.

Table 6: Scores and results for current performance - water and sewerage companies

Unweighted score

No	Output	Weight	A	B	C	D	E	F	G	H	I	J
	Water supply, levels of service
1	DG2 - 96-97 risk of low pressure score	1.5	5	49	14	48	42	40	50	42	20	23
2	DG3 - 96-97 unplanned interruption score	1.5	50	49	48	5	50	45	12	40	5	40
	Sewerage service, levels of service (WaSCs only)
3	Sewer flooding incidents 1996-97 (capacity) score	1	35	44	50	35	5	24	33	24	46	13
4	Sewer flooding incidents 1996-97 (other causes) score	1.5	50	34	45	41	49	41	37	25	44	5
5	Company assessed risk of flooding more than once in ten years (score)	1	43	46	31	5	50	43	46	37	46	49
	Customer service
6	Company contact score	1	14	50	44	11	20	5	32	5	8	29
7	Assessed service score	2	40	20	25	50	5	20	30	20	10	40
	Environmental performance (WaSCs only)
8	Category 1 & 2 pollution incidents 1996-97 per million equivalent resident population (score)	1	40	41	29	50	17	41	5	36	32	36
9	Percentage of equivalent population served by STWs in breach of their consent (score)	1	34	42	49	47	40	5	39	50	16	5
10	Unsatisfactory combined sewer overflows as a % of the total (score)	1	39	5	29	47	50	40	7	14	34	21
	Water resources
11	Hosepipe bans since 1992	1	50	39	50	50	50	23	50	13	35	5
12	Leakage - leakage vs resource matrix	2	40	30	20	5	30	40	20	30	40	10
	Totals
13	Total unweighted WaSC score		440	448	433	394	407	366	361	335	336	277

Weighted score

No	Output	Weight	A	B	C	D	E	F	G	H	I	J
	Water supply, levels of service
1	DG2 - 96-97 risk of low pressure score	1.5	8	74	20	73	63	60	75	62	29	34
2	DG3 - 96-97 unplanned interruption score	1.5	75	73	72	8	75	68	18	60	8	60
	Sewerage Service, levels of service (WaSCs only)
3	Sewer flooding incidents 1996-97 (capacity) score	1	35	44	50	35	5	24	33	24	46	13
4	Sewer flooding incidents 1996-97 (other causes) score	1.5	75	51	67	61	73	61	56	38	67	8
5	Company assessed risk of flooding more than once in ten years (score)	0.5	21	23	15	3	25	22	23	18	23	25
	Customer service
6	Company contact score	1	14	50	44	11	20	5	32	5	8	29
7	Assessed service score	2	80	40	50	100	10	40	60	40	20	80
	Environmental performance (WaSCs only)
8	Category 1 & 2 pollution incidents 1996-97 per million equivalent resident population (score)	1	40	41	29	50	17	41	5	36	32	36
9	Percentage of equivalent population served by STWs in breach of their consent (score)	1	34	42	49	47	40	5	39	50	16	5
10	Unsatisfactory combined sewer overflows as a % of the total (score)	1	39	5	29	47	50	40	7	14	34	21
	Water resources
11	Hosepipe bans since 1992	1	50	39	50	50	50	23	50	13	35	5
12	Leakage - leakage vs resource matrix	2	80	60	40	10	60	80	40	60	80	20
	Totals