Value for Money and the PSC

Value for money (VfM) has two basic meanings within the PPP development process;

Firstly, that there is an absolute benefit to a country of implementing projects through a PPP modality. This Toolkit provides guidance to ensure that PPP projects in general, if implemented through established procedures will provide general value for money.

The fundamental objective of PPPs is to maximize efficiency and generate investment that is value for money.  PPP procurement has the potential to offer better value for money through:

• Allocation of risk to the party that can manage it best;
• Performance based payments;
• Capturing private sector innovation, commercial and management expertise by involving the private sector more centrally in the provision of assets and services;
• Use of long-term contracts whereby bidders focus on the whole life cycle cost of projects and not just on the upfront capital costs.  This can lead to more innovative designs with lower life-cycle costs and higher maintenance and operational standards; and
• Better project delivery than the public sector; much reduced time and cost overruns
• Increasing the tax revenue base of the country.

The PPP approach, in addition to leveraging additional funding for the highway investment program, also secures significant risk transfer to the private sector and brings private sector expertise to bear on the design, construction and operation of the facility.

General VfM is secured, at this level, through considering the appropriate PPP procedures in this Toolkit.  The obligations within PPP contracts which the Contracting Authority has entered into as part of the PPP roads program should be devised with the objective of ensuring value for money to the public sector. 

The Second meaning; is that a project, undertaken through PPP, delivers a net benefit relative to a public procurement procedure for the same specific project i.e. the country receives a better deal through PPP than public procurement for any proposed project.

This is analyzed through a procedure for comparing public and PPP projects through use of the 'Public Sector Comparator' (PSC). This second, relative procedure is the subject of this section of the Toolkit.

It should be noted that a PPP project of any reasonable size will have a macro-economic impact. However, when comparing public and PPP projects the relative difference in macro economic terms is likely to be small, except possibly for extremely large projects in small low-income countries, where particular care, in terms of macro-economic impact, is needed under both PPP and public procurement.

The VfM in relative terms, considered through the use of the PSC, is a micro economic financial tool that compares the project costs and revenues of the same project implemented under two different development scenarios i.e. public or PPP procurement.

However, while it is generally accepted that a VfM process is required to choose between conventional public procurement and a PPP modality for funding, the level and extent of the analytical process is still debated. In general, the developed economies have used a far more sophisticated procedure to test and ensure value for money. However, there still remains some debate over both its theoretical basis and practicability of application and especially;

• Whether the use of the sophisticated procedure (PSC) in developing economies is justified if, through lack of finance, the project will not be implemented;
• Whether its use, in fact, in both developed and developing economies can also be justified on the grounds of assumptions and data needs as it always involves comparing two hypothetical.

The key-determining factor in deciding whether to develop any specific project as a PPP is value for money, whether in general or in comparison to public procurement.  Each project is evaluated on its own merits under both scenarios and if it does not offer VFM, it will not be undertaken on a PPP basis. Most experienced PPP organizations indicate that the VfM check will be undertaken at a number of stages in the PPP implementation process. It is important to note that Value for Money analysis is performed on a financial, and not an economic, basis.

However, especially in a developing economy, a practical consideration is that the later the evaluation within the project cycle, the less likely the project will be dropped after a considerable procurement process and considerable human and financial costs have been expended.

For example, if the government does not have the resources to implement the project through the public budget, the implications of the analysis could well be ignored although the VfM analysis could still likely suggest concession conditions helpful to the both partners before contract signing.

A range of PSC issues and recommendations are provided by Leighland and Shugart in the following PPIAF Gridlines reference.

Is the Public Sector Comparator right for developing countries? J Leighland and C Shugart. PPIAF Gridlines 2006

If the government has the resources to fund the project and the conclusions are overwhelmingly not to use PPP, then the project could well be taken out of the PPP program. However, in many countries there are likely to be many toll/high grade expressway projects ranging from the financially viable to the completely unviable. There may also be substantial amounts of public sector funds (national budget and soft/hard loans) potentially available. The use of VfM and a simplified PSC may help to decide on the type of procurement and even if PPP is not suitable for initial development in some or many cases, PPP may be proposed in some form including for the subsequent operation and maintenance.

The financial terms of PPP contracts should be scrutinized by the contracting authority/highway authority's financial advisers. This includes undertaking sensitivity analysis and an assessment of equity returns to the PPP company at traffic volumes in excess of both the PPP company's forecasts and contracting authority forecasts.

In broad terms, the assessments of the financial adviser should conclude that the contracts entered into represent value for money for the public sector and any revenue sharing arrangement included in the contracts is such that the PPP company will not earn excess profits from these schemes. Should such excess profits appear later, the contract may allow such profits to be limited to the users' benefit, or shared with the government or in any other way beneficial to the public.

In developed economies, ministries of finance have tried to apply other tests (such as risk assessments) as to whether a project (or service) should be developed via PPP. However, clearly, value for money is a primary concern in PPP projects. In situations where there is a clear possibility to fund a project either by public procurement or through the private sector, VfM/PSC analysis provides a sound basis for choosing one or the other through development modality.

What is the PSC?

This concept is used in the UK and some other countries but it has had mixed success and initially was not easy to apply. The basic concept is that when a PPP project is proposed it is compared in financial terms with the cost of the public procurement of the same project. Other tests include whether the project has high risks, might fail and become an unintended contingent liability for government.

In most developing countries, where there is a huge financing gap, the choice is between implementing a project with the private sector or not funding it at all. The lack of public finance is therefore often the reason to justify PPP. In such a case, the European Investment Bank (EIB) comments that where public finance constraints are real and severe, the only public alternative would be to postpone investments or even cancel them, with consequent major economic loss from doing nothing/doing little e.g. port or road congestion costs continue to escalate.

EIB therefore recommends that the PPP option should be clearly favored provided that it has a real impact on public finance (substantial transfer of risks to the private sector) and that it yields a satisfactory economic rate of return.

Public-private Partnerships for Transport Infrastructure Projects, Patrick Boeuf, European Investment Bank, 2003

One practical difficulty is that whole life financial costs are difficult to accurately quantify under public procurement where historically the public sector has severely under resourced highway maintenance. Further, the financial outcome of decisions possibilities (options) not selected are difficult, if not impossible, to determine so that VfM decisions are unlikely ever to be verified even over time.

The VfM is, to an extent, verified and captured through the general VfM process because the Toolkit identifies the variables involved in measuring VfM which can at least help ensure that they can be achieved through a PPP project. However, without VfM and the PSC, the Toolkit does not provide a means to assess whether a project should be implemented through PPP or public procurement. This is the purpose of this section.

In the initial/early stages of project evaluation, a partly qualitative approach to VfM analysis can be considered. This was referred to in this table. This involves the Government asking key questions on the suitability of a project for PPP and to which qualitative answers can be provided if hard data is not available.

This process can be used both at an early stage of the PPP process and later with the PSC evaluated during and after the FS study process when more hard data will be available. Sometimes, for reasons of time, budget and institutionally, the PSC process is undertaken only once.

Criteria for VfM

Projects which demonstrate value for money under PPP generally show potential under the following criteria for VfM:

• There is a major capital investment program, requiring effective management of risks associated with construction and delivery;
• The private sector has the expertise to deliver and there is good reason to think it will offer VfM;
• The structure of the service is appropriate, allowing the public sector to define its needs as service outputs that can be adequately contracted for in a way that ensures effective, equitable, and accountable delivery of public services into the long-term, and where risk allocation sharing between public and private sectors can be clearly made and enforced;
• The nature of the assets and services identified as part of the PPP scheme are capable of being costed on a whole-of-life, long-term basis;
• The value of the project is sufficiently large to ensure that procurement costs are not disproportionate;
• The technology and other aspects of the sector are stable, and not susceptible to fast-paced change;
• Planning horizons are long-term, with assets intended to be used over long periods into the future; and
• There are robust incentives on the private sector to perform.

Source: List modified but based on PFI, UK Government, 2005

It should be noted that whether a project is funded from domestic or foreign sources is not considered within VfM or the PSC, although the risks attached to each type of funding must be considered within the risk matrix.

For a project to make progress in the PPP cycle there must be positive answers to most of these questions and are thus undertaken in the best possible VfM way. Using the Toolkit will help ensure VfM.

This more pragmatic approach is suggested in the early stages of PPP in developing economies. It takes into account that while the PSC is acknowledged as important for example in the UK and Australia, its actual use in other countries has had very mixed success.

It has also been acknowledged that in order to test the PSC's ex ante accuracy, experience over a significant period is necessary which is not available in most countries. As a country gains substantial experience in the application of these guidelines, it is suggested that the case for a detailed PSC be reviewed.

Further, the report by CEPA below recommends that undue emphasis should not be placed on the PSC because it is only one factor in the PPP decision making process.

Value for Money Assessment Guidance. UK Treasury, 2006

Public Private Partnerships in Scotland, Evaluation of Performance, 2005, CEPA.

Quantification of the PSC

A Public Sector Comparator (PSC) is used by a government to make decisions about VfM in PPP projects. It involves testing whether private investment proposal offer value for money against each other and in comparison with the most efficient form of public procurement. PSC provides a benchmark for estimating value for money between alternative bids and between hypothetical public procurement.

Risk is at the heart of all PPP projects. PPP projects are all about the treatment of risk and uncertain costs. Hence, the importance of the identification, allocation and mitigation of risks within the PPP process. Allocation of too much risk to the private sector will almost certainly result in downstream financial problems, just as allocating insufficient risk is not obtaining all the advantages of PPP.

It is not always easy to understand that if too many risks are transferred to the private sector, value for money will decline since the premium demanded by the private sector in compensation will outweigh the benefits of PPP.

Too much or too little risk allocation will not ensure maximum VfM. This is the link to VfM. If there is too little risk transference, and therefore too few benefits, why use the private sector when their higher financing costs are not likely to be outweighed by other benefits. If there is too much risk transference, the private sector will demand higher returns i.e. excessive premium which will result in lower or even negative VfM i.e. if negative, it should probably be done as a public procurement project.

Risk is about uncertainty which includes potential for gain and exposure to loss. Risk has a cost but its uncertainty makes it difficult to identify and estimate this cost, either easily or exactly. Public Sector Procurement tends not to include the potential cost of risk, e.g. time and cost overruns but the private sector generally includes risks in cost estimates and, more importantly, takes the maximum action to reduce and/or avoid such risks.

Public-Private Partnerships: Affordability, Value for Money and the PPP Process Frédéric MARTY CNRS – GREDEG – University of Nice Sophia-Antipolis OFCE – Innovation and Competition Department /OECD - Working Party of Senior Budget Officials

Value for Money and the Public Private Partnership Procurement Process October 2007 Irish Government

The UK Treasury provides detailed spreadsheets which allow a quantitative assessment of the level of Value for Money generated by a PPP project. This guidance requires quite detailed inputs.

Source: Financing Roads in Great Britain Peter Mackie and Nigel Smith Institute for Transport Studies University of Leeds

Partnerships Victoria Technical Note also provides a comprehensive report on how to prepare a VfM and PSC.

Public Sector Comparator-Technical Note; Guidance Material. Partnerships Victoria 2001

The advice in this Note is detailed and covers definitions, use, the main components in the preparation of a PSC, risk, taxation, a detailed numerical example and discounted cash flow methodology and PSC related statistical techniques.

The above Note also shows that the public sector procurement cost will be made up of;

• The costs of risks retained by the government
• The 'Raw' or basic costs and revenues of the project (All direct and indirect capital and operating costs and revenues)
• Adjustments for treating all public and private bids on the same basis e.g. tax
• The cost of transferable risks

In the above WB reference, H Kerali proposes a consistent process to Partnerships Victoria but at a more summary level which may be useful to prepare an analysis of the PSC, with the following data being required:

1. Risks and sensitivity

Risk matrix and Sensitivity analysis: A risk matrix is constructed through;

• Identification of risks involved in the project;
• Assessment of the impact of these risks;
• Assessment of the likelihood of such risks arising; and
• The calculation of the financial impact and ranges of possible outcomes;
• Sensitivity analysis is also undertaken which assesses the impact of changes in key variables on the financial outcome;
• Risk, sensitivity and probabilities allow estimates to be made of the impacts and likelihoods of individual risks. Monte Carlo simulation (Module 2 -> Risk) is most often used for this; The result provides the "most likely outcome" of a PPP modality.

2. Project Costs and revenues

Capital costs: These should reflect the full resource costs of the project, including opportunity cost of public assets used in the project, and adjusted for risks.

Operating costs: The whole life cost of maintaining the asset to the same standard as required from the Private operator.

Projected revenues: Included only if bidders will be allowed to set tolls.

Discounted cash flow: Selection of the Discount Rate is the most important issue and should represent the real opportunity cost of capital, adjusted for inflation (& subsidies, if any), for public projects.

Government issued bonds can be used as a guide. But note that the discount rate is not the interest rate of private finance!

3. Other Adjustments

Loans from the IFIs are generally subsidized and need to be adjusted to reflect commercial ratings. As noted in Partnerships Victoria other adjustments may be needed to ensure 'like with like' is the basis of the comparisons.

Example of PSC for Proposed Motorway Project

In the below-referenced presentation, H Kerali provides an example of a PSC for a motorway project. This is described below.

Public Sector Comparator for Highway PPP Projects Henry Kerali Lead Transport Specialist, World Bank 2006

Project characteristics: Motorway project involves the design, construction, operation and maintenance of a high quality motorway.

Private sector bidders are expected to:

• undertake the detailed design and construction of the Motorway to the requirements of the Client
• procure finance for the associated capital costs; and
• operate and maintain the Motorway to the requirements of the Client over a concession period of 30 years

Cost estimates (see table):

• Initial estimated construction costs by the Client = EUR 388 million
• Estimate of the Client's overhead costs = EUR 49 million
• Total capital cost to the Client = EUR 437 million
• Estimated construction period = 3,5 years
• Past history of road construction: Cost over-runs range from -11.5% to +138%, average =+44% (adjusted for inflation over construction period)
• Construction duration ranged from -27% to +230%, average= +84% of original estimate (cost is included in over-runs)
• Expected value of the cost overrun is EUR 172 million (44%),
• Risk adjusted total estimated capital cost = EUR 609 million
• O&M cost estimates
• No previous experience of public O&M costs to the specified standard
• Estimated annual costs for public O&M to the same standard ranged from EUR 1.37 to 2.27 million, with an average of EUR 1.45 million
• Economic and social costs of road closure for periodic maintenance assumed between 4% to 6% of total project benefits
• Government payments
• Capital cost contribution to the project = EUR 110 million
• Availability payments by the Client to Concessioners comprises fixed and indexed components in both local currency and EUR , with allowance for lane closures during periodic maintenance
• Weighted availability combines both local and foreign payments assuming long-term currency inflation
• Total availability payments = EUR 427 million over 30 years

The results show in general terms that the PPP modality is Value for Money under Bid 1 i.e. there would be a saving of EUR 22.4M over the public procurement cost.

Experience of the European Investment Bank and the Public Sector Comparator (PSC) and Value for Money (VfM).
In ten EIB projects evaluated, in only two countries had there been a formalized PSC process, although a third had used an ad hoc system. The Bank did not normally review the PSC, although the assumed cost and benefit figures were often used for the Economic Return (EIRR) calculation. However Bank economists said that they would encourage the Promoter to make use of a PSC in its own review of alternatives.

The Bank did not normally consider whether a particular PPP structure offered VfM compared to other possible structures. The exception to this was a motorway project, in which EIB carried out a VfM exercise which showed that, in economic terms, the chosen structure was not the best option.

It is also notable that a PSC was not used for one country's motorway program, and a subsequent review by the national audit office pointed out that the program had not even been preceded by an assessment of VfM.

Application of the PSC

The UK was one of the first to introduce quantification of the PSC and it has had some substantial benefits but not always in the areas it was designed for. For example, it made the argument stronger in some cases for the PPP program and also reinforced assessment of the operational parameters of a particular PPP project, thus avoiding political pressure to implement projects in less than effective ways.

However, in recent years there has been growing criticism of the PSC in the UK and has been seen as another cost burden rather than adding anything substantive. Reference, Roe and Craig, below argue that it should be scrapped because, in their view, in a PSC analysis;

• Data cannot be accurately obtained
• Costs are not always adjusted for some major risks
• There is no consensus on the discount rate
• Data and assumptions can be manipulated
• High PSC preparation costs and time consuming
• Futile second guessing by government and consultants as to what will happen in hypothetical future situations
• Impracticability of cancelling projects by the time bid data is available
• If public resources not available, PSC of little use

Consequently, this has led to two situations;

• In developed countries like the UK, there is still a move to softening the use of the PSC with simpler procedures such as standard spreadsheets that can be used in-house by departments.
• In the developing world, the original logic behind the PSC remains valid, but needs to be applied in a more practical way. It is suggested that governments and PPP advisors need to ensure that;
  • project preparation is undertaken soundly and especially the risk management aspects,
  • that the optimism bias and political pressure is countered and
  • that possibly only a selected number of examples are subject to quantification of the PSC.

However, the main concerns of whether the project offers value for money and what are the key advantages and disadvantages of a particular project under PPP still need to be analyzed.

It is possible that a PPP project is of such a magnitude in a small/medium country that it will have major macro economic impacts, not least absorb most of the supply of capital, raising interest rates and hindering smaller projects. Analysis of such risks will be the role of the Ministry of Finance/Risk Management Units/Central PPP units.

If it is decided that the macro impact is too great, the project could be deferred, phased, downsized or transferred back to the public sector program or a combination of these measures. While this is possible for energy, dam and similar, for highway projects this may be a relatively uncommon occurrence but for which, if it does occur, the project planners will need to consider the macro economic implications at an early stage.

As discussed above, using the PSC in any meaningful way according to its original objectives is probably not very feasible. However, PSC can be used as an aspect of general project appraisal and used to ensure or reinforce better project design and to support negotiations.

Following best practice, as described in the Toolkit and applied to each country flexibly, VfM should ensure that, at the very least, proposed PPP projects have an absolute net benefit for the country concerned.

Is the Public Sector Comparator right for developing countries? J Leighland and Chris Shugart, PPIAF Gridlines. 2006

Review of Partnerships Victoria Provided Infrastructure. Fitzgerald, Peter. Final Report to the Treasurer, State Government of Victoria, Australia. 2004.

Public Sector Comparator. Supplementary Technical Note. Department of the Treasury and Finance, State Government of Victoria, Partnerships Victoria. 2003.

Reforming the Private Finance Initiative. Roe, Philippa, and Alistair Craig. London: Centre for Policy Studies. 2004.

Public Private Partnerships-Guidance Material: Framework. Queensland Government 2002

Public-private Partnerships in the Road Sector; Caroline Visser International Road Federation 2008.