In a recent post we outlined our four step minimum risk procurement strategy in which we discussed the creation of an evaluation framework. This is something that I cannot stress the importance of enough — not only is a good evaluation framework the only way to accurately measure the relative merits (or otherwise) of submitted tenders, it is a necessity for public sector organisations with regards to transparency and the avoidance of challenge in their procurement process.
With that in mind, in this post I want to focus solely on the creation of an evaluation framework and more specifically the evaluation matrices that seek to objectify and quantitatively score key aspects of tenders submitted for a strategic commissioning or outsourcing contract. With a well-assembled evaluation framework in your back pocket you can be sure of not only picking the right provider for your unique needs, but also of being able to justify your selection as being correct.
The Evaluation Matrix
In a nutshell, an evaluation matrix sets out to:
- Establish what your evaluation criteria are
- Set out how each criterion (and sub-criterion) will be weighted in terms of importance
One must drill down to such a level where criteria scoring can be done with no ambiguity and absolute transparency. Although your top level criteria (e.g. price, delivery, quality) may be broad, the underlying sub-criteria will leave little room for imagination. Examples of sub-criteria include:
- Cost of ownership of the solution (price)
- Robustness of the implementation plan (delivery)
- Quality of the fitness for purpose of the solution without customisation (quality)
These criteria can be preceded by “gateway” criteria. These are “deal breakers” that receive either a pass or fail score. Examples of gateway criteria would be a provider having insufficient indemnity insurance or a previous record of fraud or bribery. These gateway criteria should be considered up front so that time is not wasted assessing tenders that will automatically be rejected by default.
Each main criterion will attract a percentage weighting that reflects its importance to the procurement as a whole. Subsequently, sub-criteria will be awarded a proportion of that percentage. A finalised evaluation matrix would usually look something like this:
You will note different scoring methods on the far right of the spreadsheet. The definitions of these scoring methods are as important as the percentage weights of each element. Your evaluation matrix should be accompanied by appropriate definitions:
Determining Criteria and Establishing Scoring Methods
In order to produce an evaluation matrix that is fit for purpose, an organisation must involve all the key people and departments that will have any material involvement with the procurement and/or solution. Consider key users, people who will depend upon it, people who will input information into it or require information from it, and so on. Identify all touch points within the organisation and try to involve a representative from all relevant areas.
At this point you should hold a workshop (or workshops) in which the scoring criteria and their respective percentage weights are defined. The workshop(s) should serve as an educational tool to clearly establish the way in which the evaluation framework as a whole will function. If the procurement method being used has particular requirements or restrictions then these should be outlined and incorporated. At the first workshop stage no criteria have to be laid out — representatives can take the newly-learned information away so that they can collaborate subsequently to define, refine and agree the appropriate criteria and sub-criteria.
It is important to note that there may be additional subsidiary tender documents in addition to the evaluation matrix that aids stakeholders in scoring as objectively as possible. Further clarification of sub-criteria is encouraged if it helps in producing final scores and/or clarifying the thinking behind scores.
The key at this point is that no stone should be left unturned. All key players must be involved in the process and they all must be committed to including all relevant factors within the evaluation framework. There is no going back from this step — once the evaluation framework has been finalised and tenders have been submitted, the scoring must be done on the basis of the agreed framework and no amendments can be made.
Once the criteria and scoring methods have been finalised we advise that you then produce a clarification and evaluation guidance document that can be taken away and used to assist those conducting the scoring process. This document should explain the evaluation framework, provide clear instructions as to how criteria should be scored and set out how the incoming bids will be clarified, scored and the scores moderated.
The Tender/Evaluation Matrix Cross Reference
An evaluation matrix will be of little use if it does not align with submitted tenders — it is of vital importance that one reflects the other in such a way that systematic scoring can be carried out. It is a matter of personal preference as to whether you produce the tender template or the evaluation matrix first, but they must ultimately align before you go out to tender.
The key question to ask is this: based upon the structure of the template tender, are you likely to get back all of the information you require and will you be able to interpret that information consistently in order to produce objective scores that can be referred to in terms of picking a winning contractor and justifying that pick?
What you should aim for is a ‘grid’ in which the rows are made up of template tender headers and the columns are made up of evaluation criteria. Scorers should be able to cross reference one with the other so they know what elements of the tenders to consider when scoring any particular criterion. This can be of great assistance to scorers when working through large piles of bids.
The key is in making the scoring process an easy exercise in objective thinking, where scorers are working within a clear framework having well defined parameters and mapping of criteria to bid content. Consistency and objectivity of scoring across different tenders is absolutely vital — we’ll discuss more on how that’s achieved in your procurement process in a future post.
Ready for Tender
At this point you should be ready to issue template tenders (as part of the overall tender bundle) to providers for them to complete and return. Once in receipt of those tenders the next stage of the procurement stage kicks in. This involves potential clarification of the tenders followed by further collaboration between key stakeholders within the organisation to agree scores and determine the competition winner. This process deserves a post of its own and is something we will be discussing in the future here on the blog.
Photo Credit: just.Luc