Major IT projects often involve large business change initiatives. And, no matter the size or the complexity of the undertaking, you would hope that all of your business outcomes will be delivered on time, to budget and within the resource constraints at your disposal. However, progressing such a project to a positive conclusion is remarkably challenging, and this statement is true whether applied to projects in the public or the private sector.
Such projects require meticulous planning and careful management to ensure their success, and never is this more important than when the safety of potentially millions of people is reliant on the outcomes of this work. In these circumstances governance has to be strong, or you increase the risk of the project failing and – worse – risk the well-being of individuals when they are at their most vulnerable.
With this in mind, our article this week focuses its attention on NHS 24’s project, public healthcare’s new £117m IT system, the use of which has had to be suspended just 10 days after going live on the grounds of ‘patient safety’.
The Scottish Health Board, who are responsible for the delivery of these services, began using the system on the 28th of October 2015. It reportedly crashed shortly afterwards.
After further attempts, the Board have decided that they had no other choice than to revert to their legacy system. A relaunch of the system is expected in 2016 after the completion of remedial work.
NHS 24 timing and budget
This project was considered to be one of the most important the government was to introduce, but as it was delivered two years behind schedule and reportedly £41.6m over budget (an increase of 55% on the original estimate) it will be important to review what lessons can be learned from the experience to ensure greater efficiencies in future.
The Auditor General for Scotland has done just that by producing two reports – one for 2013/14 and another for 2014/15, outlining her findings and the Board’s proposed corrective actions.
The Auditor General’s review of NHS 24
The NHS 24 system was designed to handle patient calls, not unlike the 111 service provided by NHS England. But it has been dogged by issues over its five-year development life.
It’s understood that back in 2011, at the procurement stage of the project, the system was split into two lots. The first, for software and applications, was won by Capgemini, and the second, for infrastructure was won by BT.
But if these lots were to be procured in silos, how were Capgemini and BT going to communicate to ensure that their respective solutions would complement one another? Ideally, a team would aim to work collaboratively to achieve the best value and the best outcomes for the project.
The Auditor General’s 2014/15 report makes reference to this when she states that:
“…the Future Programme Director identified that the Board needed to recognise and address its role as Systems Integrator… ensuring that those systems function together. This role demanded a significantly more robust engagement with the external suppliers…” (2014/15 audit)
So, from the beginning, the roles and responsibilities in delivering this change and ensuring a fit for purpose outcome from the project, required more clarity and definition that it appeared – from the outside – that it had received.
The auditor recognised that one of the key problems that permeated this project was that “the use of a team whose experience was not aligned to the management and delivery of such a large and complex contract”, which led to a “…flawed procurement and contract preparation, further undermined by unrealistic timescales and poor contract management”.
Before everyone starts beating the drum on ‘another failed public sector IT project’, it’s important to point out that in our experience of being asked to deal with hundreds of major projects ‘after the event’, more private sector projects go wrong than public sector ones. It’s just that the media do tend to talk more about it when it’s the public money that’s been at stake. As our project recovery work is fairly evenly split between the two sectors, our experience is relatively representative.
So, to be fair to the Board of NHS 24, this theme of over-schedule, under-resourced and over-budget is not uncommon. That does not provide an excuse, but it does put into context that these projects are challenging to manage no matter who is in charge. IT projects, due to their complexity often seem to run into similar troubles, especially when the individuals or teams in place have not had to deal with a business change project of this scope or size before. In situations like this it is often the case that critical aspects can be overlooked, making it even more difficult to achieve a fit for purpose outcome.
A team in this position is naturally likely to lean heavily on their suppliers to help guide the process, relying on their expertise in having worked on similar projects in the past.
This too is alluded to in the Auditor General’s report where she points to the:
“over-reliance on suppliers for the testing and evaluation of technical aspects of the contract”.
Specialist suppliers have an implied legal responsibility as ‘experts in their field’ to tell the client if something isn’t quite right or is likely to result in a mismatch with their expectations, both pre-contractually and post-contractually. This is known as their ‘expert responsibility’.
By leaving the vendor ‘to it’, the client can avoid eroding this responsibility from a legal standpoint. So, in principle, there’s nothing wrong with the team being overly-reliant on the vendor’s advice. However, there is a caveat to that. The client can’t just ‘leave the project to fail’ because it’s the supplier’s responsibility. That would be negligent behaviour.
Where (and if) the supplier isn’t achieving the right outcomes, the client has to realign its management process to enable the supplier to achieve those outcomes – and in particular, that includes managing its own team where its own behaviours do not align to support the supplier in their endeavours. An example of this would be where a client-side team gets too involved in telling the supplier ‘how’ to do its job, when they should be focusing on ‘what’ needs to be achieved instead.
From the Auditor General’s comments, it also seems that some responsibility for the issues faced on this project may be laid at the client’s own door. It looks as though the relationship may have been poorly managed, with business outcomes and ‘vision’ not being clarified well enough at the outset – preventing the supplier from being able to advise appropriately. This relates directly to the Auditor General’s finding that there was: “poor programme definition combined with ongoing and significant changes to scope and functionality, leading to increased development costs”.
Furthermore, when problems began to arise, it seemed that the governance structures that were in place were not fully aligned, with the Auditor General stating that the: “governance and escalation processes relevant to the current implementation not working fully to allow early identification of issues by the Board”.
It was also noted that “In June 2014, NHS 24 appointed a Future Programme Director. This followed five changes of personnel overseeing the Future Programme between 2009 and 2014”.
In a board meeting held in 2014, the Future Programme Director identified the comment we made earlier in this article in respect of the Board understanding its systems integrator role in view of how it went about the procurement, ongoing management and governance of this business initiative.
So while heavy criticism is levelled at the team who managed the programme, we must be mindful that it appears a number of the supporting mechanisms appear not to have been appropriate either. The report seems to indicate that failings at board level to understand their own role and responsibilities, could have led to the creation of a team that was not as fit as it could have been to manage the project in the first place. Yet, it took three years and several changes of project leadership before the team made the Board aware of this in November 2014. And throughout that period the governance structures used to identify developing issues have been reported as being ineffective. In other words, a domino effect that was likely to have a larger cumulative impact on programme delivery.
When looking at high-level figures it’s easy to be critical, but in spite of the alleged inexperience of the team, poor project definition, ineffective governance and churn in project leadership, the team did manage to hold the project together to its completion.
Good behaviours over intention
The important thing, however, is that once these problems were realised, the Board sought to build a more robust, skilled and experienced team, and did so. They didn’t just talk about it.
We talk at length about the importance of the Intelligent Client Function (ICF), an expert team with wide-ranging expertise across the contract structures, technicalities and project management of such enormous undertakings. They are essentially the ‘central intelligence’ that helps to drive the project; and this is precisely what NHS 24 did produce in the end.
NHS 24 began working with NHS National Services Scotland (NSS) to build a specialist team that included a Systems Architect and a Test Manager. They had also established a Programme Management Office (PMO) within Capgemini’s offices in Glasgow where weekly joint-executive meetings were held. By February 2015 they believed they had the right expertise in place to guide the project and the appropriate governance to foster the collaboration and trust necessary to recover the project and drive innovation. In light of all the challenges they faced, this approach deserves good merit.
Given the challenges that have arisen, and the ramifications of the delays and overspend, it’s important that this information is used to encourage better governance going forward and that any lessons learned are implemented in future projects. The Auditor General says as much in her 2014/15 report. She recommends that the Board undertake a comprehensive ‘lessons learned’ exercise covering the entire programme. In particular, she outlines the following areas for careful scrutiny:
-
1. programme definition and approval
-
2. programme management, including roles and responsibilities, resourcing and the experience and qualifications of those involved
-
3. procurement processes, including support received
-
4. technical evaluation and management
-
5. overall governance and risk management
-
6. financial controls and reporting
So across these six areas, what lessons can be learned?
1. Programme definition and approval
Defining precisely (quantifying) what the programme will seek to achieve and how it will achieve it is crucial. The traditional CIPs procurement method focuses on inputs in an effort to complete the process quickly. When dealing with transformational projects involving mission-critical systems, however, much more time needs to be spent pre-procurement, understanding exactly what business outcomes and objectives the organisation will be able to achieve once the implementation is complete. This provides internal stakeholders with the clarity and understanding of what clear benefit and operational change the project will provide once it has been completed, and agreement can then be reached on this. It also makes clear to the supplier what is expected of them to achieve the end result of the project and in what context.
Once that’s documented, teams can work backwards to map precisely how a new solution will enable this change; once handed to the vendor, there should be no question as to what they’re expected to achieve, and they can not only leverage their expertise to help you get there, but they will be accountable to do so.
More commonly, however, procurement teams can get caught up in the technical aspects of a project. When you begin defining technical inputs to deliver an outcome, you’re often stepping into the vendor’s territory – restricting their options to deliver a fit for purpose solution that will deliver your envisaged outcomes. This isn’t a problem if you understand the risks and you have the internal technical expertise to manage an input-based project. With some integration projects, they are often a mix of input-, output- and objectives-based requirements, so you just need to be clear what the delineation is between those aspects, manage and contract for them accordingly.
2. Programme management
When managing such a complex programme, it’s critical that the team responsible have access to their own domain expertise and the qualifications necessary to consider both the technical aspects of the solution and the big picture/impact of the project across stakeholders and the wider organisation.
This team should be involved from the very beginning, from creating the business case, through procurement, and throughout the life of the project. We refer to this as the ICF (Intelligent Client Function) team. Within this team, the roles and capabilities must be clearly defined, and – if possible – audited, to ensure the skills you have will drive a fit for purpose outcome. This team must be well resourced, otherwise they’ll be unable to marshal the full depth of their experience to support the programme to achieve the right business outcomes.
3. Procurement processes
Undertaking the procurement for a transformational IT project is unlike most other procurements. Ideally, there should be a great deal of research and pre-contractual due-diligence that goes into the process long before contracts are awarded. It is here that many ‘big IT’ problems begin, where a rush to procure systems and services leads to misalignments at a deeper level that the procurement team hadn’t considered. Importantly, it’s much more difficult to back-track once a contract is awarded.
In an effort to win a contract, some suppliers will promise the earth without necessarily taking the time to assess the scope of a project on a deeper technical level. Make sure you have a pre-contractual scoping process in place with supplier(s) so that you get the right advice on what is achievable with your project, what is not and what compromises you will have to live with. If this is structured in the right way, both process-wise and contractually, supplier(s) will provide you with much better holistic advice as to how you can get to a fit for purpose solution.
4. Technical evaluation
We always advocate that where you are looking for a provider to deliver a solution to achieve business outcomes, that the requirements definition, market engagement, procurement, contracting and management process differ to those needed when you require a provider to work to a specific set of input-based requirements. In the former, you are reliant on the provider for the ‘how’ of what they will do to deliver a solution that will help you achieve the outcomes you want from the project. In the latter, you will need specific and detailed technical domain expertise so that the process of delivery is achieved and managed in the right manner.
However, even where you are contracting for a set of business outcomes or objectives (the former scenario), it can be enormously helpful to have access to specific technical domain expertise as part of your Intelligent Client Function. Not necessarily as a full-time resource – but individuals you can pull in as and when necessary. This is not so you can ‘tell the supplier how to do its job’ when matters go off track, it is purely for you to have that assurance and validation that what you are being told by the supplier (or in fact your own team), is supported by evidence as being correct. Sometimes, and for the right reasons, teams can get lost in the fog of delivery. Technical evidence used in the positive sense (for support and enablement) rather than in the negative sense (to prove someone is liable), helps remove the emotion and subjectivity when projects aren’t moving in the direction you need. You then use the evidence to change the direction of the project so that it meets your goals.
5. Governance and risk management
With great spend, comes great responsibility. With so many interconnected activities to monitor and support, the inherent risks can be great. To manage these risks and to resolve issues before they can have costly consequences, a robust governance structure should be implemented to allow for the effective handling and resolution of issues. Where issues arise, it needs to be clear how these should be dealt with and what process ought to be followed for escalation. If this process is unclear, a lot of issues can be left to bubble under the surface, culminating into much bigger problems later down the line.
6. Financial controls and Reporting
In our experience (your own experience may be different), many major projects go off the financial track because of three key issues:
1) A lack of clarity as to the business outcome the project is supposed to deliver so internal stakeholders are misaligned over their expectations and suppliers aren’t clear about what is expected of them.
2) Inadequate pre-contractual due diligence by selected vendors to understand what is expected of them from the client’s business case, and to understand the business case clearly enough to be able to inform or ‘warn’ the client about what it will or will not receive, and what compromises it will have to live with, given the financial, time and resource constraints of the project (or the organisation).
3) A poorly resourced Intelligent Client Function (ICF) team that doesn’t have the appropriate experience, programme, business, management, contracting reporting tools to ensure the fit for purpose alignment that the project’s needs.
Conclusion
With public sector projects being under such great scrutiny, it is easy to look at the inherent challenges and point the finger. But with so many factors at play in these large and complex IT projects, it is remarkably difficult to appreciate the sheer scope of the difficulties faced by the teams responsible for delivering these transformational programmes.
What is important is that we take stock of how the project was managed, so that we can apply these ‘lessons’ to our own projects. Despite the challenges faced by NHS 24, they have demonstrated a very progressive attitude to change and we hope that this results in a positive outcome for them when the system comes back online in 2016.
If you are interested in understanding more about pre-contractual due diligence, download our Improving IT Projects whitepaper below.
