Assessing risk in ICT/digital sourcing guidelines

Updated: 13 Jul 2021

Contact owner

Guidance for buyers of ICT/digital goods and services on how to navigate risk when buying from the ICT Services Scheme.

1. Purpose

These guidelines aim to assist NSW Government agencies in the identification of whether the procurement of an ICT/digital solution is low risk or high risk.

They aim to support agencies existing internal processes, policies and templates in relation to risk assessment in ICT procurement.

1.1 Objectives

The key objectives of these guidelines are to:

assist agencies’ considerations in identifying and assessing risks in ICT procurement
create awareness as to the importance of identifying and assessing those risks in accordance with agencies’ internal processes
provide non-exhaustive and non-mandatory definitions of low-risk and high-risk ICT procurement
provide greater assurance to agencies in relation to which ICT Purchasing Framework to use based on the level of risk.

1.2 Scope

These guidelines are directed at all officers, consultants and contractors procuring ICT/digital solutions on behalf of NSW Government.

These guidelines have been developed specifically for the procurement of ICT/digital products and services under the ICT Services Scheme and designed to assist agencies select the most appropriate contract under the ICT Purchasing Framework.

Risk management, like other management systems, should be designed to meet an agency’s specific needs. Therefore, these guidelines do not replace existing agency-specific or NSW Government policies, procedures or guidelines relating to identification, mitigation or management of risks. They only complement them.

Core Requirement 1.2 of NSW Treasury’s Internal Audit and Risk Management Policy for the NSW Public Sector (TPP 15-03) requires department heads and governing boards of statutory bodies to establish and maintain a risk management process that is consistent with the current Australian/New Zealand (AS/NZS standard on risk management).

Standards Australia has adopted the international standard (ISO 31000), which it has titled AS/NZS ISO 31000: 2009 Risk management – Principles and guidelines.

2. Key considerations

2.1 Risks in ICT/digital sourcing

NSW Government agencies rely on information and communications technology (ICT) and digital technologies to successfully carry out their missions and business functions. The ongoing development of ICT/digital solutions has resulted in increased complexity, diversity, and scale of the way NSW Government agencies procure these.

As a consequence, the potential for risk has also increased. Potential threats can include purposeful attacks, environmental disruptions, human/machine errors, and structural failures. Accordingly, as an agency, you need to be mindful that when procuring an ICT/digital solution there might be risks that could impact:

agency’s strategic objectives at an enterprise level
agency’s ICT operations and service delivery on a day-to-day basis
the delivery of an ICT project on time, budget, scope and quality
when applicable, other NSW Government agencies or other jurisdictions
the public at large.

Effective risk management in ICT/digital sourcing is therefore crucial.

Furthermore, risk management should be ongoing. This means risk assessments should occur throughout the ICT procurement life cycle, from planning, through sourcing and on into management.

In terms of identifying and analysing risks, we recommend you follow this process:

First step: Identification of risks in ICT/digital sourcing

Follow your internal risk management frameworks, policies, guidelines and templates to identify all the applicable risks associated to sourcing ICT/digital solutions.

In most cases, these documents would have been designed by your agency’s relevant team to:

build internal understanding of risks and their management to better achieve value for money across your cluster
manage their relevant ICT supply chain
inform employees, consultants and contractors about risk and the importance of its effective management in their procurement activities
establish effective oversight, transparency and accountability for risks within your agency in accordance with your agency’s needs
embed risk management as a core component within your agency’s management systems and business processes.

To support your agency’s risk assessment capabilities to identify risks related to your ICT/digital sourcing activities, you need to be mindful of:

The future effect of uncertainties on the objectives related to your ICT procurement; which may or may not happen and for which you can plan for based on their likelihood and potential consequences.

Source: ISO 31000:2009

In other words, when establishing the planning, sourcing and managing strategies related to your procurement, you should at the same time identify the potential events that may affect your ability to achieve the desired objectives of that procurement and think about the possible future consequences at different levels.

From an ICT/digital sourcing perspective, we recommend you break it down into cause, risk event and consequence as follows:

Cause. What could trigger a risk event?. For example, ageing hardware, end of life or support
Risk event. What could happen? For example, server XYZ failure
Consequence. What is the impact or effect on objective(s)? For example, application 123 outage causing business process disruption.

By understanding potential risks in your ICT/digital sourcing activities and finding ways to minimise their impacts, you will help your agency avoid these consequences from happening in the first place and/or recover quickly if an incident occurs.

Second step: analysis of risks

Once you have identified the risks associated with your ICT/digital sourcing, then analyse those in terms of impact and likelihood.

In accordance with Treasury’s Risk Management Toolkit for NSW Public Sector Agencies, agencies should determine which approach(es) they will use to determine their likelihood and impact tables consistent with their own internal frameworks and policies used by the specific Cluster/agency’s risk management function.

Likelihood

In general, likelihood is a weighted factor based on a subjective analysis of the probability that a risk event could occur impacting not only your procurement’s objectives but also your agency, other agencies and others.

When procuring an ICT/digital solution, likelihood determination should consider factors which include for example:

Threat assumptions that articulate the types of threats that the system or the component may be subject to, such as cybersecurity threats, natural disasters or physical security threats
Exposure of components to external access
Identified system, process, or component vulnerabilities
Empirical data on weaknesses and vulnerabilities available from any completed analysis (that is, system analysis, process analysis) to determine probabilities of threat occurrence.

Impact

In general, impact is the effect including the degree of harm that the risk event could have on your procurement’s objectives, your agency’s mission. Impact is likely to be a qualitative measure requiring analytic judgment.

The following tables are provided only as a guide for you in understanding the likelihood and impact of risks:

Table 1. Likelihood
Likelihood	Description
Almost certain	High likelihood of risk events occurring (in most circumstances).
Probable	A risk event is likely to occur (to be expected).
Possible	Would not be surprised if risk event occurred (foreseeable/capable of happening at least once).
Unlikely	The risk event could occur at some time but is unlikely (not to be expected).
Rare	Within the realms of possibility but extremely unlikely to occur (exceptional circumstances only).

Table 2. Impact
Impact	Description
Very High	The consequences would create severe problems to the agency or its customers.
High	The consequences would create major problems to the agency or its customers.
Moderate	The consequences would create moderate problems to the agency or its customers and so, the procurement could be subject to significant review.
Minor	The consequences would affect the efficiency or effectiveness of some aspects of the ICT solution but would be dealt with internally.
Negligible	The consequences are dealt with by routine operations.

Risk matrix

A risk matrix then may be used to determine the level of risk by combining its consequence and likelihood of single risks. We recommend that if your agency has risk matrix templates available, you use these as appropriate.

Below is an example of a risk matrix table with different escalation points, provided as a guide only.

Table 3. Risk matrix
Impact / Likelihood	Negligible (1)	Minor (3)	Moderate (5)	High (7)	Very high (9)	Risk
Almost certain (5)	Minor (5)	Moderate (15)	High (25)	Extreme (35)	Extreme (45)	Very high (26 to 45)
Probable (4)	Minor (4)	Moderate (12)	High (20)	Extreme (28)	Extreme (36)	High (18 to 25)
Possible (3)	Minor (3)	Moderate (9)	Moderate (15)	High (21)	Extreme (27)	Moderate (7 to 15)
Unlikely (2)	Minor (2)	Minor (6)	Moderate (10)	Moderate (14)	High (18)	Minor (2 to 6)
Rare (1)	Negligible (1)	Minor (3)	Minor (5)	Moderate (7)	Moderate (9)	Negligible (1)

Based on the above assessment, your ICT/digital sourcing can then be rated. This rating will assist you in identifying whether your procurement is low-risk or high-risk.

2.1.1 What is low-risk ICT procurement?

Low-risk ICT procurements might include procurements where the impacts or potential impacts have been identified as minor or negligible.

It might also include procurements where impacts have been identified as moderate and these moderate impacts are very unlikely to materialise.

As a guide, some of the factors to consider when determining whether an ICT procurement is a low-risk procurement include:

the sourcing complexity is extremely low, very low or low – for example, a routine procurement method for a routine solution
the technical complexity is extremely low, very low or low – for example, a proven solution with no inter-operability requirements
the level of priority is low, very low or low – for example, solution is not a direct enabler in the relevant agency’s strategic plan
the interdependency with other factors is extremely low, very low or low – for example there is no external dependency (federal, local, private or interagency) and there is no interdependence with other projects or services
the agency’s capability with regards to the proposed solution has been previously demonstrated – for example, it is a multiple recurring project or BAU
the supply chain risks for the particular solution are easily identifiable – for example, the agency has appropriate level of visibility and understanding of how the technology is developed, integrated and deployed, as well as the processes used to assure the integrity, security, resilience and quality of the relevant products or services.

This definition is not intended to be exhaustive nor mandatory.

It is only intended to provide guidance to identify ICT procurements that are low risk.

Agencies are reminded to follow their internal processes and that ICT procurements need to be assessed against their own risk management framework. This is because the perception of risk level and complexity can be quite different from agency to agency

An example of a low-risk ICT procurement is one that might involve some of the following characteristics:

the supply of mature software ('commercial off-the-shelf' products or solutions) that require minimal to no customisation and have no security concerns
the supply of mature hardware where there are no security concerns
in a technically routine manner that requires little or no integration or dependency with other systems
the use of a mandatory contractual framework that does not require non-beneficial variations
involving few or no subcontractors
with a schedule requirement that is achievable
when it includes pre-existing in-use product or service that requires only a minor adjustment, enhancement or addition, for example an add-on to enable the earlier product or service to mature or be fully implemented correctly, especially when not part of a complex or high-risk core system
the products or services are standard in the industry
the market for that specific ICT solution is mature.

In general, a low-risk ICT procurement will include procurements where a whole-of-government risk assessment has been implemented and classified as low risk.

2.1.2 What is high-risk ICT procurement?

High-risk ICT procurements might include procurements where the impacts or potential impacts have been identified as high or very high.

It might also include procurements where impacts have been identified as moderate and these moderate impacts are probable or almost certain to materialise.

As a guide, some of factors to consider when determining whether an ICT procurement is a high-risk procurement include:

the sourcing complexity is high or very high – for example, multiple suppliers are involved in the delivery of the solution with varying service levels
the technical complexity is high or very high – for example, extremely new or emerging technology proposed or an unproven solution or complex inter-operability requirements across multiple platforms
the level of priority is high or very high – for example, a mandated priority project in documents such as the NSW Budget, Premier’s Priorities, State Infrastructure Strategy, NSW Government Digital Strategy, Election Commitment, or is a response to a legislative reform
the interdependency with other factors is high or very high – for example, the solution is fully interdependent on other projects or services
the agency’s capability with regards to the proposed solution has not been previously demonstrated – for example, very few numbers of projects of this type previously delivered over the last 5 to 10 years.
the supply chain risks for the particular solution are not easily identifiable – for example, the relevant agency has decreased level of visibility and understanding of how the technology that they acquire is developed, integrated and deployed, as well as the processes used to assure the integrity, security, resilience and quality of the relevant products or services.

This definition is not intended to be exhaustive nor mandatory.

It is only intended to provide guidance to identify ICT procurements that are high risk.

An example of a high-risk ICT procurement is one that might involve one or more of the following characteristics:

emerging software technologies
developmental hardware
being integrated into complex and dependent systems
multiple layers of contractors and sub-contractors providing the services in a complex commercial and contractual framework
multiple non-beneficial variations are required in the contractual framework
an aggressive or unrealistic delivery schedule
there are privacy or security concerns
it is a bespoke solution
it is a time-and-materials engagement rather than a fixed price
where there are complex consumption-based mechanics (for example, procurement of mobile phones might seem low risk, but telecommunications expense management is critical in controlling consumption-based risk/value).

2.1.3 What about ICT projects?

Depending on the actual project and specific needs, agencies might use traditional waterfall project methodologies or lean principles and agile techniques for their project implementation. They both have advantages and disadvantages. Choosing the right path is out of the scope of these guidelines.

Risks profiles for each approach will vary project by project. All ICT projects have some risk, regardless of your project approach.

Some of the considerations in terms of risk management for each project approach include:

Table 4. Agile against waterfall approach
Agile approach	Waterfall approach
Agile is simple to understand in principle but hard to do well in practice. It requires real commitment and first attempts might not go very well.	Sometimes it is not easy to define requirements (upfront), especially early in some types of projects. The assumptions upon which early-stage plans are based may be very flawed and too often are taken as being based on certainty.
Agile requires high levels of collaboration and very regular communication between developers and users. This is always desirable but may not always be feasible and requires continual commitment and time.	In waterfall, the scope for invalid assumptions is unlimited. If you add this to the high cost of making changes later in a waterfall project, it is easy to see why some are very expensive, over budget and late. Too often, assurance of products being fit-for-purpose is demonstrated very late in waterfall projects.
Agile is very intensive for both developers and users. There can be reasons that may prevent this for example if developers work on multiple projects at one time.	Waterfall projects don't have to be but tend to be made up of 'teams within teams'. This can be a major risk to any project.
There can be less of a blueprint of what the final deliverable will be. This can make it harder to gain commitment to the project by stakeholders at the early stage.	Communication can be a far higher risk, especially when there is limited early review of outputs and deliverables or when one-way methods of communication are used to convey requirements
Agile can be challenging when there is a supplier-customer relationship. Customers typically want to know what they are getting for their money as early as possible. It can be far harder to estimate timescales and costs as there is less 'definition' to base estimates on.	The bigger, longer, and more complex the project, the riskier it is. Risk is highest at the end of a project.
Agile can be very challenging on much larger projects or where co-location is not possible. In Agile there can be a great reluctance (by some) to adopt or accept deadlines. Projects don't exist in isolation so when this happens it can be a real issue.	Traditional projects require time and cost estimates at the project start, when teams know the least about the project. Estimates are often inaccurate, creating a gap between expected and actual project schedules and budgets.

2.1.4 Risk assessment result in ICT procurement

Based on your risk assessment, you will be able to provide an overall rating to your ICT procurement.

The following table is provided as an example:

Table 4. Overall rating
Overall rating (as per table 3)	ICT procurement risk rating	ICT Purchasing Framework contract
Very high	High risk	Refer to MICTA/ICTA contracting framework. It is the contract to be used for high-risk ICT procurement where the total contract value is higher than $1 million (excluding GST).
High	High risk
Moderate	Low risk	Refer to Core& contracting framework. It is the contract to be used for low-risk ICT procurement where the total contract value is up $1 million (excluding GST).
Minor
Negligible

Once you are able to identify risks in your project and put plans together for them. You will be in a better position to handle these risks to ensure a higher probability of project success.

Even if a procurement was rated as a low-risk procurement, this does not mean the status is permanent.

An ICT procurement might have been incorrectly assessed as low-risk or the risk profile might change in the future.

Therefore, it is important that agencies have measures in place to allow them to have proactive contract and risk management on an ongoing basis.

2.2 Price vs risk

Agencies should not rely on the estimated dollar spend as a substitute for assessing whether the proposed ICT procurement is low or high-risk.

Cost associated with procurement is only one factor to assess the risk profile. For instance, the fact that the value of an ICT procurement contract is quite low (for example $20,000) does not mean that the procurement becomes a low-risk ICT procurement automatically.

If there is a risk that the ICT procurement will exceed the contract threshold, agencies should not split the value into lower-price components. Agencies should estimate the total price over the term of the contract including any optional extension periods and not a price per annum.

When determining the most appropriate contract to use under the ICT Purchasing Framework, agencies should consider the following scenarios:

A contract with a supplier may result in the need for related flow-on contracts with the same supplier or an alternative supplier. The cumulative price of the first contract and any potential flow-on related contracts may exceed $1 million (excluding GST).
The agreed scope of works may change during the term of the customer contract, resulting in an increase in contract price to greater than $1 million (excluding GST).

When agencies select which contract to use, they should factor in these potential scenarios.

In addition, agencies should be mindful of situations where poorly managed procurements costs might generate unintended risks.

These include for instance the procurement of ICT/digital solutions where a bundle deal is negotiated with a supplier and more items are purchased than currently required, on the basis of price.

For example, this could result in having extra licences or extra modules that are unable to be transferred to another agency.

In this regard, value for money does not mean the lowest price or deepest discount. Utilisation is key to value for money.

2.3 Mitigating strategies

Agencies should have mitigation strategies to reduce the risks and impact. Agencies should refer to their internal processes and guidelines in this regard.

One of the important tools your agency will have in their ICT/digital sourcing activities is a robust supplier risk management framework. For instance, mitigating strategies to manage ICT supply chain risks should be aimed at reducing the supplier risk impact severity or probability of occurrence.

An overall framework will enable agencies to manage ICT supplier risk throughout the sourcing lifecycle. In addition, a dedicated commercial manager for the supplier will also be key.

The tables below have been provided as an example:

Table 5. Example
No.	Question (refer to ICT/digital sourcing checklist)	Yes/No (if no, use MICTA/ICTA)
17	The agency has appropriate level of visibility into, and understanding of, how the relevant solution is developed, integrated, and deployed, as well as the processes, procedures, and practices used to assure the integrity, security, resilience, and quality of the relevant products or services	No

Table 6. Mitigating strategies
Mitigating strategy (to be conducted first)	Likelihood (table 1)	Impact (table 2)	Risk rating
The relevant team routinely requires suppliers to provide details of their own sourcing or outsourcing and offshoring arrangements	Unlikely	Moderate	Moderate (low risk)
The relevant team regularly monitors the operational, ethical and financial risk and performance of ICT suppliers	Unlikely	Moderate	Moderate (low risk)
The relevant team audits the supply chain to ensure compliance with the relevant regulatory and legal requirements	Unlikely	Moderate	Moderate (low risk)

3. Important tools

3.1 Within the ICT Services Scheme

In the first instance, agencies should refer to their internal policies, framework, processes and templates.

To assist agencies further, please refer to a number of tools we have available including:

ICT/digital sourcing checklist DOCX, 691.34 KB to assist agencies in using the Core& Contracts adequately based on risk identification
Risk Identification Toolkit XLSX, 90.3 KB to assist agencies in identifying relevant risks for their ICT/digital sourcing activities
Supplier Security Assessment Questionnaire XLSX, 40.4 KB. Used by agencies to ensure supplier's security risks are minimised.

3.2 Other important tools

In addition, NSW Treasury has developed a series of templates so that agencies can tailor them to their own needs: