Missing Data Is A Known Unknown

March 16, 2021
Procurement

There is a famous quote about the fragility of knowledge by Donald Rumsfeld, the hawkish US Secretary of Defence during the Iraq war:

“as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.”

Following the legal judgement against the UK Government for delaying publication of their contract notices, the question of how many contracting records are yet to be published is effectively a known unknown for the UK.

In relative terms, the UK publishes more data on procurement than almost any other country, but our inability to know how much contracting data is missing indicates how far the UK and other countries have to travel if we’re to achieve true transparency.

In an attempt to answer this question we’ve taken the last four years of contracting data for central government in the UK and compared it to the spending data for the same period. If we count how many suppliers have been awarded contracts each year between 2017 and 2020 we find an average of 6,401 suppliers have been listed each year. However, when we look at the spend data, the average number of suppliers being paid by central government entities during that period is over 31,852.

So an 80% difference.

When we reconcile this data to supplier records, something that takes significant amounts of work, including manual and automatic matching, we still end up with a 78% difference.

Does this mean that around 80% of award notices are missing? Not quite. This analysis can only indicate that there’s an awful lot of data missing, but it can’t say precisely how much data is missing.

The sheer scale of the differential indicates that there is a problem here. For instance a search on Contracts Finder, England’s procurement transparency database, for a prominent London Borough returns just two contract award notices. It simply isn’t credible that a council in London has let only two contracts in four years. The only reasonable explanation is that they have failed to publish the data required.

We know that the team at the Cabinet Office has been working hard to improve this data, but the responsibility for compliance lies with the publishers. It is not unreasonable to suggest that the Cabinet Office has better things to do, the publishers knows what they should be doing, they have the tools to do so, why can’t they publish this information?

Rumsfeld was roundly ridiculed for his quote, somewhat unfairly to my mind, trying find new ways to describe our collective ignorance seems like a useful endeavour, despite what else you might be getting up to.

Covid-19 has put a spotlight on all of our governments, their integrity is very much at stake and transparency is a chance to turn known unknowns into known knowns. Governments should think earnestly about the benefit of choosing collective enlightenment over collective ignorance.

If you’d like to discuss our government data or research capabilities, please get in touch.

August 3, 2021

New Carbon Reporting To Impact 1,000 Suppliers

New rules to help reduce CO2 in public contracting show that at least 720 suppliers will be required to provide a detailed...
August 3, 2021

Trade, Policy And Procurement Meet Again.

The UK's Trade and Agriculture Commission has published its final report, along with twenty-two recommendations, one of which is to direct the...
August 3, 2021

McGowan Government Enhances Aboriginal Procurement Policy

In 2018, Western Australian Labour Premier Mark McGowan introduced a landmark Aboriginal Procurement policy. The objective of the policy was to improve...
August 3, 2021

Are Trade Agencies Becoming More Like Hedge Funds?

Trade promotion, where a country promotes its companies and products to be bought by another, has a certain image. One imagines suave...
August 3, 2021

Meeting The Research Challenge

Our clients expect us to answer tough questions. Questions like, "what's the total value of public contracting worldwide?" or, "who's the leading...
August 3, 2021

US Consults On Social Procurement

The US Government's Office of Management and Budget (OMB) has requested submissions on the "methods and leading practices" to assess whether agencies'...
August 3, 2021

Harnessing Procurement To Tackle Climate Change

As experts in procurement and open data, we are pleased to see the recent announcement that global confectionery giant Mars is harnessing...
June 29, 2021

Western Australian Govt Rolls Out New Procurement Process.

We're always pleased to see governments reviewing their procurement processes with a view to greater transparency. The Western Australian Government has recently...
June 9, 2021

Commit To Net Zero To Win Government Contracts.

....So says the Cabinet Office, with new measures released on World Environment Day that will require businesses to publish clear and credible...
June 2, 2021

Major Procurement Project For Inverness Castle

This week sees Highland Council publish the main construction and refurbishment tenders for the Inverness Castle Transformation project, through Public Contracts Scotland...
May 27, 2021

Six Considerations When Marketing To Governments.

We might say marketing to the government is similar to marketing to any other target market, particularly another B2B target market. And...
May 17, 2021

Data Mastery.

It's easy to say that you're a 'world leading' enterprise. It's much harder to back it up. But we genuinely believe that...
May 4, 2021

Recommendations For Tackling Gender Disparity For Governments

Clearly, the UK Government is a leader in the field of open data and transparency: it is this commitment to transparency as...
June 2, 2021

Does Government Business Correlate To Gender Disparity?

As part of our Procuring Inequality report, we've analysed whether winning government business equates to a gender pay disparity. By linking gender...
May 5, 2021

The Worst Performing Suppliers To Government In The Gender Pay Divide.

As our analysis of the gender pay gap within strategic suppliers to government from our Procuring Inequality report, showed a broad overlap...

Newsletter

Compelling research, insights and data directly into your inbox.

Recent media stories

Search

Scroll to Top