Review how the government measures the release of data by developing performance indicators for quality and usefulness

The Victorian Government IT Strategy 2016-2020 highlights significant changes in the way that the government thinks about data. It builds on the Data.Vic access policy released in 2012, and states that
Government information and data should be open and transparent wherepossible. It should also be treated as an asset, meaning it must be accurate, not duplicated, stored sensibly, protected from unauthorised access where necessary, available when needed and shared as required.

More and more people, and government agencies, are realising open data can build useful applications. We can use the data to build for example, applications that help society, or investigate how effective policy changes have been over time.

The trend of making more and data available will continue — that’s a given.  But what we need to find out (with your help) and agree on very soon is, what are the characteristics of data you value the most, what drives improvement in quality  and what makes it useful?

What's happened so far

Data Vic Access Policy

The Data Vic Access Policy released in 2012, makes it the State’s default position to make datasets freely available to the public.

The policy outlined the following expected benefits, however no framework for realising these benefits has been developed.

  • stimulating economic activity and driving innovation and new services to the community and business;
  • increasing productivity and improving personal and business decision making based on improved access to data;
  • improving research outcomes by enabling access to primary data to researchers in a range of disciplines; and
  • improving the efficiency and effectiveness of government by encouraging better management practices and use of the data.

Definition of high value

Since 2014, we’ve seen an increased focus on trying to define what might be ‘high value’ sets. Here’s the current (and probably improvable) definition of high value data as set out by the Data Vic Access Policy guidelines:

“A high value dataset is defined as a dataset that is likely to be of interest to the Victorian community, and/or a dataset that has potential for valuable reuse. A dataset should be considered ‘high value’ if it:

  • is central to the department/agency functions e.g. DTF and budget data;
  • has been requested via ‘suggest a dataset’
  • has previously/regularly been provided under the Freedom of Information Act 1982;
  • supports a major reporting process of government e.g. annual report data;
  • planning data;
  • spatial data;
  • transport data;
  • administrative data; and
  • financial data.”

Through the development of performance indicators for quality and usefulness the way in which we measure the value of data should be reconsidered.

Where we are today

To date there are 6198 datasets that are discoverable.

In line with world-wide trends in open data transport and spatial data is the most downloaded content.  The Victorian government agencies who own this data ( PTV, Vic Roads and Department of Environment, Land, Water and Planning (DELWP) ) are not surprisingly the leaders in Victorian Government data release.

How can we ensure quality and usefulness in the data we release

The Victorian Government wants to get more value from open data. Although the concept of open data has been embraced throughout the world and many jurisdictions have open-data policies, the take up and measurable benefit has been limited.

In supporting this objective, Action 2 of the IT strategy requires the review of data release processes and the development of indicators for quality and usefulness. 

The purpose of the development of indicators is to create a better understanding of the value of data collected and released to the public. 

What we’d like to happen next: Our project  

This project is to work out which indicators we can use to improve the quality and usefulness of the data we release.
About the project in general
The desired outcome for our consultation with you is to build a set of indicators for quality and usefulness that:

  • the community who use government open data will share;
  • provide a useful input into their interaction with the data; and
  • improve the experience of using government open data.
  • areas to we’d also like to better understand
  • what indicators will drive better data release practices in Government agencies and improve the usefulness and quality of data we release?
  • what indicators lead to a better experience and do they differ between different user groups for example, researchers, data scientist, developers?
  • how do people who use the data, value the data and what makes it useful to them?

The draft indicators: Seeds of discussion

What are indicators?    

In general, there are  a set of core indicators we use to measure quality and usefulness of data the Victorian Government releases.
Different users place differing value on core indicators:

  • researchers value historical data; and
  • developers value regularly updated data.

Government agencies also place different emphasis on quality and usefulness than end users, so for this reason we’re considering two sets of indicators ; one for end users and one for government departments.
We’ve derived these ‘first working drafts’ in Tables 1 and 2 below from a range of international sources.

They’re to help us get the discussion with you started, nothing more. We’re happy (and we hope) to end up with a different, improved list.

Draft indicators for open- data consumers ( developers researchers etc)

Draft indicators

Draft indicators for open-data consumer (developers, researchers etc)

Designed to help people choose the open-data that best suits their purposes. (One step before the real life end user or the application’s release.)

  • Discoverable make sure open data is discoverable
  • Re-usable format make sure the data are available to the widest range of users, for the widest range of purposes
  • Reduces barriers to access ( such as registration to access) make data as easy to access as possible
  • Granular original un-modified form, at its finest level of granularity
  • Immediately intelligible information about data is written in plain English.Should include enough information about strengths and weakness, analytical limits and security requirements as well as how to process the data
  • Authoritative questions you should ask to make data trustworthy
  • Linkage indicate where data is connected and what datasets are more insightful when combined with others
  • Timely  (early as possible) timely release of data increases its value and adds to the trust in data released by government
  • Comprehensive all parts of the dataset is released
  • Accurate data released is as up to date possible and is maintained
  • Opportunity for feedback this may include process to access the data owners to clarify understanding etc.
  • Conformance data is structured in line with accepted standards
  • Openly licensed data released under an open licence, allows flexible reuse, preferably Creative Commons BY licence
  • Relevance data includes appropriate amount of information

Draft indicators for government (release open data)

Objective: To provide a framework and good business reasons/motivation to release

  • Data release process data release process and release mechanism is built into business processes
  • Data release is linked to business unit objectives open Data links back to achieving aims and objectives of the Department or agency
  • Strategic use of government data strategic use and release of data to solve complex social and economic challenges
  • Provision of data for internal government use including council to reduce effort and duplication in collection - Provide data that can reduce duplication of collection and improve insight
  • Inter-agency collaboration on data collection to save time improve process, reduce requests
  • Engagement with other market players using data government data providers working with the market place of data users

We would like to hear your views

  • We are happy to receive your views in any form but here are a few questions to get you thinking:
  • Are there different motivations for data users and data releases?
  • Is separating the indicators for end users and government data release?
  • Are there indicators that in your opinion that don’t quantify quality?
  • Are there indicators that in your opinion that don’t quantify usefulness?
  • Are there missing indicators?
  • How would you define high quality?
  • Does high quality need to be defined?

Tell us what you think