Throughout 2018 we conducted research into open data practices in the Australian government sector to inform Victorian policy on open data. The past decade has seen rapid developments in how government data is produced, assessed, released and re-used.
What is open data?
The term open data can refer to many things. Public records, annual reports, transcripts and statistics collected by local, state and federal governments can all be classified as information that rightfully belongs in the public domain.
Sharing and releasing data collected by government departments and agencies in formats that can be easily discovered and analysed helps the public service, business, researchers and citizens work more efficiently and effectively.
Opening government data so that it can be viewed and re-used by the public increases dialogue, trust and transparency between governments, their various stakeholders and citizens.
Since 2009 governments around the world have established 'open data' policies that are based on the idea that government data should be anonymised and made freely available online and "open by default", meaning that government data is published as close to the time of its production or collection unless there is a valid reason for it not to be.
Proponents of open data argue that it in addition to these social and political benefits, it can boost productivity and economic growth. But the 'vision' of open data has been a difficult one to realise especially because demonstrating the value of open data when it is used or re-used is not easily communicated.
Why study attitudes about open data?
To date, there have been few in-depth studies of how people working in the area of open data perceive the current state and future of open data.
To better understand this in the Australian context, The Victorian Government and the University of Melbourne partnered in a year-long research project that seeks to investigate the enthusiasm, caution, contradiction and potential that surrounds open data.
Our motivation in doing this research was to reflect back on ten years of open data practice and consider how the ideology of open data has been picked up, translated and evolved within the three jurisdictions of Australian Government -local, state and federal.
We hope that this research contributes to future thought leadership in open data practice.
Our research questions
- What are the attitudes of government agency workers towards the implementation of open data policies and what factors contribute to the success or failure of open data programs?
- What are the indicators of open data maturity and how can open data programs be improved?
- What factors contribute to, or limit, the release of open data?
The history of open data
The open data community is analytical, critical, and obsessed with measurement. Can we say that we actually understand what has happened with regard to open data around the world over the last decade? After ten years, are individuals, organisations, and governments able to get involved with open data more easily? Are stakeholders really starting to work on new open data initiatives based on evidence or research, or being steered by whatever might be the hot topic of the day? Are activists chasing un-achievable dreams or going somewhere tangible?
Walker et al., The State of Open Data (2017)
The ‘story’ of open data extends as far back as the first half of the 20th century. During this time, ‘openness’ as a political concept began to emerge through the work a number of social theorists.
American sociologist, Robert K. Merton, identified in 1949 that political programs exhibited manifest functions which are easily recognisable, and the intention behind them is conveyed with adequate transparency. For Merton, paying attention to the hidden consequences of political decision-making on the everyday lives of individuals was a matter of significant concern to citizens.
Philosopher, Henri Bergson, coined the term ‘open society’ in 1939. In an open society, the government was expected to move away from de-personalised and authoritarian ways of governing and progress towards more collaborative, flexible ways of working where governments endeavoured to be more transparent and accountable.
These ideas remained largely philosophical until the 1980s when they were eventually put into mainstream practice through the Free Software Movement, which is attributed to activist and programmer Richard Stallman. Declaring that ‘all software should be free’, the Free Software Movement and its offshoots – the LINUX operating system, open software and the Open Source Initiative – laid the groundwork for the collaborative, decentralised sharing of knowledge and programming code. Open source realised, through networked technologies, the potential of the ‘open society’.
In a 2012 blog post written for the Open Knowledge Foundation blog, Guillermo Monecchi writes:
We, as the government, must tell the people what we are doing and how we are doing it. Our knowledge should be open. We have the duty to publish our knowledge and let others use it, so that we can participate actively in communities, propose changes, and act as an innovation tactic in every task we face. Because we are paid for that: for building knowledge infrastructure available to people to do whatever they want, within the law. Exactly the same thing they do with the streets.
This 'infrastructure' view of open data arises from the view that a more efficient and innovative public service can be developed if organisational silos are broken down and if the community are consulted about the design of policies and services.
The open data movement was given new life in 2009 when on the 21st of January it became enshrined in US Government policy. A day after his inauguration, then President Barack Obama delivered the Memorandum on Transparency and Open Government to the heads of government agencies. It directed them to enact policies which reflected the Administration's principles of transparency, public participation and collaboration. The memorandum was followed by the launch of the US government's open data portal data.gov in May 2009, with 47 datasets published by Federal Agencies (Madrigal, 2009). The UK government followed shortly after in January of 2010, then Australia, Canada and Singapore in 2011 and the EU, Japan and South Korea in 2013.
The second decade of open data is less certain than in previous eras. The initial optimism and commitments have been eroded by scepticism about it value and the level of return on investment that can be demonstrated when the whole purpose of open data is that it is provided freely and without a requirement to report back to data custodians and publishers about how data is being used.
While there are various models for the sharing and public release of government data being practised at the federal, state and local level in Australia. While an overall guiding principle for open data in government has arisen out of efforts which have aimed to respond to the increasing calls for transparency around the inner working of government and the public service, open government data is fast becoming recognised as an important resource for the business sector, researchers and civic or entrepreneurially minded members of the community.
Methods
Our study is based on qualitative and quantitative research that was carried out between 2017 and 2018 primarily in Australia, Singapore and London. Having studied open data practices through semi-structured interviews in the past, our aim in this research was to continue to study open data practices among those directly implicated by conceptual, technological and operational developments in the area of data collection, information management and publishing of government data sets. In interviewing and surveying stakeholders in open government data, we engaged with data custodians, policy makers, those designing digital services for citizens, technology vendors in the private sector, activists, evangelists and academics. The inclusion of a broad cross-section of Australian government workers at different career stages with varying degrees of involvement in open data initiatives and varying levels of open data literacy.
The main research method we used was the semi-structured interview. An advantage of this method is that it allowed us to engage in conversations about government practices from within government by talking to people working in existing open data programs or who had worked in the field in the past. To set up interviews we drew on existing contacts who had worked with or were known to the personnel in the Victorian Government’s open data program. We also asked interviewees for their advice on other people who they thought we might want to interview.
We approached the interviews with a structured set of topics and questions tailored to the individual and the organisation but we never considered this to be fixed. In allowing the conversations to develop in different directions, and by picking up on cues and following the interviewee’s interests and experiences, we uncovered a number of issues that we had not thought about in advance.
Findings
Our study found that:
- there is a great deal of enthusiasm for open data but people have very different views on what 'open' means
- new ways of defining 'openness' are needed for open data programs to continue to be vibrant and sustainable
- open data is perceived as a valuable public asset but there is much debate about what high-value datasets are and how they might be leveraged
- continued funding is a concern for those working open data programs but there is little evidence of proven benefits of releasing data
- there's a strong need to develop greater literacy around open data and this ranges from being able to 'read' and 'analyse' open data sets through to being able to recognise and predict the ethical and legal issues related to data release
What's next? Putting the findings into practice
We'll workshop each of the report's recommendations with key stakeholders in Victorian Government. Then we'll develop a plan for their implementation.
Throughout its implementation, the process of implementing our findings will also be studied as an extension of the original research project.
We hope that this observational and ethnographic work evolves into a set of tools and resources that support open data practices for the Victorian public servants and Victorian citizens.
Updated