We see open data as part of a broader trend towards “open government” where open data combines with social media, mobile technology and other feedback mechanisms to transform the relationship governments have with citizens, providing better and more relevant public services (which we could broadly open data terms “citizen-centric”). Open data also has the power to improve people’s lives through private or third sector innovation based on publicly available data sets, resulting in valuable services and economic growth (which we might call “consumer-centric” open data).
Our analysis of these global trends and our experience has underscored that the US government is right to think of transparency and open data as an evolutionary process that must adapt flexibly to changing priorities. Public sector managers and citizens have changed the way they use the available information:
– Feedback between citizens and users has diminished over time, but there is growing external interest in more specialized data sets.
– Performance dashboards gave public servants a head start in enabling better management and cross-agency comparison, and now they have an appetite for more granular analytics capabilities to perform sophisticated analysis and drive agency change.
Where publishing raw data sets has been useful for specialist developers, in our experience, it hasn’t always reached its full potential. Experience so far shows that data is most useful when it is published in a consumable way, and usability (for both the specialist developer audience and the non-specialist citizen audience) should be a key consideration in making open data a reality. The under-exploitation of data that is already in the public domain may be due to two reasons: the data is not always in a standardized format and the information market may be underdeveloped. Rather, the information must be easily accessible, well organized, intuitive to use and understand, and the data sets must be complete. To harness the full potential of open data, both for citizens and businesses (and to minimize the burden on public services), we believe it is important to:
– Short-term release commitment, but focus on rigorous data hygiene standards for data currently being collected – data is only as good as its quality. Inaccurate data sets can be misleading at best and harmful at worst. Data quality is central to the open data agenda, but we agree that the cost of completely ‘clean’ data can be prohibitive and therefore a compromise must be found between quality, cost and timeliness.
– Reduce, Reuse, Recycle: The burden on public service providers can be minimized by prioritizing the publication of data sets that are already being collected, for example, management information on service quality and results, rather than to collect new data sets. The provision of context will also be important for the publication of service performance data so that they can fully understand this information and avoid any misinterpretation of this data by the public.
– Make it valuable to citizens: Open data has been most empowering and successful both in the US and internationally when it has been real to citizens, that is, when individuals can relate to the information provided and can affect a real change. Successive examples show that open data that is provided at the local (eg city), neighborhood and even street level can be more effective.
– Standardize to facilitate comparability and interoperability: standardized data sets in consistent formats will allow citizens/users/developers to easily compare and analyze information; this is particularly important for organizations such as the police or local authorities where citizens may want to compare organizations with their peers.
– Turn data into intelligence: Many public sector organizations are rich in data, but bringing together the different data sets to create usable analytical intelligence to provide insights (whether to managers, commissioners or the public) can be more challenging. Widespread adoption of analytical and predictive modeling techniques that analyze and visualize information would provide easier-to-consume insights for those inside and outside organizations.
– Make targeted investments to turn “data” into a “service”: the government needs to make limited and targeted investments to transform some crucial data sets into “services” making them easily consumable; good examples are combinations like crime maps. This is particularly important for the citizen-focused open data agenda and will help establish a market and stimulate demand for more published data sets.
As part of the prioritization process and as discussed in our overview, we believe the government is right to think of transparency and open data as an evolutionary process that responds to changing needs (of users, developers, and public services). over time. Based on our experience, we have developed a framework for approaching open data in this way and adapting to the needs of users as they become more sophisticated and that could be useful in developing an organization’s open data strategy and identifying the steps that should be prioritized.
In some circumstances, it can be prohibitively expensive to collect and process a raw data set, so it may be worth evaluating the feasibility of encouraging the private sector to pay for and commercialize the data. In such circumstances, the government should consider forcing the buyer to publish the dataset in its original format. If the government can make targeted investments to turn data into a service, there may be scope to sell these services to the public and create a revenue stream. Although beyond the scope of inquiry, the government may wish to consider charging for access to anonymous data sets if it can generate direct commercial gain, for example pharmaceutical companies and health data.
The proposals must also apply to frontline public service providers in the non-public sector, such as charities or private sector companies, if the transparency agenda is to be meaningful and accountability real. It would also help build the case for mixed provision of public services. We urge the government to implement measures to ensure the consistency of the data collected and its presentation to allow comparison between providers.
One approach may be to include the publication of data as an evaluation criteria in the tenders for incentive providers. As a further step, the government may want to consider the feasibility of making data collection and consistent reporting a contractual obligation. If this is sought, it is worth considering transition arrangements for current contracts until the contract is complete.
While the legal and regulatory framework exists, it will be important to give more consideration to guidelines and standards to protect personal data under the open data agenda. These should include clear guidance on what constitutes personal data in this context (for example, taking into account the many decision notices under the Freedom of Information Act) and standards for anonymization and pseudonymization (which have great potential in the context of health data). People must be protected on an equal footing and consideration must be given to how data sets could potentially be merged or reconstituted across government so that standards are consistent and robust.
To improve accountability and public service performance, the publication of existing management information should be prioritised, which should not represent an additional burden for organizations as they will routinely collect and report this information. However, we recognize that different levels of management information are currently collected; from operational data to service performance measures and then information on products and results. We recommend prioritizing the publication of results, products and measures of service performance. Most operational data is unlikely to be meaningful to the public and will need to be contextualized to ensure it is not misinterpreted.
An effective way to ensure that open data standards are incorporated would be to make them a contractual obligation. In addition, we agree with the government’s objective of making the public sector an intelligent and demanding customer. As part of this, it would be useful to create focus groups that allow vendors and other stakeholders to contribute to open data standards in productive ways.
