10 Years of Open Data

Choosing a point of origin for the modern open data movement is challenging. I, like many others, choose December 7th, 2007 as an arbitrary point in time, as it was this day that 30 individuals met in Sebastopol, California, to discuss how to define “open public data”. This makes today the ten-year anniversary of that historic meeting. From this day, change has certainly occurred in how open data is thought of and used. 

December 7th was not the seminal event that kicked off the open data movement. open data has a complex genealogy. It is a convergence of several US and European efforts and movements. The open data movement has its roots in the geospatial and public sector initiatives going back decades. Linked open data was the original idea behind Tim Berners Lee’s HTML project that kicked off in 1989 at the CERN labs in Switzerland.

The Sebastopol meeting was important. It was the first attempt to codify and define “open Public data”. The writings of thought leaders since that meeting reflect a shift in attitude about the value of open data and its effect on society. I have broken down the history of that thought into ranges of years below.

Several articles have already been written on the origins and history of open data. Below is a list of some of the works that I consider to be some of the most fundamental to open data, written by Simon Chignard, Beth Noveck, Tim Berners-Lee and Sir Francis Maude. Many of these works are written after 2009 but illustrate what the thinking was between the Sebastopol Meeting and the Memorandum issued by the White House on President Obama’s first day in office.

I chose all of these perspectives on what open data means, its value now, and what it was at a given time because they fit within the timeline of the ten years between Sebastopol and now. They also represent different types of thinking regarding the use of open data:

• A Brief History of open data. Simon Chignard, 2013
• Unleashing the Potential Forward by Sir Francis Maude, 2012
• Wiki Government: How Technology can make Government Better, Democracy Stronger, and Citizens More Powerful. Beth Simone Noveck, 2010
• “The Single Point of Failure” in Innovating Government. Beth Simone Noveck, 2009
• “Linked data – the Story So Far” in Semantic Services, Interoperability, and Web Applications: Emerging Concepts. Christian Bizer, Tom Heath and Tim Berners-Lee, 2009.
• “Linked data on the Web” in Proceedings of the 17th International Conference on World Wide Web. Christian Bizer, Tom Heath, Kingsley Idehen and Tim Berners-Lee, 2008.

Many of the original thinkers behind open data wanted the concept to be about transparency. This point comes through clearly when one looks at early works from between 2007 and 2009. These two years act as bookends to the idea that open data can transform government through transparency. One of the most important moments of the Sebastopol meeting was the publication of the 8 Principles of open public data. Most people at this meeting were concerned with access to public data. Simon Chignard noted that some of the people at the conference were interested in “radical transparency”:

“Open data is no longer the transparency tool that its promoters had imagined. Lawrence Lessig dissociated himself as early as in 2009 from the idea of radical transparency. Other participants of the Sebastopol meeting showed their support to WikiLeaks, whose praxis is significantly different from that of open data! Researcher Beth Noveck, who took part in the creation of the open data policies of the first Obama administration, has also expressed doubts concerning the ability of open data to improve per se the governance of public affairs. open data is not enough to open organizations and promote new practices of more transparent and open governance.”

At the other end of this period, we see Beth Noveck working at the White House and participating in the creation of an open data Memorandum on President Obama’s first official day in office.The subject of the memorandum reinforces the Sebastopol meeting’s objective: open data is a vehicle of transparency for government. The subject of the memorandum had four words: Transparency and open Government.

From reading Chignard we know that most of the early thinkers backed off from the idea of data for transparency. In 2012, I read an article by Guillermo Moncecchi of open Knowledge International on the subject of open data as infrastructure. This idea radically changed my perception of open data, and represents a paradigm shift in the movement. Mocecchi states in the opening paragraph  of his article: 

“The most common argument in favor of open data is that it enhances transparency, and while the link may not always be causal, it is certainly true that both tend to go hand-in-hand. But there is another, more expansive perspective on open government data: that it is part of an effort to build public infrastructure.”

His argument continues down the path of infrastructure. His example is using a shapefile to map all of the street lights in his city. He asks, “Is this transparency? No. But it is helpful.”

It wasn’t just me that adopted this mantra. Across the US, there was a renewed focus on open data as a citizen’s right for the sake of solving problems, not just regarding transparency. Code for America Brigades were at the height of their success, using public open data to build solutions for government, for example. Certainly, transparency efforts were still around. The old metric of success, “How many data sets did you publish?” was giving way, however, to “how much reuse are you getting through your APIs?”.

The post-2016 era of open data is far different and more uncertain than in previous eras. We may have always been thinking about new use cases for data release, but one point was made clear by civic leaders like Charlotte’s CIO Jeff Stovall, who said “open data is a one-way street. You can’t go back.” I attended the event when he made this statement. I felt reassured that no matter what, open data had a place in government. It was chilling then when I heard the Master of Ceremonies at the 2016 White House open data Summit say, “Welcome to the first and possibly last White House open data Summit.”

It’s not the end of open data. Far from it. The momentum, in the US at least, has swung back to state and local governments publishing data and seeking an audience. That audience has been elusive. Past models of thinking included the “citizen data scientist” and “open data intermediaries.”. Cities and other public sector agencies want to publish data to solve problems but they want to do it internally. In the ten years from Sebastopol to the last month of 2017 we have seen different models being used and still in use.

open data today seems to be as much about building data stories and engaging with the governed as much as it is about empowering civic tech. Sometimes it is still about transparency. Budget portals and police incident data are still used as civic accountability tools. The Sunlight Foundation has survived as has Code for America. That state and local government as well as some Federal agencies are more thoughtful about why they are publishing data and how to derive value from the exercise means the new normal is not the end. Perhaps, we are simply on the slope of enlightenment.