Dan Whaley & Hypothes.is: Annotating the Web

by Chris McGivern & SF Team, 4 February 2019

Distortion of information on the Internet has been an issue long before you might have heard the term ‘fake news.’ Who can you trust to tell the truth? How can the average person determine the quality of research that lies behind an apocalyptic headline? And whose narrative is driving the agenda?

Quality of conversation should be central to human interaction, but what we read through the lens of the web is often blurred by untruths and agendas, a distinct lack of critique and, as we have seen over the past few years, perhaps even foreign interference. Dan Whaley is attempting to change this with Hypothes.is, an annotation tool that promotes a more open web and improves discussion and discourse.

The Fellow: Dan Whaley

Dan Whaley came face-to-face with the degradation of public discourse in the early 2000s. After starting - and selling - the first travel booking organisation on the web, he founded Climos, a startup looking at large scale mechanisms for carbon reduction.

“I became a daily reader of climate change and environmental news,” Dan explains. “I saw first-hand how the narrative was distorted purposefully by certain corners of the media.

“I was frustrated by a lot of the discourse out there, and the weaponization of the Internet by folks with an agenda.

“It was deliberately being used to force certain policy outcomes, to constrain governmental ability to meet international obligations, and to limit our ability to enter into climate change treaties.”

To prevent this problem, you could try to create a new form of better, more open media. But in reality, you can’t out-megaphone the people who already have a worldwide - and captive - audience. But what if you could lay a truth lens over the Internet? What if anyone could see the arguments against a point of view or the discussions that have led to a particular conclusion?

“Once you start going down this route you are inexorably drawn towards a solution that looks like web annotation,” says Dan.

The Project: Hypothes.is

With Climos winding down, Dan became drawn further into this idea of annotating the web. There were other projects touching on annotation, including a San Francisco company called Reframeit.

“They were doing something very similar to what I imagined this thing to be,” says Dan. “Except it wasn’t open source, it wasn’t a standard, and it was a single, venture-backed monolithic company. They gave up the ghost soon.

“My friends were starting to get tired of me talking about this crazy idea,” he continues. “It was obvious it was actually time to do something, so in 2010 I started the background research that led to Hypothes.is. When we began, it had a focus on fact-checking as the primary challenge.”

Dan got in touch with people that had worked on similar projects and asked to interview them about their experiences, and why we still didn’t have this capability.

“The trick was that it had to tick a bunch of boxes,” says Dan. And these ‘ticked boxes’ became the basic organising principles of Hypothes.is:

• It had to be annotated, supporting line by line analysis
• It has to be able to work on any page or document without the permission of page owner
• It had to be open source, at least for the first implementation
• It had to work towards global standards so that an ecosystem of interoperable systems and services could emerge.
• It had to be federated, like the web, so that anyone could build a service or a client.
• Eventually, it had to ship everywhere, working natively with browsers and other applications
• It had to support different communities with diverse perspectives
• The first organization behind it needed to be a non-profit

Hypothes.is started life as a Kickstarter fundraiser and quickly the scale of opportunity became apparent. The universal capability of Hypothes.is was applicable to a huge range of use cases across many domains, and for many user needs, whether uses and users, whether individual, group, or for public discourse organisation.

Soon, code was being written and a community was starting to develop. But money, as always, was an issue.

Foundation Funding

Making the decision to be a nonprofit seemed obvious for Dan. But this meant funding options were limited.

“I realised we needed a certain type of funding,” he says. “I met Rufus Pollock, from the Open Knowledge Foundation, and he suggested we give the Shuttleworth Foundation a try.

“They turned me down the first time I applied– but they also told me to keep trying. I got in on the second time, a year later. I was told it was because I had started thinking about the project more broadly, and our ability to serve more diverse use cases and to partner with other organizations. That awareness of the bigger picture was the thing that tipped the balance from their side.

“The Foundation is an incredibly rare and unique organisation that provides substantial funding - essentially a million dollars over three years - to pursue open source technology projects in the public interest.

“And that really - as a category - doesn’t exist anywhere else,” he says. “There are very few other foundations which support open source technology out there but the focus on early stage, open source civic tech with this kind of support, and with the other kinds of resources that Shuttleworth provides, doesn’t exist in any other forum.”

The Fellowship

Dan Whaley began his Shuttleworth Foundation Fellowship in March 2013. His objectives were to:

• Further develop Hypothes.is
• Roll out an open-source platform to enable conversation, collaboration and critique on any online content, regardless of the medium,
• Empower communities and experts to engage in thoughtful analysis and public discourse in ways, and at scales, never before possible

Over the course of his three-year Fellowship, Dan grew the early-stage, fledgling idea of Hypothes.is into a fully-formed organisation supporting annotation anywhere on the web.

The project also launched the Annotating All Knowledge coalition. The coalition seeks to understand how to implement a standards-based annotation layer over the web, using exploration and collaboration, very much in the same spirit as the Shuttleworth Foundation.

The Foundation actively encourages collaboration in two distinct ways. First, there is the principle of leaving something behind for the world to build on. In the case of Hypothes.is, Dan was able to apply an annotation tool based on an original idea and code - AnnotateIt - created two years earlier by Rufus Pollock and Philipp Schmidt as a small Foundation project.

Secondly, collaboration is encouraged between Fellows. “I counted maybe nine or ten collaborative projects that were active during the time I was a Fellow, says Dan. “I think I had ongoing projects with four or five of the other projects. It’s pretty unusual.

“The projects are very different, with esoteric communities that are completely awesome. I still collaborate with Adam Hyde. We are partners with Coko Foundation, and it’s an ongoing thing.”

Personal Reflections

“Having got grants from a bunch of organisations now, there is really nobody else out there that creates as much of a family as the Foundation does,” says Dan. “Or that puts as much time and energy into supporting people. I really can’t say enough wonderful things about them - just awesome, awesome people.

“I think the distinctive things about it are the regular dialogue amongst/between the Fellows and the twice a year Gatherings.

“At times I would say that when you have a ton of stuff to do, twice a year was overload. You struggle to find the time - that was the only issue for me.

“But I also met a ton of good people and it went smoothly,” he continues. “I wish I was still able to go to more Gatherings - I just never have taken them up on it because I am too swamped.

“It was a little challenging but just an amazing experience. There is all the other foundations, and there is the Shuttleworth Foundation: they are that distinctive in their approach. Those three years went by in a blink of an eye.”

Hypothes.is Now

Today, Hypothes.is continues to mature, both technologically and organisationally. With a staff of fourteen full-timers, Dan Whaley and the team are expanding and deepening the product as a general purpose application you can use anywhere on the web.

“Eight years in, and we have 200,000 users and 4.5 million annotations,” says Dan. “It’s bringing arguments about specific details at a fine grain level to the online document itself. You don’t have to go elsewhere to read the rebuttal - it’s right there.”

Supported by several funding organisations, Hypothes.is signed their first commercial contract in 2017, and have plans to make it possible for people and organisations to run their own servers and publish annotations into a universal client.

“It has a particular utility in certain vertical markets,” explains Dan. “The goal is to continue to make to improvements, refine and extend the technology to get it more deeply embedded in things.

“Research, education, journalism … there are key product areas we are targeting which will help us start to generate more revenue, so we can fund the development of this for the benefit of humanity but be sustainable at the same time.”

Final Words

“The Shuttleworth Foundation is one of the most extraordinary and important organisations out there working to support a fundamentally different approach to solving problems,” says Dan.

“It encourages entrepreneurs seeking a different path than the typical for-profit venture path to solving problems. People that truly care about and are passionate about the change they are trying to make and doing hard things where the traditional funding models don’t apply.

“Finding organisations that fund those projects is difficult. A lot of people don’t get it - they aren’t qualified to assess these kind of projects or they feel like there is a lot of money out there for technology, and what’s lacking is other kinds of things. I don’t know what planet they’re living on!”