# The Knowl Post

Imagine, if you will, the year 2020, when a billion people around the planet are at their screens. And each is able to withdraw from a great repository any fragments of anything that has been published, as well as the private documents he or she has access to. So, you’re able to bring to your screen not just encyclopedias, not just novels, not just the works of Horace and Cicero and Marcus Aurelius and Shakespeare and Goethe, but obscure stuff from South America and Africa that people have written in the last 5 minutes. And [you're able] to make comments and footnotes and to transclude and quote from anything else that’s published, with automatic royalty.

That’s Ted Nelson, the prophet of hypertext, writing in Byte in 1990. We still have a few years to go before 2020, and yet almost everything Nelson foresaw is already upon us. More than two billion people around the world are sitting at their screens. The encyclopedias and novels are online, along with Horace, Cicero and the rest. Twitter will show you what’s been written in South America and Africa in the last five minutes. For commenting, transcluding and quoting, we have Facebook and many other thriving channels of commerce and communication (even blogs like this one). So what remains to be invented before we arrive in Nelson’s stately pleasure dome of Xanadu? Well, “automatic royalty” hasn’t shown up yet. More surprising—and more annoying—we still don’t have footnotes on the web.

It’s a worrisome lack. How are scholars to annotate and document—not to mention digress and distract—without footnotes or some similar device? In a recent essay Alexandra Horowitz cites Edward Gibbon, J. L. Austin, Nicholson Baker and David Foster Wallace as authors whose work would be diminished or even destroyed if stripped of notes. I would add Vladimir Nabokov to the list, for his novel-in-notes Pale Fire. Martin Gardner deserves mention as well—not for his columns in Scientific American, where footnotes and marginalia were sadly forbidden, but for his annotated editions of Lewis Carroll and others. And the most important example of all is the Talmud, the archetypal hypertext of modern times. We should be able to do stuff like that that in HTML!

Why are footnotes not a standard feature of web life? Perhaps because the founders believed that the central mechanism of the web—the anchor tag <a href= … >—adequately served the purpose. And people do use <a> tags for notes. For example, there’s the Wikipedia protocol, where clicking on a footnote link[1] sends you on a trip to the bottom of the page, where you’ll find a helpful caret to take you back where you came from. This scheme does a good job of mimicking the ink-on-paper experience of footnotes—and ignores all the capabilities and possibilities of a new medium.

Back in 1995, the HTML 3.0 proposal included a <fn> tag for footnotes. But HTML 3.0 was stillborn, and the 3.2 version that replaced it a few months later dropped footnote support (and much else). HTML5 now offers the <aside> tag, which sounds like it ought to be the semantically correct way to mark up footnotes. But I have yet to see any website in the wild actually using <aside> that way. And the (still tentative) HTML5 standard suggests that footnotes were not the the original intent of the tag:

The aside element … can be used for typographical effects like pull quotes or sidebars, for advertising, for groups of nav elements, and for other content that is considered separate from the main content of the page.

It’s not appropriate to use the aside element just for parentheticals, since those are part of the main flow of the document.

Yeah, whatever. But what makes the web such a grand playground is that you can always build your own tools and toys if you don’t like the standard kit. Which brings me to all those blue links with dotted underscores in the paragraphs above. I assume you’ve tried them out by now. The device is called a knowl, and I discovered it the other day while browsing on the home page of the American Institute of Mathematics in Palo Alto. The knowl’s inventor is Harald Schilly of the University of Vienna.

It’s all done with a dab of jQuery and a dollop of CSS. The markup in the HTML file is just a mutant anchor tag:

<a knowl="wikitag.html">Wikipedia protocol</a>

where the normal “href” has been replaced by “knowl.” The jQuery code installs an onClick handler for each instance of this structure found in the text; the onClick function calls out to the server, loads the content of the note, and adds that text to the document tree as a sibling of the current node. CSS rules add a bit of border styling. The little drawer-like pane opens just below the current paragraph (or whatever other HTML element contains the reference to the knowl).

The knowl is not quite everything I would wish for in a footnote utility. A minor issue is that the “next-sibling” rule for placing the text of the knowl doesn’t always do the right thing. A less-minor issue is that the authoring process is overly arduous. Every knowl goes into a separate file, so writing the text requires a change of mental focus and also creates a lot of file-system clutter. I’d rather include the text of the note at the point of reference (the way it’s done in LaTeX, say). That could be accomplished with an easy change to the JavaScript. On the other hand, the system’s reliance on JavaScript is in itself problematic. The world of knowls is off-limits to those who write on hosted platforms such as WordPress.com or Blogger, because they forbid JavaScript in posts. I wonder if the same visual effects could be achieved entirely with CSS3 animations?

When Schilly and his colleagues at AIM developed the knowl, their goal was not to satisfy my footnote fetish. As a matter of fact, they seem to have a rather different vision of how knowls might be used—not as a medium for the Shandean digressions of self-indulgent writers like me but as shared nuggets of wisdom, offered as a public resource. David Farmer, director of programs at AIM, writes: “I envision a time when the Internet has a repository of such knowls, reliable and ready to be referenced anywhere.”

Unfortunately, there’s a technical impediment to making that vision a reality. When I started writing this post, I thought it would be only appropriate to transclude the definition of transclusion that’s given in one of the knowls above. So I constructed a knowl linking to the file on aimath.org where that text exists. This knowl has such a link. If you click it, you’ll find that it doesn’t work: The little blue drawer slides open, but it is empty. The reason is that the page you are reading now was downloaded from bit-player.org, but when you click that knowl, it tries to access HTML content from aimath.org; that attempt runs afoul of the browser’s “same-origin policy.” There are ways of evading this security provision, but they come with a faint scent of hackery. Without them, though, I don’t think we can have our public repository of knowls. It looks like we’ll each have to serve up our own wit and drollery.

Update 2012-02-28: Harald Schilly has shown me that transclusion is not a vain dream after all. The solution relies on a fairly recent technology called Access Control for Cross-Site Requests, and it requires a cooperating server to set an appropriate flag in an HTTP header. This knowl is fetched from appspot.com, the hosting arm of the Google App Engine, which allows cross-site connections from anywhere. If you are reading this page with a recent Webkit browser (Google Chrome, Safari) or a recent Mozilla browser (Firebox), the magic drawer should open with a note inside. But Opera does not implement the method, and I think Internet Explorer is also a holdout, although I don’t have the means to check. I was totally unaware of this trick, and I’m grateful to Schilly for enlightening me.

^ Hi! I’m a Wikipedia-style footnote. I feel lonely and exposed and painfully out of context down here.

This entry was posted in computing.

### 13 Responses to The Knowl Post

1. bbot says:

And, of course, knowls don’t show up in RSS feeds either.

2. Markk says:

I’m having a hard time seeing why Wiki footnotes don’t meet what you want.
Is it just the popup right below there or ? Reading this post I like the Wiki footnote way better than the knowl for footnoty things. That is my personal preference though.

3. brian says:

@bbot: Good point!

To my surprise, though, I find that the knowls do work, after a fashion, in my newsreader (NetNewsWire 3.3). They lose their styling, but the Javascript works, sliding the text open and closed at the appropriate spot.

4. brian says:

@Markk: If we could all agree on some suitable semantic markup—which would identify a hunk of text as a footnote or an aside or whatever, without saying how the browser ought to display it—then we could all have our own personal preferences (at least if we’re all willing to write our own stylesheets). You could see wikinotes and I could see knowls. But it’s been 20+ years and we still don’t have consensus on the right HTML tag. I may be the only one who cares.

As for *why* I prefer the knowlly solution: To my taste, it’s the better compromise between unobtrusiveness (when the note is closed) and frictionless accessibility (when it’s open). Anything that involves scrolling the window causes me to lose my place in the main text. Because a “page” on the web can have unlimited length and isn’t all within view at the same time, the Wiki notes seem more like endnotes than footnotes. You need to leave a bookmark to find your way back.

5. Tony says:

Right… the same-origin policy is going to get in the way of transclusion as Nelson envisioned it. His version required that the transcluded data come directly from the original source (that would facilitate his payment model, of course, as well as providing automatic updates).

If you don’t need to support outdated browsers, you can get around the cross-domain restriction with HTML5′s postMessage function. That allows (relatively) safe transport of data between cooperating web pages (both sides have to be set up to allow the data transfer, and the receiver can’t see anything that the sender doesn’t explicitly send). It’s supported in recent versions of all the major browsers. There are good examples here: http://davidwalsh.name/window-postmessage and here: http://davidwalsh.name/window-iframe

For years I’ve been daydreaming about a project called the Talmud of Science. We take a dozen or so deep and difficult papers—the kind that everybody knows but nobody reads—and we do the exegesis thing on them, with expert commentary, historical background, marginal glosses, added illustrations, and so on.

Charles Petzold did something almost exactly like what you’re describing in his book The Annotated Turing.

7. Barry Cipra says:

I very much like what I’ve seen of knowls. I like the way they open and close without a jarring dislocation of the text (or whatever) that links to them.

One thing I noticed, which may have to do with the “file-system clutter” you referred to: When I read breadbox’s comment, I didn’t know where the excerpt he quoted came from, so I had my browser search for it, but couldn’t find it because the quote is in a knowl rather than the text. Fortunately I chose “Talmud” to search on rather than, say “exegesis,” so the search took me to the one paragraph with that keyword, and I guessed the quote was in one of the knowls there, so it wasn’t that hard to find. But it would be nice if there were something that facilitated this sort of search.

8. brian says:

@Barry Cipra: Search is another issue I didn’t even think about. Browsers include only visible text in search results, and that’s not likely to change, even if we wanted it to. And it’s not just the browser search function. On a larger scale, I’m pretty sure that search engines will also ignore the knowl content, so you can’t Google for it either. This is not ideal and probably not easy to fix.

9. David Farmer says:

One of the services knowls provide is a simultaneous generalization of what \cite{} and \ref{} do in LaTeX.

For example, suppose you are reading a (math) paper and it says “by equation (2.3) we have…”. In the PDF or HTML version, that equation number is a link to another part of the paper. It is better to have a knowl that makes the equation appear under the text you are reading, instead of jumping to another place and then navigating back (especially if your PDF reader does not have a “back” button).

The same idea applies to bibliographic references, and of course footnotes as Brian described so clearly.

I’d like knowls to make wikipedia better. Nearly every wikipedia article would be improved by converting many of the internal hyperlinks into knowls which tell you a one or two paragraph version of what is in the linked article.

10. Cody says:

In Mac OS, holding command-control-d while mousing over a word or selection (in most applications) will quickly display a dictionary or thesaurus entry. (And I guess also a wikipedia entry in 10.7, though I’m on 10.6.)

11. What about the <details> tag? According to the proposed spec, it “Defines additional details that the user can view or hide.”

Seems like what you are thinking about.

As far as global repositories of footnotes goes, getting the semantics right will lead to artificially intelligent (see Google’s search engine algorithm) systems to create and search the repository of <details> tags.

12. brian says:

@Collin Reisdorf: The W3C is pretty explicit about that one:

The details element is not appropriate for footnotes. Please see the section on footnotes for details on how to mark up footnotes.

(The section on footnotes discusses <a> and <aside> and other stuff already mentioned above.)

13. Darius Bacon says:

Another problem with Wikipedia-style footnotes: you click one and teleport to the bottom of the page, but *not* to the exact target, because browsers only scroll to where the bottom of the page coincides with the bottom of the window; the footnotes typically all fit in that last screenful that won’t scroll further. You try to remember the number and scan for it in an article-worth of notes.

Here’s an example of footnotes-on-hover: http://www.gwern.net/DNB%20FAQ — it’s got some drawbacks too.