The most recent, and recommended, format for discovery is XRD (currently at Working Draft 15). XRD supersedes the older XRDS format, which has a convoluted evolution that can be traced in twists and turns through XRI Resolution, Yadis, and XRDS-Simple.

These older specifications define both a data format and a resolution protocol. The resolution protocol suffers a problem, however: it can’t distinguish between a host and a root-level resource. A succinct technical statement is provided in Eran Hammer-Lahav’s host-meta draft:

Because there is no URI or a resource available to describe a host,
many of the methods used for associating per-resource metadata (such
as HTTP headers) are not available. This often leads to the
overloading of the root HTTP resource (e.g. ‘http://example.com/’)
with host metadata that is not specific to the root resource (e.g. a
home page or web application), and which often has nothing to do it.

Take, for example, http://jaredhanson.net/, the URL I use to identify myself online. It has an XRDS document located at: http://jaredhanson.net/meta.xrds

<xrds:XRDS xmlns:xrds="xri://$xrds"
           xmlns="xri://$xrd*($v*2.0)">
<XRD>
  <Service>
    <Type>http://portablecontacts.net/spec/1.0</Type>
    <URI>http://www-opensocial.googleusercontent.com/api/people/</URI>
  </Service>
  <Service>
    <Type>http://specs.eaut.org/1.0/template</Type>
    <URI>http://jaredhanson.net/</URI>
  </Service>
</XRD>
</xrds:XRDS>

Among the services advertised are an address book available via Portable Contacts and an EAUT-based email address to URL translation service. The address book is very specifically associated with me; it is my address book. The EAUT service, however, is simply a service that is running at the domain jaredhanson.net; it is not associated with me directly. This case exhibits the conflict between the host and the root-level resource.

The new XRD specification does not define a resolution protocol. That task is left up to other specifications such as host-meta and LRDD. host-meta, in particular, addresses this conflict by registering a well-known URI for metadata about the host.

Looking again at my digital identity, things have been broken down into two separate documents. One for information about the host at: http://jaredhanson.net/.well-known/host-meta

<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"
     xmlns:hm="http://host-meta.net/xrd/1.0">
  <hm:Host>jaredhanson.net</hm:Host>
  <Link rel="lrdd" template="http://jaredhanson.net/meta.xrd?id={uri}"/>
</XRD>

And another for information about me at: http://jaredhanson.net/meta.xrd

<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <Subject>acct:me@jaredhanson.net</Subject>
  <Link rel="http://portablecontacts.net/spec/1.0" href="http://www-opensocial.googleusercontent.com/api/people/"/>
</XRD>

Using this schema, my address book remains in the XRD that describes me. However, WebFinger (the modern EAUT equivalent), which utilizes LRDD, moves to the host-meta XRD. This is a much cleaner approach, which resolves ambiguities about exactly what discovery is being performed on.

While I don’t claim to be particularly adept at web design, especially compared to the other two featured individuals (Tim Van Damme and Rogie King), my use of Ajax to display a status indicator was highlighted.

Since that time, I’ve had numerous people inquire as to how I added this functionality to my website. In response, I’m supplying the source code that I use. Download: web-presence.zip

Each of the instant messaging networks I have an account on (AIM, Google Talk, Windows Live, Skype, and Yahoo!) have some form of API or web-based badge which can be queried using HTTP. On my server, I am running PHP code which issues requests to the particular service and translates the response into a JSON object. When my page loads, Ajax requests are made to retrieve these JSON objects. Based on presence, the image is updated to reflect whether I am available or away.

For those interested in implementing similar functionality, this code should get you off the ground. Remember to use the required credentials or keys for your account. I’ve made notes where insertions are necessary. Good luck!

In the past, I’ve followed previous IIWs through blogs and mailing lists. I’ve always been impressed with the caliber of people that attend and the quality of work that results. This was the first IIW I attended in person, and my expectations were met.

IIW is organized in unconference style, with the agenda set by the participants. A grid is drawn on the wall, with meeting areas along the X-axis and time sessions along the Y-axis. With each day divided into slots, anyone can propose a topic and schedule it for an available room and time. The format works very well, and allows people to tailor the event to their desired focus, be it business, legal, technical, or cross-discipline.

Primarily for my own records, I’m listing the sessions I attended here. Where available, I’ve also linked to the notes found on the IIW wiki.

Day 1
Session 1: OpenID Artifact Binding (notes)
Session 2: Activity Streams
Session 3: Email Sucks What’s Next

Day 2
Session 4: Building Action Cards
Session 5: What an RP Needs
Session 6: The Hammer-Stack (notes)
Session 7: Webfinger
Session 8: Salmon Pixie Dust

Day 3
Session 9: Information Sharing
Sesison 10: WRAP
Session 11: Public Key Discovery
Lunch: Getting data into XRD
Session 12: Rich Sharing on the Web

The knowledge and ideas gained during these sessions will be influencing my projects going forward. Expect more detailed information on these topics to show up in future entries here on this blog.

I also took some photographs from the opening talks which are available on Flickr.

The publish/subscribe paradigm is commonly found throughout software engineering. In essence, a component of a system publishes events as it performs its tasks. For example, a video encoder would post an event when it has completed encoding.

Separate components register as subscribers, and receive notice when an event occurs. In our example, a content management system could take the encoded video and publish it to a website. Meanwhile, a separate component could email the director to inform him that his video is available online.

A broker in the middle (a “hub” in PubSubHubbub terminology) is responsible for delivering published events to any interested subscribers. This decoupling creates a flexible architecture, in which separate components are isoloted and can evolve independently.

There have been many attempts to define a standard publish/subscribe protocol in the past, including the XEP-0060 extension to XMPP, AMQP, competing WS-* efforts, and RestMS (which I’m unfamiliar with and need to research). None of these protocols have achieved widespread adoption. PubSubHubbub is an attempt which initially appears like it may have a better fate.

Architecturally, PubSubHubbub is decentralized and designed to scale to the size of the web. Because of this, it doesn’t suffer from any slow down or failure at a central point. I found the touting of this benefit quite humorous given the location of the meetup at Twitter’s headquarters.

One of the potential limitation of the protocol, however, is that it is server-to-server only. It operates on the backend, and is not attempting to reach the last mile to desktops or mobile phones. I suspect that this drawback could eventually work in favor of XEP-0060, which could act as a compliment to PubSubHubbub by filling this gap.

Interestingly, Brett remarked on how slowly the web actually changes. He revealed that the egress from the reference hub on App Engine is only 24Kb/second, including all of Blogger, LiveJournal, and FriendFeed. Taken at face value, this is a remarkably low figure.

To kick off the evening, Brad Fitzpatrick and Brett Slatkin (both of Google) presented on WebFinger and PubSubHubub, respectively. Brad stated that he “would like to totally decentralize social networking,” which is a common link between the two efforts. This post will outline Brad’s talk on WebFinger; PubSubHubbub will follow separately.

WebFinger attempts to solve the problem that we don’t have good identifiers for people. Over time, people have become accustomed to identifying themselves by an email address. They sign into websites with this as their ID and print it on their business cards. People instinctively recognize anything with an @ sign in it as an email address.

From an engineering standpoint, however, this is a rather stupid heuristic. Something that looks like an email address may, in fact, not be one at all. A Jabber ID is a common example. To distinguish between the two in a computer program, a URL must be formed with either a mailto: or xmpp: scheme to represent email and IM addresses, respectively.

However, it would run counter to usability to avoid “email address-like” identifiers, so a solution to this problem is necessary.

For its part, WebFinger has minted the new URI scheme acct: to denote a generic account. From this generic concept, the WebFinger protocol can be used to determine the capabilities of the account. For example, whether it can be sent email or instant messages (or both), or where that person’s address book is located.

It is important to note that this new acct: scheme imposes no burden on the user. Ideally, people will remain completely unaware of its existence. They will continue to use email address-looking strings as usual, and applications will simply treat them as account identifiers internally.

Of course, any new protocol hinges its hopes on adoption by other parties. Both Google and Yahoo! have deployed experimental endpoints. With the support of two of the big-cos, as well as independent implementations, WebFinger’s potential to become a key piece of enabling infrastructure is promising.

WebFinger derives its name from Finger, a protocol from the dawn of the Internet which was used to look up information about people on a network. One slide in Brad’s deck contrasting the two illustrated that Finger ran over TCP, whereas WebFinger uses HTTP (”the new TCP”, sigh), an aside which I found amusing.

For technical details, Eran Hammer-Lahav has a guide to implementing WebFinger, in addition to other useful information on the topic.

The web originated as a set of documents and files served up for people to read and occasionally edit. Indeed, it is this web that we all still experience as we surf from page to page reading news or looking at photos.

Underneath all this, and unseen except by software engineers, is a massive web of data. Sensors of all kinds are now measuring everything that can be quantified, from people’s location and biological functions to the environment we all share. All of this data is flowing through the network.

The web we experience will continue to be dominated by documents — e-mail, blogs, and news. And while many sites are data-centric — Google maps, Weather.com, and Yahoo finance — it’s the web that we can’t see that surging with data. It’s not about us, it’s about servers in the cloud mediating entire pipelines of data, only occasionally surfacing in a browser.

But this new influx of data is presenting challenges. Conventional wisdom often means engineers approach data with a document-centric mindset. One has to look no further than the vast assortment of XML schemas for evidence of this.

Data and documents are different, however. The suggestion, and one I agree with, is that we begin to think of data as a stream “extending infinitely forward in time” which can be “split, sampled, and filtered.”

Conceiving of data as streams moves us out of the realm of static objects and into the realm of signal processing. This is the domain of the living: where the web is not an archive but an organism, reacting in real-time.

The initial focus and reaction around Google Wave centers, rightfully, on communication. In particular, the real-time aspects were highlighted and comparisons drawn to Twitter and FriendFeed. Some even tout it as the “one true” communication tool.

It’s important to note, though, that different people have different needs. For example, both Scott Rosenberg and Fred Wilson find the interface to be complicated. Fortunately, under the open development model Google is taking, others are free to design simpler interfaces.

Looking at things from a technical perspective, the communication-centric mindset is noticeably absent. The general architecture is nothing more than an instantly distributed version control system, complete with access control and an extensible content model.

True, that forms a solid foundation for Google Wave the communication tool. It also forms a solid foundation for myriad other applications, including those yet to be imagined. (I’m starting a timer; stop me when the first FUSE file system is released.)

While its interesting to explore the impact of Wave on existing ideas, I also think its important to step away from drawing comparisons. The “killer app” for Wave is likely something that doesn’t exist currently. The synthesis of features in the underlying technology will allow something new to germinate.

It’ll be interesting to see how this evolves. I fully expect to see many front-end interfaces to Wave designed. Some of those will certainly not be geared toward communication uses for the back-end.

A concise description of what Wave is, exactly, seems to be hard to formulate. You have to see it to understand it seems to be a growing consensus. That is a bit frustrating to those trying to conceptualize, but it may be appropriate if Wave truly is a leap forward for communication.

Tim O’Reilly has the best description I’ve come across:

Jens, Lars, and team re-imagined email and instant-messaging in a connected world, a world in which messages no longer need to be sent from one place to another, but could become a conversation in the cloud. Effectively, a message (a wave) is a shared communications space with elements drawn from email, instant messaging, social networking, and even wikis.

TechCrunch also has an extensive rundown of Wave and the capabilities that were demonstrated today.

Wave is the result of a two-year long effort led by Lars and Jens Rasmussen, the creators of what became Google Maps. Their success on that project gave them the opportunity to pursue this open-ended idea. While they have raised the hype meter to new levels, only time will tell if they have another hit on their hands.

Wave remains in the prototype stage at this juncture. It was announced at a developer conference for a reason: feedback is needed from the engineering community. For it to gain public acceptance, developers will need to embrace it and build capabilities around it.

My interest has certainly been piqued. I’ve spent the day reading articles, watching videos, and researching specifications. Expect follow-up posts with more detailed thoughts.

Content, as a concept, encompasses a wide area. It could be a newspaper article, a radio program, a TV show, a spreadsheet or presentation. Creating and managing content has been one of the primary purposes of computers, ever since they came into existence.

With the rise of the Internet, and particularly social networking, there has been a renewed focus on people and the role they play in a system. Proper emphasis on individuals and groups makes collaboration more efficient.

One of the problems surrounding identity in the past has been the lack of a widely adopted standard. This has made it very difficult to build solutions that are interoperable. Thankfully, OpenID is gaining traction, along with usability improvements through EAUT and other initiatives.

Underlying OpenID, the XRD specification, emerging from earlier work on XRDS-Simple and Yadis, will tie people and content together through services that allow interaction among the two.

There is a lot of work remaining, but its conceivable at this point to envision a future where signing up for a service is as simple as entering your email address and approving access. When the mesh realizes that level of usability, innovation will both flourish and be self-reinforcing.

Developers would be well advised to pay attention to these standards. Integrating these technologies will allow software to participate in the larger web-based ecosystem, both deriving and yielding benefits from and to the whole.

As I noted in my comment:

The standards are all coming together. In my mind, the flow looks something like this:

1. People become directly addressable (i.e. through email addresses).
2. Programs dereference the email address and get an XRD document, which identifies associated services.
3. Programs interact with the discovered services to publish and subscribe to content.

However, due to time constraints, Chrysalis does not support the latest version of Rake (currently 0.8.3). The 0.8 branch reworked some of the internals, and Chrysalis was developed against 0.7.3. As such, I often need to quickly switch between multiple versions of Rake.

Thankfully, the executable RubyGems places in the bin directory makes this easy. I had long been curious how this works, so I took the time to figure it out.

The executable scripts generated by RubyGems, in this case for Rake, look like the following:

require 'rubygems'
 
version = ">= 0"
 
if ARGV.first =~ /^_(.*)_$/ and Gem::Version.correct? $1 then
  version = $1
  ARGV.shift
end
 
gem 'rake', version
load 'rake'


There is a bit of regular expression magic that works on the first argument to the command, accessed through ARGV.first. The =~ is the positive match operator for regular expressions. If a match is found, a portion of it will be stored in the global variable $1.

The goal of the RubyGems-generated script is to match a user-supplied version number, without interfering with any other arguments the actual script may expect. In order to do so, the version argument is surrounded by underscores. This is enforced by using the ^ and $ as achors for the beginning and end of the string, respectively.

The result of all this is a simple way to switch between versions easily on the command line.

$ rake _0.7.3_ --version
$ rake _0.8.3_ --version