Feed on
Posts
Comments

Announcing SLAP

SLAP - Simple Lightweight Announcement Protocol

Today, to only two close developer friends, I’m releasing a first prototype of a new project called SLAP (for Simple Lightweight Announcement Protocol). We’re going to begin multi-server testing of it as soon as possible. If you’re not a developer, you may not immediately grasp exactly what SLAP is all about, but I’m hoping it will help all of us establish our own Social Networks of One, as Jeff Pulver calls it, and not be slaves to the whims of services like Twitter or Facebook, that now determine when and how we can communicate.

SLAP is a project I’ve been developing on and off for a couple of years. SLAP provides a very minimal and fast way to alert content or status subscribers that new or updated information is available, without them having to poll constantly for it.

By eliminating polling we benefit both the subscriber and the publisher. The subscriber gets informed of new content immediately, with no polling period latency and no resources of computing or bandwidth are expended between announcements. Likewise, the publisher does not need to provide sufficient bandwidth and computing resources to respond to polling requests, the vast majority of which result in no new information.

Unlike HTTP and XMPP, SLAP is connectionless, employing UDP (User Datagram Protocol) as its underlying transport mechanism. This is why no significant server or network resources are used between announcements. Although XMPP provides the same immediacy of delivery and elimination of polling, it requires long-lived connections to be maintained even if the frequency of announcements is very low, e.g., once a week. HTTP does not require long-lived connections, but instead requires establishing and terminating much TCP connection overhead in a short amount of time to deliver a very small amount of data.

The SLAP model

SLAP defines announcements in terms of feeds and posts to those feeds. Each feed is identified with a URI and each post with a perma-link URI. Further, each announcement is uniquely identified independently of the feed and post it is related to. This allows many announcements of the same feed and post. For example a post may be initially created, then edited many times, or commented on many times. Each edit or comment can result in a new announcement of a change to the post.

Publishers and Subscribers

There are two basic roles in the use-case model for SLAP; the Publisher and the Subscriber. The Publisher creates new feed content and emits a SLAP event every time that feed content changes that goes to every Subscriber. The Subscribers receive the announcements and respond with an acknowledgement. The announcements and acknowledgements are all delivered using UDP, which means no connection overhead for either the Publisher or the Subscriber. The protocol is tolerant of off-line Subscribers and noisy, unreliable networks. Publishers retry unacknowledged announcements at a fixed period and for a limited amount of time. One of the strengths of SLAP is that a missed announcement is not the end of the world. The Subscriber can periodically check the Publisher for all feed content and discover a missed post that way.

Strictly speaking, SLAP is not intended to deliver content, but only to announce that new or modified content is available on a feed and to provide the URIs for the feed and the post that allow the subscriber to fetch the content. SLAP does provide for a content summary that may in many cases be all the content there is. Microblogging or automated system status updates would tend to use this feature. In these cases, an attribute of the message could show that the summary is the entire post content. This is an optimization that enables a subscriber to avoid making a content fetch when the summary is all that there is.

Having the publisher be responsible and obligated for content storage allows a subscriber that has been offline for an amount of time to fetch any posts it missed simply by reading the feed as it normally would.

Separation of subscribing and fetching from announcing

The core SLAP messaging services deliver and acknowledge announcements, but play almost no part in establishing or terminating subscriptions. Nor do they specify how fetching of post content happens. Subscriptions can be established in any number of ways, as long as the required information for announcement delivery and authentication is provided. The subscription protocol is an open development issue, and your comments and suggestions are welcome. Personally, I see a simple extension to RSS as being the most obvious route. This would allow common blogging or microblogging applications like WordPress or Twitter to add announcement support with a relatively small modification to their current RSS support. RSS feeds could allow SLAP-enabled subscribers to discover the SLAP parameters and automatically connect to those feeds providing SLAP announcements.

Authentication

Any protocol that “pushes” information to a destination opens a possible hole for spammers. One of the challenges to making SLAP work is enabling subscribers to authenticate incoming announcement messages as to their source and to do it relatively quickly. This first version of SLAP uses a simple MD-5 hash of the message contents with a subscriber generated key that is exchanged at subscription time, ideally using secure communication.

Try it, you’ll like it.

SLAP can be used just for publishing, just for subscribing, or both. You’ll need to run it on a machine that you can open the firewall on, and receive UDP messages on ports 14947 for the subscriber and 14948 for the publisher. Or you can use any SLAP server that you’re given access to.

Many thanks to Jeff Pulver for creating a stimulating mini-unconference on March 10th called Social Media Jungle Boston, and particularly for inviting me to speak on the topic “The End of the Compuserve Era of Social Networking.”

Jeff Glasson of PerkettPR did yeoman duty videotaping all of the presentations that day. I can’t imagine how much time he’s spent editing it down. Here is Jeff’s recording of my session.

I had a brief interchange with Jeff Cutler on Twitter just now about the old bloggers-aren’t-journalists debate. This on the heels of listening to Daniel Schorr on NPR yesterday talk about how it’s disturbing to him that anyone can “publish” things on the internet and no editor or staff fact-checks it or holds it to any journalistic standard.

For what it’s worth, I think journalists are unnerved and defensive because one of the mediums that they’re published on now is open to anyone to publish.  They seem to feel that this is unfair. Because their access to the web is controlled, they seem to feel everyone’s should be.

Mr. Schorr, on that same NPR show, said that the internet was like a market or bazaar where anyone can come and shout anything they want and be heard by anyone who cares to listen. This is a particularly apt analogy. If we apply it to journalists on the internet, we see that it’s like making them stand on a milk crate and announce the news on the street corner just like everyone else, whereas before they had special arrangements. They spoke from a balcony with a loudspeaker.

And that’s the inversion that I think not everyone appreciates. It’s not so much that we, the bloggers have invaded the journalists’ turf, it’s rather that they’re now forced to operate on ours. A medium used to come with a certain implied or built-in credibility and certainly privilege. If you got to speak, it was sort of assumed that at least you had your facts more or less straight.

But access to a medium does not necessarily equate to credibility and really, it never has. Anyone with sufficient money can print their own newspaper. But just because something is printed on sheets of paper that look like a newspaper doesn’t mean it’s true. Trustworthy journalists and news institutions earned their stripes over the years by doing their jobs well and being proven out by facts and people’s real-world experiences.

I think journalists should believe in their own credibility a little more and realize that most of us out here reading things on the internet value reputation and track record just like we did before. Sure, there are still plenty of people who’ll choose the equivalent of the supermarket tabloid or Fox News, but you never had them anyway. It not that you’re standing on a street corner with everyone else that matters, it’s what you say when you get there.

My Social Media History

A couple of weeks ago, I was contacted by Ravit Lichtenberg (@ravit_ustrategy) to help with some perspective on a blog post she is preparing. Ravit is the founder of Ustrategy LLC, whom I met through one of her clients, Julie Rorrer (@juliedarling), a fellow Connecticut Twitterer. (Really, how else does anyone meet these days except through Twitter?)

I’m afraid I gave Ravit way more information than she was really interested in, but in the process of answering her questions, I really got into the retrospective mood and went into some detail of how I got into this online maelstrom called Social Media. After sending it off to Ravit, I got to thinking it might make a good blog post. I mentioned to this Ravit in a subsequent exchange today and she encouraged me to make it public. So, here, with almost no editing excecpt for formatting, is “My Social Media History.”

How did you start with Social Media?  Why did it attract you?

Ravit,

I hope this isn’t way more information than you wanted, but once I got started, I found myself enjoying the romp through history. :) I’ll send the answers to your original questions tomorrow, ok?

As best I remember, I got introduced to social media through videoblogging. Specifically, I started watching Rocketboom back in 2005, when Amanda Congdon was the host. I began to comment on the rocketboom site and on Amanda’s personal blog. I also got interested in a little video blog called DriveTime, which featured a guy named Ravi Jain and his wife Sonia from Boston who did a talk show in their car during their daily commutes. What really struck me about Ravi and Sonia’s show was how intimate the videoblog medium was. After just a few episodes, I felt like I knew them personally. Video is very unique that way. It forms a personal bond the way writing or audio never can. And I suppose that was the “social” aspect that really drew me in. The possibility of getting to know smart, creative people through their web presence really intrigued me.

In July of 2006, Amanda Congdon had her infamous firing with Rocketboom’s founder Andrew Baron. After leaving Rocketboom, she started her own personal videoblog from Connecticut, which I started to comment on fairly regularly. Finally, she landed a new gig and kicked it off with a launch party in New York City in Sept 2006. The party was hastily arranged at the new offices of the pioneer video hosting service blip.tv. I decided to go with the hope of meeting a real “web-celebrity”. I will never forget meeting Amanda. I introduced myself and she said, “Oh! You’ve been commenting on my vlog!” and gave me a big hug. I have to say, if anything cemeted social media for me, it was that experience. Here was someone known by millions, but still accessible and a “real person”.

Now, you can’t be into videoblogging and not know about Steve Garfield. Steve is the Father of Videoblogging and figured out, on his own, how to post video to a blog back in January of 2004. Steve runs a monthly meeting called Boston Media Makers that attracts people in video, audio, radio, TV, newspapers, art, acting, film, blogging and just about any other topic under the heading of “media”. I started going to those meetings back in late 2005 and met Steve (with whom I’ve become friends) and Ravi Jain.

In October 2006, I also started my own videoblog, Joe’s Video, Etc. I used a digital still camera (and still do) that takes video and did some simple editing with Windows Movie Maker. I’d started doing some political text blogging earlier than that, but my largest contribution in blogging came with video.

Later that year, and into the first part of 2007 and SXSW, I got onto Twitter and have been totally hooked ever since.

You are, as you point out, not the usual demographic for all things social media and microblogging (I’m probably on the edge myself). You’re also not in the (stereo)typical geography—i.e., Silicon Valley, NY, Austin, Portland.  You must have a truly unique perspective on things and how Social Media is being used around you

I am near Boston, though, which has a very prolific and active social media community, many of whom are involved in marketing and PR. Over the past year or two, I’ve made many new friends, including Chris Brogan (@ChrisBrogan), Laura (@Pistachio) Fitton and ex-Bostonian Julia Roy (@juliaroy).

How do YOU define social media?

Social media is people conversing and sharing using text, audio, video and (gasp!) actual in-person meetups. Social media also has an explicit notion of networking. Building a social network of “friends” or “followers” is intrinsic to the notion of social media.

What do you use it for?

Meeting people in a purely social sense. Social media has allowed me to find a large group of people that I like and that like me. And the great thing is that liking each other is the primary thing that binds us together. Not work, or profession or common interests, although there are a lot of those. Yeah, I would say that’s the primary thing.. people I like who like me, too.

Secondarily, it has turned into my method of getting work. Networking has always been a key to success in business, but social media has kicked it into orbit. I’ve found the key is to NOT try to exploit it overtly for business. That’s where a lot of social media and esp. Twitter newbies fall down. They try to use it as another channel to push their message out. The thing you have to understand about social media is that it’s like a string. You can’t push on a string, you can only pull.

How do you see it evolving?

I like to say that we are in the Compuserve era of social networking, in which there are a lot of services, each providing essentially the same thing that do not interoperate, that are vying to try to capture users from each other. I hope that the future lies in open, standard protocol based social networking instead of closed, walled gardens. The Open Stack initiative is working in this direction, although so far it isn’t clear how it would find a path to success.

If you’re a golfer like me, or familiar at all with the sport, you’ve probably heard of Natalie Gulbis. She’s the leggy, blond golf pro who not only looks good, but has the game to have won on the LPGA tour, which is no mean feat. Most professional golfers struggle for years and never come close to winning a tournament. She also was a standout in the Solheim Cup in 2005, helping the US to a win in the team matches with Europe.

This week, I discovered that she’s now on Twitter as @Natalie_Gulbis. She is, as far as I know, the first top-tier professional golfer of any tour to be on Twitter. Yay, Natalie!! That she would be the first is totally in keeping with her accessible, outgoing and friendly personality.

At this writing (Dec 29. 2008), she only follows 4 people and is followed by only 234. But I give her and her agent major props for realizing Twitter is a good place to be. Like anyone new to Twitter, it may take a little time to discover how it works. I’m hoping that unlike many celebrities, she will really get in and talk with people.

Of course, someone like Natalie Gulbis needs to be careful with the public. I can imagine she has more than her share of unwanted attention. But at least on Twitter, you don’t have to hear from anyone you don’t want to. And there is so much to be gained from engaging with people and making new acquaintances and friends, not to mention drawing non-golfers into the sport.

If I may make a suggestion, it will be the same as to anyone new to Twitter. Listen first, follow first. If you hear something interesting chime in. Make an observation, a joke, ask a question or state your opinion. Most importantly, be a person, not an inaccessible celebrity. Twitter is a conversational medium, a means for introducing and connecting people, not just another broadcast channel. Let that sunny personality of yours show through. You’ll find we’re a friendly lot and you’re more than welcome to join the conversation.

WordPress plugin question

If there are any WordPress plugin coders reading this, I need your help.

I’m writing a WordPress plugin to add Open Microblogging Support to a blog. In order to do this, I’d like the plugin to add a special OpenMicroblogging Page to the blog. This special Page is supported by a custom Page Template. Page Templates are normally added to the theme directory. And therein lies my lack of understanding. How do I get a plug-in to add a page template? Copy the php file in the installer code? Seems a little dodgy, but maybe?….

@Pamelump (Pamela Leigh Seiple) just tagged me with this photo meme. I have to go to the 6th photo on my 6th Flickr page and write a blog post about it.

Ok, so the aforementioned photo (6,6) is this one:
TheBoulevardDiner-Worcester,MA 007

This photo is from a set I made about a year ago, on a very cold day in Worcester, Massachusetts. I had lunch with two diner fanatics, Scott Monty and CC Chapman at an old haunt of mine from my college days, The Boulevard Diner. You can read the story on the set page, and there’s a video to go with it, too.

But probably my most lasting memory from that day is not that photo, but this one:
TheBoulevardDiner-Worcester,MA 025
I blogged about it here: Accidently Memorable.

After spending quite a few hours on yesterdays post, OAuth and OMB post-sharing, it occurred to me with the first cup of coffee this morning that I had missed one important user interface twist in the use of OAuth that actually makes it quite necessary.

It depends on how the UI for subscribing works. If, as with a feed reader, the subscriber enters the URL of the person they wish to follow, authorization is unnecessary. The user has made their intention known to their own account, and can simply request the posting server to send updates.

However, if the the subscriber subscribes by visting the site of the poster, and the poster allows unrestricted access, then there must be a mechanism for the subscriber to tell his own site that he wishes to subscribe to the poster’s updates. This is where OAuth comes in, and it is indeed a well-matched use-case for OAuth’s functionality.

Apologies to Evan at Identi.ca.

OAuth and OMB post-sharing

This post has been superseded. I missed an important web UI issue which lead me to an incorrect conclusion.

Square peg, round hole

The OpenMicroBlogging Specification (OMB) contains a section on the use of OAuth to authorize a remote service to post to a user’s local service. After spending some time studying the OAuth and OMB specs, it seems to me that this is not the use-case that OAuth was designed to solve. A square peg in a round hole, you might say.

This is not to say that authorization is not needed for allowing post-sharing, only that OAuth was designed to address a different situation. The simplest way I can characterize the difference is that OAuth applies to a “one human, two services” problem and microblogging post-sharing is a “two humans, two services” problem.

OAuth’s intended use-case

So, what problem does OAuth fit? OAuth is intended to solve the problem of a person who has content or resources on Service A and wants to allow his/her account on Service B to access those A resources - without giving Service B his/her login credentials for Service A. Note well that it is the same person having accounts on both A and B. This is the “one-human, two services” aspect.

So, for example, let’s say you have an account on Twitter and want to use one of the many alternate Twitter clients out there, e.g., Twalala. You have an account with Twalala and would like to be able to access your account on Twitter without having to fork over your Twitter password. Using OAuth, you would be able to, in essense, tell Twitter to allow your account on Twalala.com to perform various read or post functions with your Twitter data. By the way, Twitter has been promising for months to implement OAuth for exactly this purpose. Again, note that there is only one person with accounts on both systems.

OMB post-sharing use-case

Now, let’s look at the case of two people on two different OMB-compliant sites who want to share their posts, either unilaterally or bilaterally. Let’s say I have an account “joecascio” on site subscriber.com, and my friend has an account “johnsmith” on poster.com and I would like to follow John’s updates. Note that it should not be necessary for either me to have an account on poster.com nor John to have an account on subscriber.com in order for me to “see” his updates. Indeed, this is the whole point of federated microblogging; that we can share posts without having to have accounts on many different services. So simply from this fact alone, OMB post-sharing is clearly not the “one person, two accounts” use-case. If post-sharing worked by requiring John to have an account on subscriber.com (the second of the “two services”) and then authorizing subscriber.com to let his Poster.com account post there, then yes, that would be the OAuth use-case. But it isn’t!!

So, what is an appropriate protocol or process for enabling and controlling post-sharing? I don’t have the complete answer, and I’m hoping that this post will cause some of the many smart people out there to suggest some. But I can state at least some of the problem characteristics, requirements or constraints.

Post-sharing is subscribing, not cross-posting

When I register my interest in someone else’s microblog posts, when I “follow” them, it means I want to subscribe to their post stream, just like I would subscribe to their traditional blog. If it were a traditional blog, I would do this by polling their RSS feed which is a simple GET by the subscriber. It does not involve the poster initiating a transaction to “send” me anything. But since microblogging implies a much lower latency, which would be inefficient to perform by polling, OMB introduces an optimization. The service on which the post was originally made proactively sends the new post to all subscribers. Great! We can now be notified that a new update is available within a few second without having to poll every few seconds.

BUT this is point of my observation. I would argue that this proactive send is merely an artifice of the optimization and does not change the fundamental relationship of poster and subscriber. From that basic user point of view, the fact that the poster initiates the data transfer is immaterial. The subscriber is still simply seeing something they expressed interest in reading.

This is where I believe OMB goes astray. It lets an optimization detail obscure the higher level concept. Following someone’s posts means I want to see those posts in my view, but it does not require that either party perceive it as cross-posting to my site. I could equally accomplish the same viewing of that post by fetching it on demand, albeit not as efficiently.

Subscribing is implicit authorization

Keeping firmly in mind the fundamental concept of poster and subscriber, what if any authorization needs to happen when I request to follow ‘johnsmith’? I would say that the very fact I am requesting to see John’s posts is an implicit authorization that he can send them to me. Whether he wants to let me see them is a different story and the subject of another post. Let’s assume he will let me see his posts and we want to use the OMB optimization of his service (poster.com) proactively sending them to me.

When I request to subscribe, I need to send a URL of where to send the notifications and a means to verify that the sender is who I granted the privilege to.  This could be accomplished to different degrees of security. A fairly loose method would be for the subscriber to send a callback URL in the request that encodes a UID for the poster’s account. If the request transaction is secure, we can be reasonably certain that only the the “real” poster will be sending POSTs to that URL. Of course, this does not prevent the poster from sharing the UID, but the subscriber can always simply invalidate the URL and stop receiving the notifications.

In fact, this is pretty much how the RSS cloud API works and this could be a more proper model for OMB post-sharing.

What do you think?

I would be very interested to hear what others have to say about this post. One thing it does not cover is the complementary function of a subscriber GETting, rather than being sent an update. It also does not cover the perfectly matched use of OAuth to allow a user to grant access to other of his/her services just as described in the Twitter/Twalala example under “OAuth’s intended use-case”.

Work for yourself

I just got through watching this video of Jason Fried that was recorded earlier this week in Boston. It’s the long-form, illustrated-with-slides version of the extemporaneous talk he did for RI Nexus this past Wednesday in Providence. I highly recommend this to anyone involved in software development.

Listening to his list of specific dos and don’ts in more detail just now made me realize that there’s one overarching philosophical tenet that guides all the details and that is the following.

Run your business and build your product for yourself, not for investors or shareholders or analysts or “big” customers. Run your business to sustain your own life-style and your own integrity. Build things you believe in, not what some purchasing manager put in an RFP.

Investors want you to grow without bounds. That makes you put in crap you don’t want. Salesmen want you to put in a useless feature so they can get a big sale. Big customers want you to put in things only they want. While some of these forces may have some benefits, in the long run, they only push you to do things that dilute or muddy up the product with noise. Truly great products are simple, coherent, easy to understand and perform quickly and reliably.

For me, this means satisfying the artist, artisan and craftsman in myself, not somebody who makes money off of what I do. That is where great works come from; not from a balance sheet, an enhancements list, a specification or god knows, Microsoft Project.

Older Posts »