Edit Tweets after they’ve been posted

For some reason, people often don’t see errors in text they’re editing until the text is posted publicly. I think this has to do with the fact that publishing often changes the line breaks, font, background, foreground text color, formatting or whatever so that the error suddenly becomes apparent.

This happens to me once in a while when using Twitter. In Janetter, the edit box for composing a tweet stretches the text out in one long line. But after being posted and appearing in a stream column, it’s line-broken into 4 or sometimes more lines depending on column widths. Once posted of course, there’s no way to correct the error except by deleting the tweet and re-entering it.

Now, what would be really great is to be able to edit a tweet after you’ve posted it, like you can edit a post on Google+. Twitter doesn’t support this, but Janetter could make it appear that it was simply by putting an “edit” button or link on the posted tweet and letting you edit the tweet (ideally, in place in the column) and then reposting the tweet and deleting the old one, all in one step. The new tweet would appear at the top of the column with its correct, new post time and the old one would disappear, reflecting what was actually done behind the scenes. This doesn’t do anything you can’t do yourself manually, it just makes it more convenient by combining several steps into one and corresponds better to the user’s mental model of “where” the tweet is, so to speak. The picture below attempts to illustrate how I think it could work. Click on it to get the full-size version.

I wouldn’t expect Janetter to try to maintain the conversation links to the old tweet, or anything else that gets deleted along with the old tweet. That would make the problem much more complicated and probably impossible, given the twitter API. The user just gets what they get when they manually delete and repost now.

The people I know on Twitter would love this little convenience feature. How about you?

Enable the ESC key to cancel photos and profile dialogs

This is a consistency and convenience issue. I instinctually hit the ESC key to cancel dialogs and I think it’s always good to give the user the choice of using the keyboard over the mouse when possible. Please note I would add this key response to the current method of clicking outside the dialog and not replace it! It doesn’t have to be an either/or choice. Both can work, right?

Mouse Rollover pauses column update

One of the things that drives me crazy about all auto-updating Twitter clients is that they inevitably update just as I’m about to do something with a tweet. The tweet I’m interested in gets scrolled away, and I end up clicking on the wrong tweet!

Now, I know that clicking on a tweet in a column in Janetter will stop updating in that column, but that’s not exactly what I want. I only want updating to pause while the cursor is over that column and to restart automatically when the cursor moves off, OR when the cursor hasn’t moved for say, a minute. Here’s a simple illustration of how it might work. Again, click on the picture to make it full size.

Janetter has built a really good product. I hope they’ll consider these improvements. I happily offer myself as a development tester.

The way AdTraqr works is that you create a “target” record in the database that contains a URL you want the user to be directed to when they scan the code.

The alpha test, which just ran in the New London, CT local paper The Day contained a QR code for each of three properties belonging to a realtor friend. These went into a 1/4 page size ad in the real estate supplement that is inserted in the paper on Fridays. To my knowledge, this is the first use of QR codes in advertising in The Day.

Click on the image first to get the full-size version and then try scanning the codes with your smart-phone.

What you should see is the agency’s property page for the listing you scanned with all details about price, square footage, etc.

The “tracking” part of the service comes in when you place different codes for the same target in multiple papers, real estate flyers, on the property sign or any other place. Each placement for the same target has a different QR code. When someone scans the code, it actually goes to a URL on AdTraqr.com, which increments a count in its database of how many times that particular QR code was scanned, and then redirects the user to the actual target page, in this case a real estate listing. The agent can then look at their account page on AdTraqr and see how many times each placement was scanned. This gives them direct information about the effectiveness of each placement, which can help them decide where their advertising dollars are best spent.

And of course, any business that advertises can use QR codes in this way, not just realtors. A retail store that advertises in a wide area could tell which town produced more customers by putting a different code in each local paper or flyer version that they publish.

There are many ways to use QR codes and AdTraqr. A web-savvy business will create different landing pages for each QR code or target that they produce. Coupons or discounts can be implemented this way or information on new products or services.

The exciting thing about QR codes is that they make print an extension of the internet, and that’s an intriguing concept.

AdTraqr.com is in private alpha right now. I expect to go public in maybe another month, at which time, I’ll update you. Thanks for reading, and go find a QR code to scan!

SLAP – Simple Lightweight Announcement Protocol

Today, to only two close developer friends, I’m releasing a first prototype of a new project called SLAP (for Simple Lightweight Announcement Protocol). We’re going to begin multi-server testing of it as soon as possible. If you’re not a developer, you may not immediately grasp exactly what SLAP is all about, but I’m hoping it will help all of us establish our own Social Networks of One, as Jeff Pulver calls it, and not be slaves to the whims of services like Twitter or Facebook, that now determine when and how we can communicate.

SLAP is a project I’ve been developing on and off for a couple of years. SLAP provides a very minimal and fast way to alert content or status subscribers that new or updated information is available, without them having to poll constantly for it.

By eliminating polling we benefit both the subscriber and the publisher. The subscriber gets informed of new content immediately, with no polling period latency and no resources of computing or bandwidth are expended between announcements. Likewise, the publisher does not need to provide sufficient bandwidth and computing resources to respond to polling requests, the vast majority of which result in no new information.

Unlike HTTP and XMPP, SLAP is connectionless, employing UDP (User Datagram Protocol) as its underlying transport mechanism. This is why no significant server or network resources are used between announcements. Although XMPP provides the same immediacy of delivery and elimination of polling, it requires long-lived connections to be maintained even if the frequency of announcements is very low, e.g., once a week. HTTP does not require long-lived connections, but instead requires establishing and terminating much TCP connection overhead in a short amount of time to deliver a very small amount of data.

The SLAP model

SLAP defines announcements in terms of feeds and posts to those feeds. Each feed is identified with a URI and each post with a perma-link URI. Further, each announcement is uniquely identified independently of the feed and post it is related to. This allows many announcements of the same feed and post. For example a post may be initially created, then edited many times, or commented on many times. Each edit or comment can result in a new announcement of a change to the post.

Publishers and Subscribers

There are two basic roles in the use-case model for SLAP; the Publisher and the Subscriber. The Publisher creates new feed content and emits a SLAP event every time that feed content changes that goes to every Subscriber. The Subscribers receive the announcements and respond with an acknowledgement. The announcements and acknowledgements are all delivered using UDP, which means no connection overhead for either the Publisher or the Subscriber. The protocol is tolerant of off-line Subscribers and noisy, unreliable networks. Publishers retry unacknowledged announcements at a fixed period and for a limited amount of time. One of the strengths of SLAP is that a missed announcement is not the end of the world. The Subscriber can periodically check the Publisher for all feed content and discover a missed post that way.

Strictly speaking, SLAP is not intended to deliver content, but only to announce that new or modified content is available on a feed and to provide the URIs for the feed and the post that allow the subscriber to fetch the content. SLAP does provide for a content summary that may in many cases be all the content there is. Microblogging or automated system status updates would tend to use this feature. In these cases, an attribute of the message could show that the summary is the entire post content. This is an optimization that enables a subscriber to avoid making a content fetch when the summary is all that there is.

Having the publisher be responsible and obligated for content storage allows a subscriber that has been offline for an amount of time to fetch any posts it missed simply by reading the feed as it normally would.

Separation of subscribing and fetching from announcing

The core SLAP messaging services deliver and acknowledge announcements, but play almost no part in establishing or terminating subscriptions. Nor do they specify how fetching of post content happens. Subscriptions can be established in any number of ways, as long as the required information for announcement delivery and authentication is provided. The subscription protocol is an open development issue, and your comments and suggestions are welcome. Personally, I see a simple extension to RSS as being the most obvious route. This would allow common blogging or microblogging applications like WordPress or Twitter to add announcement support with a relatively small modification to their current RSS support. RSS feeds could allow SLAP-enabled subscribers to discover the SLAP parameters and automatically connect to those feeds providing SLAP announcements.

Authentication

Any protocol that “pushes” information to a destination opens a possible hole for spammers. One of the challenges to making SLAP work is enabling subscribers to authenticate incoming announcement messages as to their source and to do it relatively quickly. This first version of SLAP uses a simple MD-5 hash of the message contents with a subscriber generated key that is exchanged at subscription time, ideally using secure communication.

Try it, you’ll like it.

SLAP can be used just for publishing, just for subscribing, or both. You’ll need to run it on a machine that you can open the firewall on, and receive UDP messages on ports 14947 for the subscriber and 14948 for the publisher. Or you can use any SLAP server that you’re given access to.

It depends on how the UI for subscribing works. If, as with a feed reader, the subscriber enters the URL of the person they wish to follow, authorization is unnecessary. The user has made their intention known to their own account, and can simply request the posting server to send updates.

However, if the the subscriber subscribes by visting the site of the poster, and the poster allows unrestricted access, then there must be a mechanism for the subscriber to tell his own site that he wishes to subscribe to the poster’s updates. This is where OAuth comes in, and it is indeed a well-matched use-case for OAuth’s functionality.

Apologies to Evan at Identi.ca.

Listening to his list of specific dos and don’ts in more detail just now made me realize that there’s one overarching philosophical tenet that guides all the details and that is the following.

Run your business and build your product for yourself, not for investors or shareholders or analysts or “big” customers. Run your business to sustain your own life-style and your own integrity. Build things you believe in, not what some purchasing manager put in an RFP.

Investors want you to grow without bounds. That makes you put in crap you don’t want. Salesmen want you to put in a useless feature so they can get a big sale. Big customers want you to put in things only they want. While some of these forces may have some benefits, in the long run, they only push you to do things that dilute or muddy up the product with noise. Truly great products are simple, coherent, easy to understand and perform quickly and reliably.

For me, this means satisfying the artist, artisan and craftsman in myself, not somebody who makes money off of what I do. That is where great works come from; not from a balance sheet, an enhancements list, a specification or god knows, Microsoft Project.

What really impressed me about Jason’s story was that he became a success doing things the way I’ve always thought they should be done. And more to the point, exactly the opposite of the way traditional management tells you it should and must be done. Some examples from his talk.

1. Don’t get venture money. Bootstrap your idea in your spare time keeping your day job. You don’t want anyone telling you what to do who doesn’t share your passion for what you’re doing.
2. Don’t have a board of directors. What do they know about what you’re trying to do?
3. Don’t write specs, write code. You can’t tell whether something is good or bad from a spec, only from working code. Do it quick. If it’s no good, throw it away and move on. People in organizations are dreadfully afraid of making a mistake. Don’t worry about it, just do it.
4. Don’t keep an enhancement or bug list. Trust your gut to do what’s most important. When you keep hearing the same thing from people you know and trust, go ahead and do something about it.
5. Try to do everything in two week increments. That is, don’t let people get bored doing the project. They do their best work when they’re excited about the outcome.
6. Don’t plan too much. My question to Jason was about whether or not his small 12 person operation would scale up. He said, “I don’t know and I don’t care. We’ll worry about that when the time comes.”
7. Don’t hire any marketing people. Your customers are your best marketing dept.
8. Trust your employees. 37 signals gives each of their employees an unlimited charge card, to spend on whatever they want, however much they want. The company bought one employee flying lessons. Not because it would be a usable skill, but because it would make him a more happy and well-rounded employee. In their experience employees do not abuse the charge privilege, but use it sparingly, knowing that it comes of the bottom line.
9. Don’t spend money on an office. Keep employees in their own homes. That’s where they’re most productive. In an office it’s too easy to be interrupted and too easy to call a meeting. Meetings are productivity killers.
10. Charge money for your product from Day One. This is a complement to the Don’t take money from VCs rule. Nothing succeeds like revenue. If you have something worth paying for, people will pay for it. If it can’t sustain itself, find something else to do.

I’m sure there were more, but those are the ones that stuck with me. As a survivor of 35+ years in the software development business, I can personally attest to the practicality of these guidelines. I have seen the opposite fail miserably for my whole career, but every once in a while, I managed to become part of something that didn’t obey the rules, and those were the memorable successes.

Don’t let the management, marketing and financial suits tell you any different. Do what you know is right. If it doesn’t work, learn from the experience and don’t make the same mistake again. But please, don’t think you can ever get it right without trial and error. It’s so obvious it’s painful, but there are still legions of project and product and people managers who wouldn’t have jobs if not for torturing you with requirements statements, schedules and bug reports.

And one thing that he swears will never be added to BaseCamp, their project management tool: Gantt Charts. Hurray!!

So I’ve been putting together the following list of what I consider to be important features that a good microblogging function should have. In most cases, I’ve approached it from how it should appear to the user. If you are coming to DevHouseBoston on Sunday, you might want to read this list over and come prepared to argue the merits of different points, add your own, or if you’re a UI designer, contribute to a discussion of how to design a UI that incorporates these features.

Remember that one of the benefits of an open system is that there can be many different UIs to choose from!

So, following, is the list in a very raw and unpolished form.

Note: DMB means Distributed Microblogging.

Feature list for dmb UI

Omnipresent replies, direct messages tabs or buttons.
Tags on other microblogs, on posts, on other tags?

Direct support for conversations, replies, etc.
Main page. Like twitter. Update input box at top, but maybe with separate scrolling list of stream.
Multiple streams showing, replies, all, selected person’s archive, including user’s archive.
“Hide” check boxes for each user. Clicking it removes their updates from stream list, and places their name in the a “hidden” list for easy reinstatement.

Also would be able to show/hide certain groups.

Conversation support. Each tweet would have a “reply” button. Clicking it would add the @user_name to the beginning of the update, and also make a database entry linking the update to one or more previous updates. The @’d user names would be like the “to” list on an email. Each update could have multiple other updates it was in-reply-to. This is a many-to-many relationship. An update can be in response to multiple other updates, and clearly an update can have many responses to it.

Updates have GUIDs. Required by IMPP.

Tabs for different subsets of users = from Mike Gaines. “tag-tab”

Separate input area for different tag-tabs.

Topics for User experience
Conversations. What can you do? How are they shown?
Multiple addressees (like email?)
Latest on top or on the bottom? User pref?
Classes of messages
Direct
Narrowcast (to a group tag?)
Broadcast (or is this simply to all tags?)
Temporary silencing (show/hide checkbox) So I can turn off someone who is usually ok, but is just ranting or something.

Ad-hoc grouping. Drag users into a tab where you see only their tweets.
Public ad-hoc groups. All your followers can see tweets. You just filter locally.
Private ad-hoc groups. Just the people in the group see the tweets.

General question.. will adding private groups destroy some of the vibrancy and serendipity of Twitter?

How to present tagging in the UI so it’s intuitive with what it does for the user, as opposed to a geeky abstract function?

What can be tagged, and hence grouped? Users, convos, even other tags? Locations? languages? specific tweets having to do with a certain subject? others?

Universally unique tags vs. private tags. UUtags would be very helpful for organizing large tagging efforts. But, how to agree on a tag for say, “SocialMediaBreakfast8″?

theRQ would be great to be able to send/post an out “out of stream” message to someone that they get upon login- wouldn’t age off. (subject to spam abuse?)

Search – Let Google do it? What about private mblogs? Can individual servers be expected to support a general search capability? Thru XMPP? Thru HTTP?

Show me when I have new, unread DMs.

Multiple addressees for DMs. Now it’s beginning to look like email.

Show me when I have new, unread messages.
Show me when I have new, unread messages from specific people or groups. (on that tag-tab?)

Auto-update (ala chat) vs. manual update.

Rich content. Photos, audio. (How to transmit to mobile devices?)

How to add these features without sacrificing the simplicity?

A few lines when you request to follow, so you can explain why. Either that or a listing of common friends on different sites.

What about the Twitter community? Can it be made part of DMB? Can we build an interface to Twitter? Will Twitter participate in DMB?

What about other communities that start up? Twitter has very amorphous sub-communities.

Implementation questions:

Use XMPP archiving or external? (some xmpp archiving options not applicable, like off the record, although they might be of interest)

How to do multiple servers?

How to map mb posts onto “collections” in xmpp archiving. Another reason to use external archving?

If we did external archiving it would relieve the burden of archiving on servers that could be used. Although they’re more likely to support archiving than pubsub.

Ok, so let’s talk about the hard bits of doing Distributed Microblogging. It’s easy to envision a multitude of servers exchanging microblog posts, and a UI that simply arranges the posts in chronological order. By the way, chronological order is easy to do if all the servers are synched with a time standard like nist.gov, and most are. But the hard bit is making it work in a way that performs well and scales to large populations of both microblog posters and readers. I’ve been thinking of some different alternatives for this, which I’ll lay out here. As always, your thoughts are welcome.

Performance

So, what do I mean by “performs well”? Well, microblog (ie, Twitter) updates happen much more frequently than what we’d consider traditional blog posts, but not quite as fast as instant messenger or chat updates. And microblogs don’t have the notion of presence attached to them. You don’t worry about whether a Twitter poster is “on line” at any particular time, although you may deduce that from the rapidity of their responses to you.

To put a number on it, I’d say that a microblog post notification should be transmitted in less than a minute, ideally in a few seconds, although a delay of up to 5 minutes is not that objectionable. I frequently will ignore my Twitter feed for many minutes, sometimes hours. I have no expectation that I will see someone’s posts immediately (i.e., less than a second) nor do I care to. Microblogging is a river of updates. You don’t expect to see every single one.

Size

Now let’s talk numbers of followers and following. A typical Twitter user has 100 or fewer people that they follow, and a similar number that are following them. But the edge cases are far bigger. Someone like Robert Scoble (@scobleizer) or Chris Brogan (@ChrisBrogan) follow literally thousands of other users, and have even more following them.

Ok, so we need to be able to update thousands of users in nominally less than a minute, but at the exreme less than five minutes. If you don’t buy that, please leave a comment explaining why these limits are not realistic.

Alternatives

RSS feed polling

The simplest solution would be polling, which is reader initiated. Followers would poll the feeds of the microblogs they were interested in. It seems to me that polling has to be discarded as a solution except for occasional or retrospective uses. I think a solution should include RSS feeds but it seems obvious to me that for someone with 100 friends to poll those RSS feeds every minute, or ideally faster because they might post something is a stupendous waste of bandwidth.

On the sending side, if that same person had 100 followers, each of those followers would be polling the person’s RSS feed every few seconds causing a very high server load. Now multiply that by however many people’s accounts are hosted on that server and the problem quickly blows up.

Notification

A much more efficient solution would be for senders (microblog posters) to notify followers when a new post has been made, and perhaps to proactively send the new content in the notification message. Then, resources are used only when there is actual traffic to send. Both senders and receivers are quiescent when no one is posting. So what are the possibilities for implementing notification? I see the following.

1. RSS cloud API – a little-known part of the RSS specification, the cloud element allows a feed to publish a web-service address that readers of the feed can register with to be notified of changes using a SOAP or xml-rpc call.
2. Jabber (XMPP) channels between DMB servers to carry notifications and content.
3. UDP notification with http callback. UDP is lightweight for both senders and receivers. No open connections are required between senders and receivers. It’s sort of like RSS cloud, but narrowly and specifically designed for DMB, as opposed to generalized RSS.

RSS cloud API

The cloud API was specifically designed with this purpose of notifying readers of content updates. Its original intent, judging from the RSS 2.0 spec was to allow actual client feed readers to register with the cloud. In the case of DMB, it would be cooperating servers that would register for notification with each other.

The problem with RSS cloud is overhead. Microblog entries are tiny and frequent compared with blog entries or traditional site updates. To require a follower server to read an entire RSS document to get 140 characters of content, and have this happen every few minutes when the poster updates would be inefficient to say the least. In addition, there is experience with the cloud api that indicates just the HTTP session overhead for notifying many users becomes intolerable, although this was from the perspective of actual clients being notified as opposed to clients’ servers being notified.

Jabber (XMPP)

Jabber is a very tempting candidate for this application, and has been getting quite a bit of discussion in the development community of late. Here’s an example. The advantage to Jabber is that it maintains open sessions between servers, which eliminates the session setup/teardown overhead, and allows for almost instantaneous notification of all “following” parties.

But this may also be a disadvantage in situations where there are hundreds or thousands of “followers” for a single sender. IM or chat is typically one-to-one, or one to a few, but microblogging is frequently one to hundreds or one to thousands. I am not familiar enough with Jabber servers in actual practice to know what their performance or connection limitations are. Anyone with Jabber implementation or operational experience is strongly encouraged to comment.

Jabber’s concept of presence could be used to keep the number of messages by only requesting messages updates when a user is actually logged into his/her microblog system. What’s interesting about this notion is what “logged in” means. Microblogging, at least the way Twitter works, does not really require the concept of presence. For instance, you can be “logged in” to Twitter, but dormant for hours not getting any updates until you request a page refresh.

In fact, one key aspect of microblogs that differentiates them from IM or chat is that they don’t typically “auto-update”. And rather than this being a disadvantage, I-and I think many others-find this on-demand update to be much more useful than a streaming IM or chat window. It’s really more like reading small blog posts. I read when I want to, not when someone else decides to say something. So, using the presence capability without auto-updating will require a little clever UI design to, in a sense, auto-logoff the user when their web page hasn’t been refreshed in a certain period of time. This then, will also require the ability of followers to query the senders they follow for “back posts”, so they can see what happened in the past without keeping a client logged in all the time to save all the posts.

UDP for notification

In the past year, I did some work on a revamped email protocol I called IMTP that uses small UDP messages to notify receiving servers that the sender has some traffic for them. The receiver then calls back to the sender using TCP to get the message body. This was based on Prof. Daniel Bernstein’s Internet Mail 2000 proposal several years ago. He proposed, quite rightly in my opinion, that mail senders should bear the burden of storing the message contents, not the receivers, and that mail content should only be sent when a receiver actually wants to read it.

The advantage to UDP is that it is very light weight for both the sender and the receiver, not requiring any session overhead or setup/teardown. If used for microblogging, UDP notification messages could take the place of the continuously open TCP sessions that Jabber employs, thus reducing the session resource allocation load on both ends.

Now, of course, the issue with UDP is that it is not guaranteed to be delivered. The IMTP service we built would retry the UDP message at some frequency until the receiver called back to either fetch or reject the message.

Conceptually, using UDP messages with a 140 character payload and a message number would fit a microblogging application very well. Because microblogging has no expectation or requirement of presence or real-time delivery like chat and IM do, a dropped UDP message is not a tragedy. If the sender retried even once or twice per minute, that would be plenty of timeliness for microblogging. Plus, if the UDP messages carry a monotonically increasing message number, the receiver can know if they’re missed a message and simply call back to the sender to get it. The microblogging UI can reconstruct the sequence easily when the missing pieces, if any, finally come through.

A notification system using UDP would seem to minimize the resource requirements on both senders and receivers, and can perform the same kind of message fan-in/fan-out optimization that Jabber could. In other words, a given microblog update body needs to be sent only once to any server, regardless of how many followers there may be on that destination server.

Comments, please!

I am very interested in what others think and have to say about these issues. This problem of efficient and timely-enough notification, it seems to me, is the tough nut to crack for a good solution to microblogging.

By the way, I am going to do a session on Distributed Twitter (or Microblogging) at BarCampBoston3 on May 17, 18. We are going to try to bring in some mobile technology folks to discuss the other really interesting issue with Distributed Twitter, the SMS connection.

Also, at BarCampBoston, I am going to try to organize some kind of group to try implementing a Distributed Microblogging application going forward.

First of all, let’s call it something more generic. I like Distributed MicroBlogging or DMB. The “Distributed” part is really the key. Unlike a centralized, proprietary walled garden system, DMB would be spread out over hundreds or thousands of different servers over the internet.

Just like email or Jabber, anyone could run a DMB server. People would register on a particular server with their OpenID and create or contribute to microblogs that other people could follow, ala Twitter. Note that people are different than microblogs as entities in the architecture. This is somewhat different than Twitter, in which there are only user accounts. This allows a form of the long-sought groups feature, implemented as microblogs that many people can contribute to.

So, people contribute to microblogs that are followed by other people. When someone updates a microblog, anyone on any server that is following (ie subscribing to) that microblog will get the update in whatever client they have running when the client fetches it. A Twitter-like client will display the posts from many different users interleaved in chronological order. Some clients could maintain a “live” real-time update, where other clients could display only on demand from the user, like the Twitter.com home page.

That’s another important point. The DMB architecture, like Jabber and SMTP, does not specify any particular user interface. How the microblogs are presented is up to a UI designer. The architecture only specifies what data are interchanged, not how it is presented.

So, that’s a very simple explanation of how I see a Distributed MicroBlogging working. There could be large public servers like Google or small private servers for individual companies or groups of people. A server could host hundreds or thousands of microblogs and users, or just one microblog with a single user.
I can envision a given individual’s domain delegating its microblogging functions to a larger server, much as an individual’s home site can delegate its OpenID functions to a large identity service company.

Next, I’ll talk about the single most challenging implementation problem for DMB – notification. How does a DMB server notify other following servers that a change has taken place on one of its microblogs?

So, what could be done about this problem? The answer would be to design a service that is distributed instead of centralized. A distributed Twitter service would operate like email. There’s no single point of failure for email because there’s no single super email server or service through which everything flows. Email is based on a protocol, not a single service, or even group of services. Anyone can run an email server and exchange mail with anyone else running a server. I think sometimes people would like email to go down for a few days because they feel overwhelmed by it, but that’s a different blog post entirely.

Of course, you might say it all flows through the Internet, but the net itself is distributed. Yes, there are backbone subnets that carry huge amounts of traffic and would degrade your service if they went down (and they sometimes do), but there are well-known ways to re-route traffic around failed links.

Twitter could most definitely be distributed. Over the past few months there have been some real developments toward this end, and conversation in the development community about ways to do it. The single most complete development done so far is Prologue, a WordPress theme that allows several people to “co-author” blog posts and share their “update stream” via RSS. It’s been most talked about as a Twitter for corporate co-workers that in a certain sense gives the long-desired groups capability to Twitter.

Prologue is a step in the right direction and hints at what is possible, but is not a general solution that would scale to accommodate thousands of users. The guys at WordPress that developed Prologue have said they’re not particularly interested in developing a fully distributed Twitter-clone, but in the hopes that someone else might pick up the ball and advance it, they’ve made the code open and available under an open-source license.

I’ve been thinking a lot about this problem myself and in this introductory post I’d like to outline what I think are the major requirements and architectural issues to be solved in creating a fully distributed microblogging platform. Do you have any thoughts to offer on this? Please comment!

1. Open
   Although it’s certainly possible to design a closed microblogging architecture (ie, distributed but a proprietary implementation), it’s certainly in the general user’s interest to make it open, which would allow for competing implementations. Why break down a walled garden only to create another one?
2. Protocol -defined
   In order to be open and distributed, the architecture must defined by a protocol, not by a server program or database schema, or other implementation artifact.
3. Minimal
   Occam’s Razor should be the guideline. A new microblogging architecture should specify as little as possible in terms of UI, security techniques, etc. OpenID is an excellent example of a standard that leaves many important features (such as authentication technique) to the various implementations.
4. Privacy-capable
   I think there is a rising concern and general wariness of the completely open and public nature of many social media and social networking applications. Users want to be sure they know who sees their posts and control who can see what “friends” they have. So, there should be a general capability to control access to your microblog and its meta-data.
5. Extensible, Forward-Compatible
   It should be possible for an implementation to add new capabilities without rendering previous versions inoperative or non-interoperable.
6. Scalable
   The architecture should be scalable to the entire internet. A given user should be able to subscribe to or service subscriptions from tens of thousands of other users.
7. Efficient
   In order to fulfill #6, the architecture must be efficient of network and computing resources. In particular, this would argue against polling RSS feeds as a way to collect input. Polling a blog once a day works, but a microblog stream like Twitter needs to react almost immediately.
8. Standards-based
   Whenever possible, new open architectures should utilize existing standardized protocols and data formats whenever possible. This does not mean that a new protocol or format is out of the question, but there had better be a very good reason why an existing standard cannot be used.
9. Open-source
   While proprietary implementations of a standard cannot be prevented, open source implementations should be favored, especially with respect to security issues. Only by inspecting the source can one be assured there are no easter-egg back doors or other weaknesses.