plaintext

I recently found that Jon Rogers, a developer from the UK, released the UnShorten WordPress plugin, which uses TheRealURL to unshorten links displayed by the Twitter Tools plugin.

That’s pretty cool, TheRealURL was designed as a web service with exactly this type of use in mind.

Between this and other users TheRealURL now serves over 40,000 requests a day. It’s nice to see Google App Engine handling that with barely using any of the various daily quotas (except for incoming bandwidth… Need to check on that one):

I needed this for a project I’m working on, so I added a new feature to TheRealURL unshortening service: JSON/P requests now return the page title (scraped from the HTML <title> tag) as well as its original URL.

For example, http://therealurl.appspot.com/?format=json&url=bit.ly/a returns:

{ "url" : "http://www.apple.com/", "title" : "Apple" }

The plain text format remains as is – nothing but the unshortened URL – so I don’t think there should be any issues for existing API users. Response times don’t appear to be affected much either. If you do get any issues, please let me know in the comments or at niryariv@gmail.com.

I just put up a quick hack I made with FeedVolley (more about FV here), that aggregates Twitter (and other media) feeds coming from inside Iran: Messages From Iran

I don’t know about news value, but it’s pretty cool to be able to refresh that page now and then and get a snapshot of the current mood and happenings, in these possibly historic times there.

It was also cool to find another use for FeedVolley, which I neglected a bit recently ;) I added some page caching on top of the existing feed caching, to allow it to handle some traffic (Slicehost’s 256MB slices seem to start sending swap alerts as soon as traffic rises above negligible). The sources are basically the ones listed here, with a few additional ones I’m trying to find. In fact, if you really want to keep a close watch on what’s going on, you may want to watch the FriendFeed stream – the FeedVolley page is really just an HTML skin to make the feed look a little nicer (hopefully).

(Favorite tweet so far: “@jonobacon IRC is blocked. Tell our regards to Ubuntu Global Jam from Iran. I’m twitting the #iranElection story from a Kubuntu machine :)“. Makes me think of starting to use Twitter again..)

rmmbr is a service I built for myself and recently opened for general use. I wanted a way to send myself notes about things I need to do, items I want to research etc. If just write this on a scrap of paper I usually seem to forget about it, so using Textmarks‘ web-SMS gateway (which is pretty nice btw – reliable and easy to interface with) I wrote a little web app that lets me text these notes to my inbox.

The examples on rmmbr’s home page are pretty much how I use it – I come across some term I want to research or am reminded of something I need to do and send an text message like rmmbr <whatever> to 41411. When I get back home an email is waiting in my Gmail inbox with the text sent, a Google search link for it, and if it’s an address or a date Gmail might also attach its “map this” or “add to calendar” links as well.

To start using it, you need to let rmmbr connect your phone # with your email address by texting rmmbr signup <your@email> to 41411. You’ll get a confirmation email and can start sending yourself messages. Just remember to always start the message with “rmmbr” – it still gets me sometimes..

I don’t know how useful it would be for anyone with an iPhone or a Blackberry, but if you’re like me and prefer to keep your cellphones small, cheap and low maintenance, you might find this a useful solution for your text-to-inbox needs. Enjoy :)

[Update: formerly rmbr, the app/sms code is now rmmbr, to prevent confusion with rmbr.com - which is a different product, not related to rmmbr in any way]

[UPDATED on April 21st, 2009 to reflect the JSON/P additions. Since it's <24 hours after the initial release, I hope it won't cause anyone problems.]

The Real URL began as a joke – after discovering, while working on another project, over 80 URL shortening services, I figured there must be room for a service that un-shortens all these URLs. (The web is overflowing with hype and blog posts/articles complaining about it just add to the noise, so it’s better to make your point by building something. My favorite example is the Twittering Office Chair).

Turns out there are already several out there: (eg, trueurl) but I built it anyway, since I had a slightly different approach in mind. The Real URL is meant to be used as a web service rather than on its own. It returns the “real” URL in either raw text, JSON or JSONP format – examples and details are on the homepage. (I added JSON mostly for JSONP, per Chris’ comment – admittedly I didn’t even know it existed ;) This enables cross site JS requests which might actually make The Real URL useful.

While I do want The Real URL to be solid and reliable in the long term, I don’t want to spend much time/money keeping it up. It’s a sustainability issue – building a system that will work reliably over a long time while requiring minimal care and resources. I made a few design decisions to that end:

• Keep it simple (always a good idea). Real URL does only one thing and is accessible in only one way (the homepage demo uses XHR to access the service, to keep it so). It now supports text/JSON/JSONP, but it’s just the same output formatted differently. Sometimes you give up some elegance to make the product useful. As in the following item:
• Deploy with Google’s App Engine. Initially it was nice, super-minimal Sinatra code. Unfortunately Google App Engine doesn’t support Ruby yet and there’s no service that offers comparable cost/stability ratio, so I rewrote in slightly less minimal Python for GAE.
• Use App Engine’s domain (therealurl.appspot.com). Buying a domain and keeping it renewed isn’t a big deal, but it still requires some attention – especially if you happen to hit a nice domain name which people try to grab or piggyback on. Sticking with appspot.domain minimizes this issue. (if the need rises I might add a “real” domain later on, but in any case therealurl.appspot.com will remain active)

If you find a use for The Real URL this or have an idea for one, please comment here or email me at niryariv@gmail.com. Let the street find its own uses etc ;)

While working on a recent Rails project, I wanted to occasionally backup the database to a remote location. Since we were already using Subversion for source control, I figured I could just use it for storing the DB contents as well and came up with a short Ruby script for this, called dbbackup (ironically, stored on a Git repo – I guess it could use a –use-git option ;)

The only Rails tie is that it uses config/database.yml to get the database name and login info, so you could easily adapt it to run on non-Rails projects too. It’s built so that it can be ran by a nightly cron task, and since it’s only sending the diffs it wouldn’t be too resource-heavy.

I wouldn’t use this for Facebook’s production servers, but if your needs are more moderate you might find it useful. Feel free to send over any questions or patches to niryariv@gmail.com. Here’s the repo URL again, with the script and an explanation on how to use it: http://github.com/niryariv/dbbackup/tree/master

First revenue ever from Prixfeed, my RSS-of-Amazon-prices app, arrived in my email today in the form of an Amazon gift certificate:

While not exactly retiring yet, I’m pretty proud of this little app. Very simply done, no marketing to speak of, barely Googleable, it somehow managed to attract users. Not many – but those who find it do seem to actually use it. It’s the micro web app: a single page contains the whole UI, an API of sorts (returns microformatted RSS) and now even a revenue source :)

I got this as a feature request from Nirmal Patel, and it turned out FV already supports it: You can use FeedVolley’s customizable templates to turn any RSS/Atom feed into a widget – or badge, or plugin, or whatever it’s called in various web apps – which you then embed in your pages.

So, for example, in http://feedvolley.com/badgeit you have a JavaScript document.write() that outputs the latest NY Times headlines, which you then can include in another page with <script src="http://feedvolley.com/badgeit"></script>


That’s a pretty bare-bones example, but obviously you can customize the HTML/JS to present the feed any way you like. Hope some folks would find this useful, and thanks Nirmal for the feature suggestion – it’s always great to see how users figure out uses for an app which you didn’t think of when making it.

I like 37signal’s Design Decisions posts, which explain the thinking behind seemingly small details in their apps. It makes sense: building the core functionality of most web apps is relatively straightforward, the real quality (and, ultimately, most of the effort) is in the details. So, here’s my take on Feedvolley’s design.

I’m not aware of any site that does the quite same thing as Feedvolley, so the first challenge is to get users to understand what it’s about and how to use it. My favorite way to learn to use something is to play around with it (not recommended with firearms, bikes and similar BTW), so the goal was to make Feedvolley’s interface invite users to do just that. That means making it as easy as possible to accomplish something, and then make it rewarding to keep playing with what was created.

To make starting out easy, the homepage is a minimal form with fields for feed or HTML page URL (one of the features that make RSS/Atom a good Web API is the fact it’s often auto-discoverable) and email (more about that in a moment). A default theme is pre-selected.

Another way to create a page is by clicking the “Create a page like this” link located on top of every user-created Feedvolley page. This lets users start with an existing page and modify the content and HTML to their needs. That’s one of my favorite web app buttons – it invites a viewer to become participator, and lets users start with something similar to what they want, and just modify it as they learn the system.

You might have noticed there is no registration step here. Personally, I hate having to register to a website in order to do anything. Feedvolley (like Notifyr) uses email as its user authorization system. Each user gets an edit link with a unique token string. This token is also kept in a cookie, so users aren’t forced to go to their email in order to start editing. “Create a page like this” also serves as backup for this system. In case a user lost the edit link, she can simply go to her page, create a duplicate and continue editing that.

This may not be perfect, but seems optimal for most users. If some users ask for a more rigid authorization system, we can always add it later on as an option.

Once a user created a page, the next step is to make further work on it possible, and worth the time. This is where the “Customize” link comes into play: users can set a page’s URL path and have complete control over the HTML, JavaScript and CSS in the theme. To make themes as easy as possible, Feedvolley’s markup closely follows that of Tumblr (Tumblr’s templates inspired the Feedvolley concept, in fact. They do great work over there). In “work with the existing environment” spirit, this lets users easily adapt existing Tumblr themes for Feedvolley and also use Tubmlr’s docs, which also saved us some documentation pains :)

As far as design goes, I’m with the “it’s done when there’s nothing more to remove” (as opposed to “nothing more to add”) school. So, I’m pretty happy with the result in Feedvolley. Some challenges remain: how to make it more obvious that a page can be customized once it’s created, for example. It’s not perfect yet and we’ve already incorporated some user feedback into it – if you have any comments or requests, please leave them here or email me directly: niryariv@gmail.com.

Feedvolley is a new project I collaborated on with Kyle Bragger. It began when Kyle and I found we each had the same idea at the same time and so we set out to build it – Kyle, who is one of the brightest coders I’ve met and among the rare total developers that master the whole range from visual design to server side, is responsible for everything that looks good in Feedvolley, as well as for the name itself.

Feedvolley basically takes in an RSS feed and displays it in user defined HTML template. You can start with a default theme and can later use another pre-set theme or customize your own HTML/CSS/JavaScript. You can set the path portion of your URL (ie feedvolley.com/) and have people view it a regular web page. People who like it can then click “Create a page like this” and start off their own pages, modifying the look or content to their needs.

Consuming an RSS feed and putting out HTML may seem a bit counter-intuitive, but I think it can have some interesting uses. One is to complement feed tools like Yahoo!’s Pipes that let you mix together and modify different feeds. For example, you could use Yahoo! Pipes to combine several job site feeds, filter only those which contain a certain keyword and present the resulting feed as an HTML page. Or you could take the output of a any feed aggregator and display it River of News style, perhaps with some JavaScript effects etc., or you might use a scraping tool like Dapper to extract certain parts of a page’s contents into an RSS feed, and display it with your own look.

Another use is to skin existing websites. You can take content from any website with an RSS feed and use Feedvolley to present it with a different HTML skin. For example, here’s the NY Times’ homepage in a different layout. There’s a nice subversive potential here, and having complete control over your HTML/JS/CSS allows for a lot of creativity. My own front-end coding skills somewhat lacking, I still managed to do a mildly interesting project like NYMinute using META Refresh tag and Flickr’s ‘photos tagged nyc’ feed.

Hopefully, users will put Feedvolley to other, unforeseen uses. I already found an unexpected one in the site itself, to display the Recent Pages feed. When I demoed it to a friend, he instantly set up a digest page of updates to his company’s Wiki site. You can get something up very quickly with just a URL and your email address, and later on you can customize it to do anything that can be done with HTML and JavaScript.

Feedvolley continues on themes from projects like Notifyr and Prixfeed. First, RSS as the web API – I think RSS goes a long way towards a common API for websites. It is read only and lacks some features, but most sites only need a read-only API, detailed information can added via domain-specific tags or microformats and, most importantly, it’s already in wide use. I think there’s a lot more to do with the RSS content already being out there in addition to the current Feed Reader uses.

Second, the idea of content modifiers (or participators, user/developers etc). Everything Feedvolley does can be accomplished with some code, but removing the setup and maintainence overhead and the need to write code opens this kind of application to a much bigger potential crowd (this is like Notifyr’s URLs that allow web literate – but not necessarily technical – users to construct a link lets people subscribe to their Flickr page). UI elements like minimal setup dialog, no registration requirement and support for copying existing pages are there to make Feedvolley something that you can play around with, tweaking it and getting instant feedback.

Really, you can :) Just follow this: Feedvolley