The Real URL
[UPDATED on April 21st, 2009 to reflect the JSON/P additions. Since it's <24 hours after the initial release, I hope it won't cause anyone problems.]
The Real URL began as a joke – after discovering, while working on another project, over 80 URL shortening services, I figured there must be room for a service that un-shortens all these URLs. (The web is overflowing with hype and blog posts/articles complaining about it just add to the noise, so it’s better to make your point by building something. My favorite example is the Twittering Office Chair).
Turns out there are already several out there: (eg, trueurl) but I built it anyway, since I had a slightly different approach in mind. The Real URL is meant to be used as a web service rather than on its own. It returns the “real” URL in either raw text, JSON or JSONP format – examples and details are on the homepage. (I added JSON mostly for JSONP, per Chris’ comment – admittedly I didn’t even know it existed ;) This enables cross site JS requests which might actually make The Real URL useful.
While I do want The Real URL to be solid and reliable in the long term, I don’t want to spend much time/money keeping it up. It’s a sustainability issue – building a system that will work reliably over a long time while requiring minimal care and resources. I made a few design decisions to that end:
- Keep it simple (always a good idea). Real URL does only one thing and is accessible in only one way (the homepage demo uses XHR to access the service, to keep it so). It now supports text/JSON/JSONP, but it’s just the same output formatted differently. Sometimes you give up some elegance to make the product useful. As in the following item:
- Deploy with Google’s App Engine. Initially it was nice, super-minimal Sinatra code. Unfortunately Google App Engine doesn’t support Ruby yet and there’s no service that offers comparable cost/stability ratio, so I rewrote in slightly less minimal Python for GAE.
- Use App Engine’s domain (therealurl.appspot.com). Buying a domain and keeping it renewed isn’t a big deal, but it still requires some attention – especially if you happen to hit a nice domain name which people try to grab or piggyback on. Sticking with appspot.domain minimizes this issue. (if the need rises I might add a “real” domain later on, but in any case therealurl.appspot.com will remain active)
If you find a use for The Real URL this or have an idea for one, please comment here or email me at niryariv@gmail.com. Let the street find its own uses etc ;)
Neat – one recommendation: maybe implement JSON with JSONP to avoid the cross domain JS problem?
Chris Dary
April 20, 2009 at 4:40 pm
@Chris – thanks! I added these now.
(Initially I chose not to support JSON in order to keep it minimal, but JSONP changes the whole picture. Now that it’s possible to use The Real URL remotely from JS, it might actually prove somewhat useful :)
Nir
April 21, 2009 at 11:29 am
[...] UPDATE #2: As a result of this post I ended up building a URL unshortening service, which I now think might actually have some uses. More here. [...]
List of URL Shortening Sites « plaintext
April 23, 2009 at 2:57 pm
[...] a comment » I discovered JSONP just recently, following Chris’ comment. Though I initially didn’t intend to support JSON, JSONP made enough difference that I [...]
JSONP, Quickly « plaintext
May 5, 2009 at 2:25 pm
Can you add an
Access-Control-Allow-Origin: *header to the API so it is usable via JavaScript?Elijah Grey
May 20, 2009 at 8:51 pm
(sorry for double-reply) ..and by usable, I mean safe to use without having to trust a third party.
Elijah Grey
May 20, 2009 at 8:54 pm
Sure thing – added.
Nir
May 21, 2009 at 2:49 pm
[...] pretty cool, TheRealURL was designed as a web service with exactly this type of use in [...]
UnShorten Wordpress Plugin Uses TheRealURL « plaintext
November 10, 2009 at 3:10 pm
Nice stuff, been looking out for a good unshortener.
One small request could you also return the content-type, content-length headers of the unshortened in the response json? That would be damn cool.
Jebu Ittiachen
November 22, 2009 at 7:47 pm
Interesting, I can look into it… What would you use it for?
Nir
November 22, 2009 at 10:44 pm
Thinking stuff like preview images, iconify based on content type, heads up on large files etc.
Jebu Ittiachen
November 23, 2009 at 7:30 pm
Added :)
http://therealurl.appspot.com/?format=json&url=bit.ly/a
Nir
November 24, 2009 at 1:44 pm
Wonderful!! Thank you.
Jebu Ittiachen
November 24, 2009 at 7:37 pm
Hi. Could I recommend adding an API call that lists all the shortening services you handle? That way a program can know which URLs to send your way and which to skip.
Examples from similar services:
http://longurl.org/api#list-known-services
http://untiny.me/api/#service=2
Best regards,
E.
ecognito
January 29, 2010 at 7:02 pm
Hi E.,
Basically every URL shortener that uses HTTP redirect (and that would be pretty much all of them) is supported. That should include all the sites listed in both of these URLs, and many more – including sites that aren’t necessarily URL shorteners but do URL redirects.
Nir
January 29, 2010 at 11:53 pm
Hello,
Quick question:
Do I understand it correct that the requests to unshorten the urls go through Google? Translating into Google adds the request (and all info about the requester) to its collection of user data?
Anyway to avoid this?
Thank you
Kate
February 7, 2010 at 11:05 pm
Hi Kate,
The only connection to Google is the fact the app runs on their App Engine. To the best of my knowledge Google doesn’t collect data from requests to TheRealURL and even if they wanted to, I don’t see how they could considering that requests don’t contain any cookie information that could attach them to a particular user.
In short, I think there’s very low chance of Google collecting TheRealURL request data. The app itself doesn’t collect user data at all.
Nir
February 8, 2010 at 8:48 am
Awesome!
I’m going to migrate my chrome url expanding extension to use your service. The service I am using is very slow from time to time so this looks like a great improvement!
Don
May 1, 2010 at 2:43 pm
Awesome! Would love to see this, esp. since I find myself using Chrome more and more :)
If there’s anything I can help with just let me know.
Nir
May 1, 2010 at 3:34 pm
I have changed my extension to use your service. I’d love to get some feedback on it. You can find it on here: http://tacticalcoder.com/blog/2010/05/chrome-url-expander-1-5-1-is-now-live/
Don
May 1, 2010 at 5:11 pm
Yup, installed it, works great.
Two ideas for possible improvements, if you’re interested:
1. TheRealURL also provides you some page info (eg, page title) if you want to display that somewhere (perhaps on the status bar?)
2. I notice some of the short URLs (eg 4sq.com) don’t get expanded. I assume you’re checking against an internal list of shortening services? That’s a tricky issue, since on one hand you don’t want to hit the server for every URL and on the other the number of shortener services keeps growing. One solution I was thinking of is running a regex on every URL to determine if it’s likely to be a shortened one: for example, URLs that follow a format of “http://x.y/z” (ie, only 1 dot in the domain part, no slashes in the path) where x and z are smaller than, say, 8 characters.
Nir
May 1, 2010 at 5:33 pm
That’s a great idea. I’ll add those changes to the next version.
Don
May 2, 2010 at 1:18 pm
I have been using TheRealURL Add-on for Firefox and it is excellent.
I recently installed the add-on for Thunderbird as well and have discovered a big problem with it. I open messages in their own windows and TheRealURL does NOT give any right click option for TheRealURL within the message window. This is a big frustration in that the extension is worthless for how I use Thunderbird.
To check on this I opened a message “In A New Tab” instead and there TheRealURL extension worked just fine. So the problem seems to be only with messages opened in their own windows as opposed to in new tabs. This doesn’t sound like a major programming problem, and I hope that it can be fixed without too much trouble.
Thanks.
JonG
June 3, 2010 at 1:30 pm
Hi Jon,
Unfortunately at the moment I can’t really support the Thunderbird version – I have little time to work on TheRealURL currently and have very little experience with Thunderbird.
However, the code is open so if you have the skills or can find someone who does please feel free to go ahead and change it (if you do, please share the fix with me so I can add it for everyone)
Thanks,
Nir
niryariv
June 5, 2010 at 12:29 am
Wonderful. Could use support for dlvr.it, which seems to have become popular with some Twitter twits. I’m using it via the Chrome add-on.
mathew
September 24, 2010 at 9:22 am
Also: bit.ly
mathew
October 11, 2010 at 2:31 pm
Hi Mathew, thanks for your feedback!
Both bit.ly and dlvr.it seem to work fine for me – are there any specific URLs you’re seeing problems with?
niryariv
October 11, 2010 at 2:55 pm
The site is overquota on Google Engine. Wouldn’t you like to release the code open source? I need it for a citizen journalism project. Best.
Hector Palacios
November 26, 2010 at 1:39 am
Hey Hector,
Drop me a note at niryariv@gmail.com – I’ll help you set it up.
- Nir
niryariv
November 26, 2010 at 12:41 pm