Tipa.li – a Tool for Easily Finding Polio Vaccination Clinics

No one is sure what caused the Polio virus, eradicated from Israel since the1990s, to reappear. The virus was detected in sewage samples in the south of the country in early summer and began spreading northwards, prompting Ministry of Health to start a massive vaccination drive.

Parents of children under the age of 9 were asked to bring them to the nearest Tipat Halav clinic for vaccination. Country-run Tipat Halav (in Hebrew, “drop of milk”) childcare clinics are a household name in Israel. Spread throughout the country, they helped it reach some of the world’s lowest infant mortality rates.

The Ministry of Health decided to create a mobile application to help parents find the nearest clinic. It commissioned one of the country’s largest development shops to create it. They decided to create a native app. This is the Android version:

It’s in Hebrew, but you get the idea

I assume the iOS version is still undergoing the App Store approval process.

We discussed the app during lunch at Gizra. We’re no doctors, but we thought we could do something about the software. Going with a native app didn’t seem ideal for a single-use application which needs to be deployed on as many platforms ASAP. As for the UI, I’ll leave the image above as an exercise to the reader.

Lunch concluded with a particularly good Malabi. It’s one of the few deserts I ever bother with, so I’m pretty sure the extra sugar is the reason I shot off a message to the Public Knowledge Workshop mailing list as I got back to my laptop, asking who’s with me – help me scrape the clinic data and I’ll take care of the front end.

Malabi, by naamanus. This is good.

Within a few hours, a person I’ve never met sent me a link to a JSON file with all the clinics. I still have no idea who this is and how s/he had this file. I geo-encoded a few, put the data on map using GitHub’s geoJSON support, and sent a link back to the mailing list. At 1am that night, Alon Nisser (whom I’ve also never met before) sent some patches that fixed the major missing parts in my code.

By morning we had a working prototype. Meanwhile, Udi Oron and Erez Segall – yet another two people I’ve never met – announced they were working on a more robust scraping code to get all the data from the MoH’s website and encode it reliably.

With the data in Alon, Udi and Erez’s capable hands, I focused on the front end with the goal of keeping it server-free – only HTML/JS code – thus allowing us to develop a quick, simple solution that’s easy to deploy and scale.

The final app is extremely simple. When opening tipa.li (Hebrew wordplay meaning “My drop” or “Tiny drop”) the user is presented with a map, zoomed to the city level and centered on her current location, showing nearby stations. Touching a marker (design donated by Ilan Dray, who I’ve also yet to meet) reveals its street address, opening hours and a phone number.

Tipa.li UI. This is all of it.

That’s all. No search or distance filtering features to clutter the UI. The user knows better – a more distant station might have better parking, for example. All the stations in the country are the one geoJSON file, so users can find clinics in other locations by just zooming and panning around.

The code is as simple as the UI. The app is one geoJSON file rendered on a MapBox map with the excellent Leaflet.js API. No searching, no AJAX calls to a backend server. Everything is client-side, served from the ultra scalable (and only occasionally down) GitHub Pages.

I’m pretty happy with tipa.li. It’s one of these rare cases where things just work right, from the start – the code, the UI, the development process. The media in Israel liked it too, giving it some nice coverage on Haaretz, Calcalist and some national radio/TV shows, helping us further the cause of opening government data. As for users – within 12 hours of posting the site on Facebook, we’ve passed MapBox’s free quota of 3000 requests. It was a good week.

Advertisement

Around Here, the New GeoCities

I know I’m not the only one who needs a map in Jerusalem:

Problem is, a simple map is not enough here. Some places just really accentuate a dimension. Australia taught us to respect distance, driving our camper all day to cover a mere wrinkle on the map. Manhattan forces you to acknowledge it is 3 dimensional. Jerusalem is impossible to understand if you ignore the dimension of time.

So I made a little app I call “Around Here” which shows wiki articles around my current location:

It’s not a lot of code, but actually pretty useful. I mostly use it on my phone, but it works on laptop, tablet and would work right here too if WordPress would allow embedding JS.

Building it made me think of how much the web has changed in the last couple of years. Have you noticed how much you can do with JavaScript now? All the awesome libraries coming out? Not even node.js, just old fashioned client-side JS.

Programming languages are like musical genres. New ones are created, existing ones change and move in and out of fashion, but every once in a while a specific genre will experiences a sudden explosion of creativity. If JavaScript is Punk rock I guess this is the mid-1970s now.

HTML5 and modern browsers’ capabilities are a major factor, but also some slower moving, deep server side changes:

1. Continued APIzation of everything  allowed me to write the app using the WikiLocation API instead of my own DB backend. An embeddable service like Disqus enables pure HTML Jekyll blogs to have active comments. If your data is not sensitive you could even use a Google Spreadsheet as your DB.
2. Decreasing costs of serving HTML/JS/CSS only pages now allow GitHub to let you serve your pages for free, with no restriction on content and almost no scaling limit. Basically you could do that in 1995 with GeoCities – look how much more you can do with just the above client side technologies now.

So, what we used to call “static pages” are now becoming increasingly powerful, and free. There’s an odd back to basics feel as we’re crossing a gap between very low cost to zero cost for serving a useful, highly scalable app. It might turn out a big deal. More importantly, coding is fun again.

KalSMS

KalSMS is a small Android based SMS gateway I’ve released as open source. “Kal” is a Hebrew word meaning “lightweight” and “easy”, so it fits KalSMS’ primary goal of being very easy to install, maintain and work with.

KalSMS is a very thin layer between SMS to HTTP and back to SMS. For example, a simple SMS weather service works like this:

1. A user sends an SMS “weather 10026” to a phone running KalSMS
2. KalSMS intercepts the SMS message, and sends it to pre defined URL. In this case http://qkhack.appspot.com/weather?msg=weather+10026
3. In this URL, a script uses Yahoo!Weather API to get the weather forecast for zip code 10025, and formats an XML response like:

   <reply>
     <sms-to-sender>
       Fair 37F today, Mostly Clear 34F tomorrow, Partly Cloudy 37F Friday
     </sms-to-sender>
   </reply>
   

4. KalSMS parses the response, and sends an SMS back to the user with the weather forecast.

In effect the phone running KalSMS has now become an SMS server, running a weather application.

I’ve been working recently on applications that are meant to be used in places like India and Africa. This taught me that (a) SMS is accessible by a LOT more people worldwide than the web, and (b) it is MUCH harder to build an SMS app than a web app.

KalSMS tries to help this by leaving the heavy lifting to a web app, and just providing a simple, as thin as possible layer between SMS to the Web and back again. This means developers can work with the tools they already know, use state of art technologies like Heroku or App Engine that make launching a web app extremely simple and cheap, and just use an Android phone to enable the SMS part.

Current solutions require either collaboration from a local cellular provider, often a challenge for low budget projects, or setting up your own server – that is, an actual computer running the SMS gateway software connected to a cellular modem. This is non-trivial to install and maintain, and since the solutions out there are tightly coupled you have to write your actual application in a specific language or framework dictated by the SMS solution.

By being Android-based, KalSMS installation is a matter of scanning a barcode, maintenance means keeping the phone working – something almost all people in the world now know how to do. Basically an Android phone with KalSMS replaces a computer, network connection, cell modem, a UPS system and whole bunch of software.

I’ve got plenty of opinions on aid efforts and their effects over the years, but in keeping with the unwritten “produce or shut up” motto of this blog I’ll just say I think things will start to improve when more people pay attention to Bill Easterly than Bono. In my corner, I hope hackers will use KalSMS for various projects simply for its simplicity and reliability – and perhaps in time will seep into the developing world projects I had in mind when building it.

How To Disable Google Buzz (er, #GoogleBuzz )

[UPDATE: I didn’t like the original text, so I cut out most of it. If you’re getting this via a feed reader and this post appears twice, sorry.]

Here’s a short guide to the confused user: How To Disable Google Buzz. It’s not in sarcastic font but if you’re reading this you probably don’t need it. (BTW, I think a good indicator for when a community gets too big is that sarcasm needs to be explained: it means members don’t trust each other’s intelligence enough to assume outlandish claims are meant sarcastically).

My point obviously isn’t that Google Buzz should or shouldn’t be disabled, my point is that most of these things take only as much of your life as you allow them, so all the hyperbole around Google Buzz (or iPad, or whatever) is pointless.

It’s my first time trying out AdSense too – please feel free to click on the ads, I’m sure these are all fantastic products and services.

While we’re at it, I’d like to suggest a one reason why Google botched Buzz’s launch like that: when you launch something on top of Gmail, you just don’t get the chance to grow organically, learning from mistakes and user feedback as you go. There are a lot of benefits to size, but also some drawbacks.

Finally, I really enjoy my new habit of adding pictures to posts. I hope readers like it too. Here’s a photo of a shiny object:

Crowds Machine, Day 1

I’ve been working on Crowds Machine for a while, starting this an idea long ago and running it for my personal use for about three years now, but only recently I took some time to make it publicly usable – the first users I don’t personally know showed up only in the last 24 hours, following its mention in Waxy.org Links.

Here’s what this first public day looks like in traffic:

Keeping in mind it’s Christmas, and the site was down for a while till I fixed up the server, I’m pretty happy with that. But the real surprise is here:

There were less than 10 crowds in the system, so this means 1 in every 4 new visitors created a crowd. That’s pretty incredible. Maybe the holiday timing helped here. Maybe allowing users to log in with their Google accounts. Having to process 12,274 feeds a day might prove a bit of a challenge though…

I always assumed there’s a high entry barrier for CM users since they have to load in quite a few feeds to make it useful. Visitors so far seem to have no issues with that, most just importing their feeds list via OPML. Obviously these are not the typical Web users, but that’s ok. Crowds Machine isn’t meant to be mainstream – that’s the whole point of it.

HelloWorldChat In Stanford Peace Dot Directory

HelloWorld Chat is a project I’ve been working on with my friend and designer extraordinaire Aviva. Basically it’s a “chat online with a stranger” site like Omegle & co, but we try to match between users from different countries (based on their IP address). Once we reach a certain level of traffic, we could start adding more sophisticated matching rules and connect between users of specific nations, based on current world events and so on.

We haven’t done much in way of publicizing the site, so it was great to be accepted into Stanford’s Peace Dot directory. Peace Dot only adds one organization per day to the directory, and we’re excited to finally get in :) They even do a little welcome video for each site, here’s Standord’s BJ Fogg welcoming us:

Thanks, Peace Dot!

Technically, HelloWorld is a simple Google App Engine based chat, written in Python. The focus was less on the chat code and more on making the application easy to customize and adapt for various sites. The look can be completely changed by editing two HTML files, and being App Engine based makes deployment extremely simple and cheap.

Sites built on top of HelloWorld might be used to foster dialog between sides in a conflict, aid in reconciliation phases, or help people going through their own struggle with a disease or addiction talk with others in the same situation. The anonymity allows users to speak frankly where it may often be difficult or dangerous to do so, and hopefully some may later move on to talk via Skype or Email – our goal is to be the easily accessed and risk free gateway to a dialog.

While forums and multi-user chats have important roles, a one to one conversation has a unique value of its own. Talking one to one makes slogans and cliches a lot harder to hide behind. It is a lesson I learned myself in the summer of 2006, when I happened into an IM conversation with a Lebanese person while both our parents’ towns were under fire by the other side. That experience was the inspiration for HelloWorld, I hope this tool would allow many more such conversations to take place.

Fall in New York

Generally beautiful, New York City is even beautiful-er when autumn foliage comes around. So it was cool to discover this Flickr set of scanned leaves from Morningside Park – literally 3 blocks from where we live.

(Via what about the plastic animals, via Outside.in)

The Long Poll: AJAX Push(like) Chat with Comet

Recently I’ve been working on an AJAX based chat application (in development..). The obvious way to do it is send an XMLHttpRequest every few seconds to check for new messages. Unless it’s a particularly animated conversation most requests won’t return any new content, so I added a simple Conditional-GET like system based on the chat’s text size. Here’s the client side implementation:

function refresh_chat() {
	$.ajax({
	  	url: "/chat",
	   	data: "format=xhr&chat_id={{chat_id}}&cur_len=" + chat_content.length,
		  complete: function(xhr){					
				if (xhr.status == 200) render_chat(xhr.responseText);
				setTimeout("refresh_chat()", 5000)
		  }
	 });	
}

And the server code that handles it:

cur_len = self.request.get("cur_len", 0)
if len(chat.content) == int(cur_len):
	self.error(304) # return 304 Not Modified
else:
	self.response.out.write(chat.content) # return new content

That’s basically the standard approach. Pretty simple, works ok (could be optimized a bit, for example return only the actual new content etc). It’s not exactly an elegant design, though. Trying to use HTTP, designed as a Pull protocol, for an application that requires Push results creates this system of frequent server requests with empty responses, kind of like the “Are we there yet?” conversations with kids on long road trips.

Jack Moffitt’s JSConf talk introduced me to the concept of Long Polling, aka Comet or (with a lot added) BOSH, as a way to simulate HTTP Push. Rather than have the client sending a lot of short, frequent requests and the server responding to each as fast as possible, long polling turns it around: the server holds the requests as long as it can, returning a response only when it has new data or a timeout limit was hit. So, instead of sending request every 3 seconds, for example, you can send one every 30 seconds.

Client side code remains almost the same:

function refresh_chat() {
	$.ajax({
	  	url: "/chat",
	   	data: "format=xhr&chat_id={{chat_id}}&cur_len=" + chat_content.length,
		  complete: function(xhr){					
				if (xhr.status == 200) render_chat(xhr.responseText);
				setTimeout("refresh_chat()", 1000);
		  }
	 });
}

But on the server side, there’s a bit of new logic to keep checking for new content while the server holds the response:

cur_len = self.request.get(“cur_len”, 0)
end_by = int(time.time()) + 30

while int(time.time()) < end_by: if len(chat.content) != int(cur_len): return self.response.out.write(chat.content) # return new content time.sleep(1) self.error(304) # return 304 Not Modified [/sourcecode] If you have any experience building web applications, you've spent a lot of effort making sure servers respond quickly to requests. Delaying the response is counter-intuitive, which in itself makes Comet useful to know, if only for its new perspective. However, this also makes production use a bit complicated, since most web server stacks are optimized for maximum requests/second rather than long concurrent requests. Content-rich sites often use separate servers for big media content for this reason, and Comet also has its own server (er "HTTP-based event routing bus") in Cometd.

Kunstler: The Tragedy of Suburbia

WordPress just added special TED support – I’m not a fan of TED’s exclusive/trendy approach, but it’s a good opportunity to post this excellent talk by James Howard Kunstler.

The lesson of design which considers its users, environment and resources vs. design that is isolated, contrived and wasteful holds true to software as well, from the UI to the server.

The Real URL

[UPDATED on April 21st, 2009 to reflect the JSON/P additions. Since it’s <24 hours after the initial release, I hope it won’t cause anyone problems.]

The Real URL began as a joke – after discovering, while working on another project, over 80 URL shortening services, I figured there must be room for a service that un-shortens all these URLs. (The web is overflowing with hype and blog posts/articles complaining about it just add to the noise, so it’s better to make your point by building something. My favorite example is the Twittering Office Chair).

Turns out there are already several out there: (eg, trueurl) but I built it anyway, since I had a slightly different approach in mind. The Real URL is meant to be used as a web service rather than on its own. It returns the “real” URL in either raw text, JSON or JSONP format – examples and details are on the homepage. (I added JSON mostly for JSONP, per Chris’ comment – admittedly I didn’t even know it existed ;) This enables cross site JS requests which might actually make The Real URL useful.

While I do want The Real URL to be solid and reliable in the long term, I don’t want to spend much time/money keeping it up. It’s a sustainability issue – building a system that will work reliably over a long time while requiring minimal care and resources. I made a few design decisions to that end:

• Keep it simple (always a good idea). Real URL does only one thing and is accessible in only one way (the homepage demo uses XHR to access the service, to keep it so). It now supports text/JSON/JSONP, but it’s just the same output formatted differently. Sometimes you give up some elegance to make the product useful. As in the following item:
• Deploy with Google’s App Engine. Initially it was nice, super-minimal Sinatra code. Unfortunately Google App Engine doesn’t support Ruby yet and there’s no service that offers comparable cost/stability ratio, so I rewrote in slightly less minimal Python for GAE.
• Use App Engine’s domain (therealurl.appspot.com). Buying a domain and keeping it renewed isn’t a big deal, but it still requires some attention – especially if you happen to hit a nice domain name which people try to grab or piggyback on. Sticking with appspot.domain minimizes this issue. (if the need rises I might add a “real” domain later on, but in any case therealurl.appspot.com will remain active)

If you find a use for The Real URL this or have an idea for one, please comment here or email me at niryariv@gmail.com. Let the street find its own uses etc ;)