Backend changes...

This evening, we are rolling out a new Lingr revision that adds some great new features including a spiffy live Lingr badge that you can embed on your website. Not so obvious was the fact that we also made some significant changes to our backend architecture.

We're committed to being open about our technology, particularly our scaling experiences with Comet. There isn't a lot of empirical data out there on scaling a Comet site (whatever data there is must be locked up inside the few companies that have done it), and we hope to change that at least a bit. We've been helped so much by the openness of others in their experience scaling Rails apps, that we want to try to give back as much as we can.  So, in that spirit, here's what we did-

In a previous post, I mentioned that we use a custom servlet running under Jetty 6 to provide the long-lived comet connections that make Lingr unique- we call these Jetty 6-based servers the chatservers. What we have done in this latest release is pared back the functionality of the chatservers to the bare minimum.

In prior releases, all of the chat activity between your browser and our service was routed through the chatservers- that is, whenever you entered a room, sent a message, changed your handle, or opened a connection to wait for events (observe), those messages were sent to the chatservers. The fact was, however, that, in all cases except observe, the chatserver simply proxied those request over to the web servers.

What we have done in this new release is to remove the middleman in the cases where we can. Now, when you enter the room, send a message, change your handle, etc., those requests go directly to the web servers via a standard XMLHttpRequest call. Only the observe call talks to the chatservers (observe still uses dynamic <script> tags).

We have also made a significant change in the way that observers get notified when someone in the room says something. In the past, since all calls went through the chatservers, they could handle the notifications themselves. Now, however, when you say something, that call goes to the web servers, so, we needed a notification mechanism from the web servers to the chatservers. We implemented this using a very simple HTTP request from whatever web server handled the say verb out to the entire chatserver farm. Currently these notifications happen synchronously, but I suspect that we may soon take them out of the HTTP request cycle and handle them in something like a custom BackgrounDRb task. As our number of chat servers grow, the viability of a synchronous notification mechanism will become less and less.

Overall, these changes should increase the stability and capacity of Lingr as more and more people are using the site.

- Danny Burkes

Comments

[Duncan Beevers]

This is a really exciting development and the idea of moving the chatserver notifications to a background task seems like the right way to go. You might be interested in Lucas Carlson's distributed computing utility Starfish
http://rufy.com/starfish/doc/

Good luck and thanks for the great service!

[Vishwa Malhotra]

Question on Observe/Back-Channel:

How do you work-around the same-origin policy of the web browser?

[dburkes]

Calls to both our Rails side and our Jetty side use the same hostname (www.lingr.com), so, the same origin policy is not a problem. On the backend, our load balancer routes the connection to the appropriate cluster based on the requested URL (layer 7 load balancing).

For people using our API (who don't live inside www.lingr.com), they are using dynamic script tags to initiate their observes, and the same origin policy does not apply to script tags.