« April 2007 | Main | June 2007 »
Zack Chandler just posted an interview with Danny over at SFRuby.org: sfruby.org/posts/interview-w-lingr. This one’s much more focused on Rails than the last interview they did.
Also, I’m extremely late mentioning this, but back in the spring Technorati Japan’s BlogTV stopped by our office in San Mateo to interview Kenn:
Kenn tells me it’s about his experiences starting Lingr. All I know is, they didn’t include the bits with me and Danny making funny faces.
- Chris
Since we first released the Lingr API, by far the most frequent request that we've had was to allow embedding a chatroom in a web page. Some of our developers have actually created such things, but they all suffered from one drawback- because our API was fully REST compliant, it wasn't possible to do many interesting things purely from Javascript. The reason for this is that the REST convention is that any operation that changes state on the server should be an HTTP POST operation. That makes perfect sense, however, it kills any possibility of a purely Javascript/JSON implementation, because, while Javascript can certainly perform POST operations, it cannot do so asynchronously, nor can it retrieve the results of a POST (well, while this isn't technically true, it is practically true). The solution that most developers have taken is to proxy the POST calls through their own servers. This works fine, but, it seems like an unnecessary complication.
We were faced with a conundrum- on the one hand, we want to be REST compliant, because we're engineers and every engineer loves a nice, cut-and-dry specification. On the other hand, we want to allow pure Javascript clients to access our full API, because, as we've learned, the most interested Lingr API clients don't originate from the lingr.com domain :-)
So today, I'm happy to announce that we've opened up all of our API methods to HTTP GET operations. Even though the documentation for a particular method may still state that POST is required, in fact, on the backend, we've disabled HTTP method checking for API calls.
In addition, we've published lingr.js, a full read-only Lingr chatroom client in Javascript. While it is missing some obvious functionality (the ability to say something, for example), it is a great starting point to embedding a chatroom in your web pages. You can find documentation on lingr.js over at the Lingr Developer Wiki. We'll continue to add new functionality to lingr.js- this is just the beginning.
If you use lingr.js, please let us know, and add your creation to the Showcase. We're eager to see how people take this ball and run with it.
- Danny
Update: we have added say capability to lingr.js :-)
We have deployed a new release that suppresses system messages in the chatroom. That means no more messages like "john has joined the conversation", "john is now known as john-boy", etc.- we reserve all the chatroom space for user messages now.
Note that system messages will still appear in the archives- we think that they are necessary to understand the context of conversations that occurred in the past.
Let us know what you think about this new feature by sending us feedback.
- Danny
To complement our recent additon of full archive search capability, we have added a new API method to access this search functionality. See the Lingr Developer Wiki for full details.
Enjoy!
- Danny
As promised over at Ruby Forum, I am proud to say that we have released our multilingual Ferret analyzer as open source. It's available now at out public subversion repository, packaged as a Rails plugin. Enjoy and do let us know if you find it useful!
- Danny
We recently added full archive search to Lingr, and I thought I'd take a moment to talk about the technical details of that, for those who are interested.
At Lingr, everything said in the chatrooms is saved into our database. This enables you to browse the archives of a room to recall some recent conversation, or to find out what someone else said about something. So, we've got around three million user utterances sitting in our database, and, we thought, why not unlock those and let people search them?
Our first thought was to just use MySQL's fulltext indexing system. But, as it turns out, fulltext indexing only works on MyISAM tables, and our utterance table is InnoDB, so, that was out the window.
So we started looking for a text indexing system that could work for us. What we found was Ferret, a ruby port of Apache Lucene. Combined with the excellent acts-as-ferret (AAF) plugin for ActiveRecord, we were able to integrate Ferret/AAF into Lingr in about two weeks.
The one issue that complicated matters the most is that Lingr hosts conversations in many different languages (English, Japanese, Farsi, etc.). This presents a unique challenge in terms of tokenizing the utterances before they are indexed. Ferret provides a very nice tokenizer for most languages based on the Latin alphabet, but other languages such as Japanese which do not delimit tokens by whitespace are problematic. Also consider the fact that it is quite common to have a single utterance that mixes languages (I guess you can thank the global ubiquity of English for that). And to put icing on this cake, for a given utterance, we have no idea which language (or languages) it is in- all we have are Unicode codepoints.
So, out of the two weeks required for integration, much of that time was spent writing and tuning our own tokenizer. Our tokenizer basically spots transitions between Latin text and non-Latin text (based on codepoint value), then applies Ferret's existing Latin tokenizer to the Latin parts, and a simple per-character tokenizer to the non-Latin parts. Because Ferret uses the same tokenizer when indexing an utterance as it uses when searching for an utterance, this means that your search terms can contain a mix of Latin and non-Latin "words", and we should handle that just fine.
For the metrics-obsessed among you, our Ferret index currently consists of approximately 3 million "documents" (user utterances). The on-disk size of this index is currently 909 megabytes. The index is updated once per minute, via a cron job, so there is some short period after an utterance is spoken when it is not indexed (maximum one minute).
Finally, I'd like to thank Jens Kraemer and everyone over at the Ferret Forum for their help and advice. If you are thinking about using Ferret, you can get some great information there. It helped us tremendously.
If you have any other questions about the implementation, feel free to ask them here in comments, or through our Feedback form.
Cheers,
Lingr is what you make it: The entire site is full of nothing but your content. Your rooms, your messages, your pictures, your tags, etc. We’ve created the space, and you’ve filled it.
Till now, however, you’ve only had two choices for accessing that data: Read through the archives, day by day, or search each room’s archives using Google. Both methods work, but neither is ideal.
Now you can search the Lingr archives directly. And man is there a lot to search…
All public room archives are searchable at www.lingr.com/search/archives. The results are sorted for you by their relevancy and their recency. We’ve grouped them by room, and by day, so that you can scan through the list faster. You can drill down into each room, and each day within each room, assuming there enough relevant results.
And you can now search every room from within that room’s archives. So if you’re in the archives of a private or password-protected room, you’ll see one of our standard search forms in the sidebar on the right side. Put your query in there and click Search archives. (Of course, you can only search the archives of private and password-protected rooms if you already have access to them. They’re not public.)
Since there’s so much data in the Lingr archives, we’ve got a lot more work to do on optimizing our search algorithms and on how we present the search results. So, as always, please send us your feedback.
And enjoy your searching!
– Chris
This morning, our web servers started receiving hundreds of requests per second from two separate IP addresses- 71.39.13.57 and 65.102.12.225. The cumulative effect of these requests was to degrade service to other users, so, I have blocked those IPs at our firewall.
If these requests came from you, and this was unintentional, please let us know, and we'll remove the blocks once you fix your software. Otherwise, the IP blocks will remain in place.
- Danny
We’re happy to announce the release of a long-awaited feature: Private messaging.
First, click a nickname in the chatters list.
Then you’ll get a popup window for private messaging, as well as the profile of the person you’re talking to.
The messages sent from here are kept private — only you and the recipient will see them.
That’s it!
Note that those private messages aren’t saved, so they’ll disappear when you hit the reload button in your browser.
Also, you can send private messages even when you’re an observer. That means you don’t need to join the conversation if you’d just like to send a quick note to a someone in the room.
As always, we have many other enhancements! The following are some of the more significant changes that we’ve made in this release:
We’ve been psyched about this new release internally for the past couple weeks, and are really happy to share it with all of you! We hope you love what you see!
Don’t forget to tell us what you think!
– Kenn
