Quick update

Over the holidays I spent a lot of time working on node-ncurses and improving it. I pushed the changes I made to the github repo awhile ago for those interested.

I’ve also recently pushed a change to node-imap which makes the fetch() mechanism async and a stream. By streaming, I mostly mean it uses event emitters with familiar event names to transfer fetched messages and their data. This should fix the reported out-of-memory issues. Message headers are still buffered internally though, since they need to be processed and turned into an object/hash.

With those changes out of the way, I’m hoping to now spend some time on node-oscar again.

FTP directory listing sorter

Today I was rather irked when I noticed that the contents of an FTP site I was browsing in Chrome was in no particular order. I was trying to find the newest file in the directory and had expected Chrome would have sorting capabilities by now for FTP directory listings.

So I took it upon myself this afternoon to write a quick little Chrome extension to do just this. It’s currently (only) activated for all ftp:// addresses. Download it here.

OSCAR: the unforgiving protocol

For the past week or more I have been struggling to find out why AOL was disconnecting me after making a connection to a second BOS server for additional services. The main BOS server would disconnect without any notice (connection reset by peer) when sending it any packets after connecting to and setting an additional connection.

This morning I was again looking into the issue and I was looking at a wireshark dump and it hit me: sequence numbers! I thought for sure AOL wouldn’t be strict about their sequence numbers, but after some testing it appears that is quite true. What was happening was I was sharing the FLAP sequence number among all connections, so by the time I had completed the sequence of commands on the additional connection, the shared sequence number had been increased quite a bit. So by the time I send another FLAP packet on the first/main connection, AOL decided it lost some packets judging by the difference between the last sequence number it received and the one contained in the current FLAP packet.

What I can’t understand is why they don’t send a FLAP error packet that would have easily highlighted the actual problem instead of forcefully severing the connection.

So a word to the wise: make sequence numbers connection-specific and (obviously) increment them only by 1 for each FLAP packet sent.

Lastly, as a quick update: I’ve got most of the basic functionality down and am now working on chat, file transfer, and direct connections.

oscar = argh

I’ve been making some real progress on node-oscar this past week. Lots of major refactoring has been going on the past couple days, mostly to support multiple server connections (because apparently no single server offers all services).

The thing that has been the most painful so far is dealing with the server-side information — adding, deleting, and modifying it. I’m not sure what exactly the AOL developers were thinking when they designed this part of the protocol, but I would have thought they would have come up with something better by now. It is very unnecessarily complicated compared to other commands and flows that occur in the protocol.

I also have yet to do any testing on ICQ, which should prove to be interesting to say the least…..

node-oscar

After finishing the majority of the main features my node.js imap module, I’ve been looking for another protocol to implement. SMTP was already started on by someone else and POP3 isn’t worth implementing IMHO. Then I started browsing the node.js module wiki page and saw people that are working on an XMPP module. So that gave me the idea of creating an implementation of OSCAR (the protocol for AIM/ICQ) for node.js.

I spent all of tonight and this morning gathering protocol information and coding and I’m now ~25% or so done. The documentation available on some parts of the protocol are completely missing (especially info about family/service versions).

tablet fun

For the past week or so I have been looking at some of the more worthwhile Android tablets that have started to crop up as of late. The Samsung Galaxy Tab (wifi edition) looked pretty nice, but the cost was outrageous and was definitely more than what I was willing to spend on a tablet.

One of the other wifi-only tablets I had looked at was the Viewsonic G-Tablet. The specs were pretty impressive and the price was pretty fair for the features and was much cheaper than the wifi-only Galaxy Tab. This evening I decided to take the plunge and go for the G-Tablet. Tonight I had the opportunity to perform the unboxing, which was quite exciting since I had only really owned/used mostly inferior tablet devices prior to this (resistive screens, half-assed Android ports, terrible performance, etc). Luckily I found out that the G-Tablet was actually being sold in two B&M stores: Sears and Staples. The Staples not far from me had just 1 left when I called, so I really lucked out.

Up until the past day or two, I had been reading up on and eyeing the Archos 101, which is cheaper and lighter. However it contains more or less the same generation of hardware as the Archos 5 IT, which I had already bought earlier this year. After less than stellar support from Archos, issues with some of the software (Android 1.6 too), a lack of significant progress on the custom ROM front, and the fact that recently my Archos 5 IT grew a bulge inside causing the back to expand, pressure to be put on the LCD, and the inability to get the power button to register, I decided to stay away from Archos this time around.

My first impressions of the G-Tablet are pretty good. When I was looking at the ports, I realized it actually had a full-size USB host port on the side, just like the Archos 101, something that I didn’t know it had (I thought it just had the micro USB connector and that an adapter would be needed). The screen looks nice, it responded very well to touches (again, I’m coming from resistive devices here), and the “TapNTap” UI layer that comes with the stock firmware is pretty nice even though it was a big sluggish in some places. The one problem I had right away was with the wifi settings. It saw my network just fine, but since I don’t use DHCP on my network, I have to specify the static IP settings manually. I did this and had the static IP option checked, but every time I tried to connect to my wireless router, it kept trying to obtain an IP through DHCP. I tried both re-enabling the wifi and rebooting to see if that might make the static IP settings take effect, but no go. So for now I ended up enabling the a DHCP server to dish out one IP just for the tablet. Hopefully one of the custom ROMs available will not have this issue.

The browsing experience was pretty smooth and pinch-to-zoom worked great. I would like to root my G-Tablet soon though to get the Marketplace and a real Android interface instead of Viewsonic’s TapNTap UI.

UPDATE: Several days ago I went ahead and installed the TnT Lite firmware (upgraded to TnT Lite 2.0 recently). After applying the marketplace fix and installing Flash 10.1 support, this tablet is great! Everything is as smooth as can be. About the only thing I miss with the upgrade to 2.0 is the ability to press and hold the Home button to bring up the Recent Apps popup, but that is not the fault of the TnT Lite developer, but a change made by Viewsonic with their latest firmware update.

node-imap

Some time back I started writing an imap module for node.js. I had most of the usual functionality finished, but I got stuck on trying to come up with a quick and easy way to parse the result of a BODYSTRUCTURE command. At the time I was hoping there’d be some giant regex that would just work for this, but I couldn’t find one.

So tonight I sat down and wrote my own BODYSTRUCTURE parser using the format detailed in the imap4rev1 RFC and it has worked with every BODYSTRUCTURE result I’ve thrown at it. Now I need to go back and refactor the rest of the module to make the whole asynchronous handling a little saner.

NodeBuilder + grappler = <3

So a little while ago I added a little thing to the top of every page that shows the status of NodeBuilder. What is NodeBuilder? It’s a script built using node.js that automatically downloads, compiles, and packages (.deb and now .rpm) the latest stable and unstable versions of 32-bit and 64-bit node.js as they become available.

Initially I just had this running in the background and it was spitting out the packages to a public-facing directory (click on the link next to the NodeBuilder status). However, around the same time I was putting the finishing touches on grappler, another project of mine. Since I was looking to create some extra (live) demos for grappler, I decided to extend NodeBuilder to use it. Performing this integration was quick and easy (as it should be) and NodeBuilder was up and serving its status to the public in no time.

Now, this is not really the best live demo since you basically have to hang out at the node.js IRC channel (irc.freenode.net #Node.js) to watch for updates to ryah’s node github repo so you know when to look for the NodeBuilder status to change. However, the reason for it was that it was a simple integration and it’s kinda neat to watch it do its thing when a new commit is made :-)

EDIT: NodeBuilder is down for now. Some changes need to be made.

node.js api irc bot

So today I spent some time writing up an irc bot (using node-irc) that lets you do node.js api lookups from an irc channel.

There are some small kludges because some parts of the api are treated specially. I’ve also had to make some modifications in a couple places in node.js’s api.markdown so that it can be parsed a little better.

I’ll probably be publishing the bot to github sometime soon, but I’d like to add some currently missing features yet before that happens.

socket.io/socket.io-node fixes

Lately I’ve had some time to fiddle with my forked node.js modules, socket.io and socket.io-node, some more. Unfortunately a good portion of work I had done on the client side (and some server side) got overwritten (mostly dealing with additional cross-domain support via XDR and stuff) some time ago. That’ll have to be re-implemented again sometime soon.

Recently however, I had been focusing on getting websockets to work again on socket.io-node. I noticed a problem in connecting to the server when I downloaded a new nightly build of Chromium. It seems that somewhere between the previous nightly build I had and this new one, the Chromium team had decided to switch to a new draft (76) of the websocket protocol.

The real problem I had was in generating the correct response using the new websocket security keys and 8-byte body. After a day or two of much debugging and trying various things, I finally got it working. Right now I have my fork of socket.io-node detect which websocket draft is being used and to use the appropriate version of the protocol, so old versions of Chrome/Chromium and newer versions should work the same.

I also made a few miscellaneous fixes to the xhr multipart streaming transport (client and server side), which I think I will end up having to applying to the xhr polling transport as well (cross-domain related).