Status updates

Too many things are going on and all at the same time. Before I forget, I thought it would be good to write about some of the things taking place.

Tomato firmware:

Fresh on my mind is the Tomato firmware updates that I mentioned earlier. The Tomato firmware is a packaged piece of software that runs many popular Linux based routers (like the WRT54GL). I’m starting to purposefully purchase devices that run Linux so that I can customize them to do what I like. In recent weeks I have have distracted from other projects due to an important need to add a new feature to my WRT54GL router, namely adding bandwidth monitoring for each device on our family network. Our Internet Service Provider (ISP) limits the total monthly bandwidth that we are allowed to consume, so the need to track who is using bandwidth is very important for us (especially since I work from my home office most of the week).

A few weeks ago I wrote a small utility (cross compiled using the Tomato based tool-chain) called ipt-parse. This tool takes the output of: iptables -L xx -vnx and produces summary statistics. This was perfect to accomplish my goal of tracking bandwidth per client, but the v1 of the tool relied on reading the contents of numerous logfiles produced by the iptables output. As time passes, there would be many logfiles and that would clutter the file-system  so I decided to find a more permanent solution, a mini database. My first thought was to use something that was designed for a small footprint, sqlite. This proved to be a BIG challenge, not entirely due to sqlite but many factors as I shall describe. Within a day I managed to modify some of the sqlite compiler DEFINE statements and successfully cross-compiled sqlite to run in the tomato shell environment. While the application would run successfully, I kept noticing that sqlite would give me lock and disk i/i errors whenever I tried to execute a command that would access a database such as:

access permission denied

disk I/O error

Reading through various discussion groups had wasted a lot of time (but was an excellent learning process, especially related to low level file system I/O). At first I was led to believe that this was related to my specific environment which was:

Router (LinkSys WRT54GL): running Linux Kernel 2.4, Busybox 1.12, cifs client

NAS (Dlink DNS323): running Linux Kernel 2.6, Busybox 1.12, Samba Server 3.0.24

The router was acting as a client using a cifs mount to connect to the NAS which was running samba server. When running sqlite on the mounted cifs share I would see the errors, but when running on the local router file-system I would not have any problems! After searching around I found this defect reported by Daniel Kahn Gillmor. After a friendly and helpful exchange of dialog with both Daniel and a Samba Server developer (Shirish S Pargaonkar) I was able to discover a little bit more detail surrounding this defect. Shirish recommended I update my Samba Server version if possible (so I did as I am able to update packages on my DNS323!). After successfully upgrading the NAS to Samba 3.3.2 and trying out the locking issue, the error reported by sqlite was different but the underlying problem was the same! Due to being stuck with the old Linux 2.4 Kernel on the Router (because some of the device drivers are binary only and won’t work on newer Kernels) I was not able to update the cifs code to use any new bug fixes (which were all based on newer Linux Kernel changes). I tried completely disabling locking on the share used by the router in my smb.conf on the NAS Samba Server and that fixed the error reported by the testlock.c program mentioned in the Debian bug, but still had problems in sqlite (not that I wanted to have to do this as the solution anyways). After having a useful dialog with Shirish realted to cifs and Samba Server communication and behaviour we realized that this old cifs client did not support the newer communication protocol with Samba Server and thus Samba Server was assuming that the client was a Windows client. Running the exact same applications from my Ubuntu box proved that there was something specific to the router software that caused the problem as Ubuntu did NOT have this problem.

Eventually I passed back and forth thinking of scrapping sqlite and using something like mysqlclient library and running mysql on the NAS (and thus avoiding direct file-system access), but trying to get the mysqlclient library compiled in tomato is no small task (this was clearly not what I wanted to spend my time working on). Finally I applied some of the lessons I learned while debugging the locking issues in cifs / samba and looked carefully at the sqlite code and #define entities. Viola, a solution:

First the compiler settings:

@$(CC) $(CFLAGS) -o ipt-parse -L. src/ipt-parse.c src/sqlite3.c -ldl -DSQLITE_THREADSAFE=0 -DSQLITE_ENABLE_LOCKING_STYLE=1 -DSQLITE_DISABLE_DIRSYNC -DSQLITE_NO_SYNC

Secondly I discovered that sqlite attempts to use posix style locking for all non Apple and VxWorks Operating Systems (this is decided at compile time). Fortunately there are multiple locking mechanisms in sqlite, and the FIRST one in the list of locking types gets used so I changed it from posix to dotfile locking by moving this entry to the top:

static sqlite3_vfs aVfs[] = {
UNIXVFS(“unix-dotfile”,¬† dotlockIoFinder ),

Suddenly sqlite was working beautifully on the Linux 2.4 based router. Now that I have a working solution with sqlite, I have added the ability to save output directly from iptables -L into a sqlite database, and also the ability to query the database and produce statistics. Due to the way in which i have this add-on working, it is fairly fail-safe and self contained. It runs via a scheduled job in Tomato and saves data on a cifs share (which for any serious users should be something like my DLink DNS323 NAS which runs 24/7). Once this data is logged (I have my cron job set to save iptables stats every 5 minutes) I can query the statistics using a vast array of querying possibilities up to the granularity on 5 minute intervals (the interval is directly proportionate to the frequency of the cron job saving the stats). Once the logging code has proven to run stable and accurately, I’ll be able to add things like event triggers (email me when a client reaches x GB’s of traffic flow within x minutes of time). The data analysis side should likely be coded in something separate from the Tomato environment (due to flexibility requirements) but be easy enough to use on many accessible platforms. My idea is to partition the work into two parts: a) the data collection and summary view that runs directly from Tomato (written in C) and b) the data analysis part which provides a robust and flexible platform to show numerous views of the collected data (perhaps a web based interface using PHP for portability). This will open up a whole new world of statistical analysis possibilities and should prove to offer great value as it improves over time (a new download is now available from: here).

Glest:

After having spent a bit of time in the past looking through the code for this strategy game, we decided to a) fix a few bugs that were affecting our multi-player experience and b) the boys learned about making 3D models in blender so that we could change some of the game content for our own use. We ended up releasing the Farmer Faction and since then the boys have been continuing there efforts in creating more content. Once this content has been refined we may post it for public download.

Teamfest:

About 6 months ago I mentioned our need to find a good digial paintball 2 replacement that we could actually have access to all the code and be able to contribute to. Blood Frontier was the engine we decided to base our replacement on. Fortunately we got involved shortly before Quinn and company were ready to release and we offered ideas and a minor bug fix (again related to local multi-player LAN play). Quinn ended up adding a kid’s mode and paintball option to the game, which showed me that this project was “open” for business as in Open Source. While we have not spent much time on “Teamfest” lately we are just starting to ramp up our efforts to produce something that satisfies our previous paintball experience. The boys are making maps and starting to create more detailed models for Teamfest, while I will begin combing through the code and soon we’ll put together a design plan.

Family:

Our family is a happy one. One important family goal is to be debt free. This goal took real shape in the summer of 2008. We have been implementing many things to attempt a more modest lifestyle whereby we would not require to borrow. This would free our availability even further for doing God’s Will and spending time with each other. We have experienced a number of personal disappointments in recent months, but we also recognize that the Lord over-sees it all. This brings us great comfort. In the meantime we are doing what we can, and leaving the rest in god’s hands and trying to to fret about it.

That’s all for now.


Comments are closed.