Thursday, December 4, 2008

QuickCheck(ing) the code i C

As part of my work at Witsbits during the summer, I wrote some C code for processing firewall configuration requests. The set up was straightforward; an URI based request sent to a server was handled by a system written purely in C which forwarded the requests to an appropriate handler function. Legal rules where accepted, translated into netfilter rules and stored in an SQL data base.

The customer would be able to add (I) and delete (II) firewall rules; the add (I) rules had several possible call structures which mixed groups, single ip's and other firewalls. Delete (II) calls could remove rules (if present). Since the add (I) calls had several different possible (legal) combinations, the code naturally became case-testing with a lot of "if(...)".. "switch(..)" etc. As a Haskell:er I wanted several ways of reassuring myself that the code I wrote behaved itself nicely and didn't mess things up. With this in mind, I started early by writing the foundation of a haskell program in parallel to the code I was writing in C. This proved to be quite helpful since it forced me to think extra carefully about what calls where legal and how I would check that a call was legal in an efficient way.

As the project progressed, I soon had a full fledged working haskell program using QuickCheck, Network libraries (HTTP,TCP,URI) and Codec.Binary.Base64.String with yet other minor auxiliary libraries. The haskell program identified itself to the firewall server, and generated valid and invalid packets on the fly, sending them across the network and recording the replies. The server, which handled the requests and replied with 200, 201 or 404 if something went wrong, also output a lot of debug info.

So, the haskell program was running controlled randomized test cases against a server! And voila, I was swatting bugs in my C code with minimal effort.

The greatest effort was put into learning to use the Network libraries properly; but besides that I must say this worked as a charm. Next time I'm faced with writing C code for a web based system, I'll start equally early on the Haskell program that will be testing it.

QuickCheck just made it quicker to find the bugs.

/Gianfranco

11 comments:

Anonymous said...

Just out of curiosity, why were you writing the actual web application in C? It doesn't sound like this app is particularly CPU bound ...

I understand that some companies wouldn't want to use haskell code in production since it could be difficult to find other coders to maintain it after you leave (not to mention that they may not have the in house expertise to configure the servers properly). But why not use something quick and easy to develop with (Perl, Python, etc.)?

Gianfranco Alongi said...

Hi Shaneal.

I believe they chose C due to their chief programmer being something of a C expert, with a lot of C experience under his belt. Also; I was not in the position to affect their choice of language.

I do however agree, although Perl lends itself to many bugs. This system really has to work fast and efficient; being able to handle huge amount of requests without choking.

Anonymous said...

hey,

neat stuff.

can you post a small sample test
showing how you use QuickCheck to test your C based cgi app?

Gianfranco Alongi said...

I shall see where I can post it, I'm not so well informed about the code posting possibilities of blogger.

Paul Johnson said...

For some time I've been collecting comparative SLOC counts for equivalent software implemented in different languages. Your parallel implementations of the same functionality in C and Haskell are exactly the kind of thing I want. Any chance of posting the code sizes?

Gianfranco Alongi said...

Hi, Paul.

I don't have access to the Firewall code writtenn in C, but I do have the haskell code of course. Sloccount reports 265 lines of code. My main issue was correctness, thus the haskell code may be a bit bloated with explicitness.

Total Physical Source Lines of Code (SLOC) = 265

Cheers.

Anonymous said...

why not poste the code on pastie.org and then just have a link to it.

i think this stuff is really interesting. If you have a tool that can save a lot of time, I would love
to see it posted.

Gianfranco Alongi said...

Hello (whoever you are).

Not a bad idea; I might do this, first I'll have to blank out some parts of the code which may not be handed out into the public.

ADEpt said...

See also http://www.haskell.org/haskellwiki/QuickCheck_as_a_test_set_generator

Gianfranco Alongi said...

Very nice comment; I was not aware of this page! This shows the versatility and strength of Haskell.

Anonymous said...

Who knows where to download XRumer 5.0 Palladium?
Help, please. All recommend this program to effectively advertise on the Internet, this is the best program!