Custom output formats in mod_ndb

Often enough, it seems like you can begin working on software project with the assumption that you won't need to, say, design a little language and build a parser for it -- and yet, before long, that's exactly what you end up doing.

In the case of mod_ndb, my web services gateway for MySQL Cluster, the little language turns out to involve page-description -- it describes how to take a table of results from a database query and present it within an HTTP response. Inside an httpd.conf file, it looks something like this:

<OutputFormat XML> 

   Table  scan = '<NDBScan>\n$row$\n...\n</NDBScan>\n' 

   Row    row  = ' <NDBTuple> $attr$ \n  ...  </NDBTuple>'

   Record attr = '<Attr name=$name/Q$ value=$value/Qx$ />' or '<Attr name=$name/Q$ isNull="1" />'

</OutputFormat> 



<OutputFormat JSON> 

   Table  array  = '[\n $object$,\n ... \n]\n'

   Row    object = ' { $pair$ , ... }' 

   Record pair   = '$name/Q$:$value/qj$' or '$name/Q$:null'

</OutputFormat>

The three data types are a Table, representing a result table, a Row of data, and a Record, which (though it's probably not a very good term for it) is a column name/value pair.

The record description breaks down to $name$ and $value$, with some flags: $value/Q$ means to put the value in quotes; $value/q$ means to quote it only if it's character (not numeric); $value/j$ means to encode the value JSON-style with backslash escapes, while $value/x$ means to encode it XML-style using HTML entities (> and so forth).

A record description contains both non-null and null varieties. A row description describes how to loop over the columns in the row, and a table description describes how to loop over the rows of the table. Everything except the $variables$, the elipses, and the \n is plain text.

The format is (I hope) concise and intuitive, and I can parse it with a simple little two-pass compiler. First read each line and build a symbol table; then walk the table and parse each format. It's not exactly how I had intended to spend the weekend, but it mostly works. Now I've got three weeks to test it and get the bugs out before the MySQL Conference.