Apatar, F/OSS Visual Data Integration

About 5 years ago I turned my lazy brain on its wheels around to help me create decent parsers to processes GSM switches configuration and traffic data. Anyone has ever deal with those data must have known the fun we could have in interacting with them.

I was a fool back then (now is not any better, I've become more foolish then ever), not knowing how things worked and fueled by young-man's passsion, I decided to create them by myself, hand-written using C, the "best programming language" ever created by mankind. :-)

I didn't just jump to pure C, though, I remember that I had first thought to create them using bison and flex (UNIX techies: they are free-software implementation of yacc and lex). For this, I studied those wonderful things about language grammars, stomata, recursive descent, right-recursive, left-recursive, the Backus-Naur Form, and many other things that I've now -ofcourse- forgotten about :-)

Apparently my learning was not enough or I was just simply to stupid to grasp the whole concepts or both. I manage to create some parsers, but they turn out to be so complex for me that I couldn't easily debug and adjust them everytime there are errors or updates.

So I turn to C. Just C this time. I even bought Peter van der Linden Deep C Secrets book. A really great and recommended book for learning C. I have some good laughs from there, too. (This reminds me, where is that book now? Hello?)

From the book, I got valuable lessons about finite-state machines. I use this knowledge to create my parsers using C, and succeeded in doing so. I could even understands the code at the time, although since it has never failed me yet, I've also forgotten about it :-)

I do remember that those experiences was while thrilling and enjoyable at the same time also hair-pulling and nightmarish. Something that I'm not sure I wanted to do again if the fish is not worth it.

Nowadays, I learn that people use these things called ETL (Extract, Transform, Load) tools for the kind of work. And so many software vendors has it now. ETL tools that can, at least in theory, read any data in any format and transform it to any format you wanted.

Remembering my past experiences, I'm amazed and thrilled to know that there is also such a tool in F/OSS world called Apatar (http://www.apatar.com). Apatar even has a visual-drag-and-drop-touch-and-go-you-name-it ability to create ETLs that they called DataMap. Many DataMap are already available in their http://www.apatarforge.org forge-site.

If only I have known about these tools back then. May have saved me some sleepless night.

Or not.