The Giles production rule system compiler has just been released! It is available for download here.

Production rule systems (or "engines" in Giles parlance) are tools that are commonly used to efficiently find patterns in streams of data where any number of data items (or "facts") can be added or removed over time. They're very commonly used to perform complex behavior detection (i.e., event correlation), like fraud detection for credit cards via transaction history or multi-part attacks against servers via combined analysis of firewall and server logs. They can also be used to provide some form of artificial intelligence, forming the core of many expert systems and automated planners.

All that sounds great, but what is Giles?

Well, first off, let me explain the motivation behind Giles. Traditionally, production rule systems are either standalone, or complex packages with APIs accessible from only a handful of languages. We wanted to build a new breed of compiler that lets users create engines that are accessible from any programming language and easily embedded inside larger projects. To that end, we created Giles.

Giles's claim to fame is that it can turn a normal relational database (SQLite in the current release) into a production rule system (engine). It does this by compiling a description of the engine into a database schema. Databases created using this schema instantly become the described engine, with no additional software or driver program needed.

This approach has immediate advantages, the most important being that any language that can access the database can be used to access and drive the engine. This makes it much easier to embed complex event correlation, artificial intelligence, and automated planning inside larger applications. Another interesting benefit is that these production systems can take advantage of the underlying database's data-safety guarantees (e.g., transactions, data durability, etc.). Finally, these production systems can handle terabytes of data, survive system crashes, and be run over long periods of time.

Production rule systems are often considered esoteric or hard to understand or use. But don't let that hold you back. One of our goals is to make these powerful computational tools more accessible to a wider audience. The distribution tar ball includes several examples that you can experiment with.

In conclusion, I hope all this sounds interesting to you. If it does, please download Giles, read the documentation, and give the examples a try. Also, stay tuned to the KoreLogic Blog where we will post various worked examples, tips, and tricks in the days ahead.