MASTIFF is a living project whose continuous goal is to provide an automated means for static analysis of files. To meet this end, the project has multiple short and long term goals in place. Recently we silently released an update that hit one of the major goals we have been working towards since inception of the project: output plug-ins.

Previously, MASTIFF had two types of plug-ins: category plug-ins, that determine the type of file that was being analyzed; and analysis plug-ins, the code that extracts and interprets the information from the file. Any output generated from MASTIFF plug-ins had to be handled by the plug-in itself. This led to multiple problems.

• Output handling code was being replicated in all of the plug-ins.
• There was no consistency with how output was formatted.
• If a new output format was desired, such as HTML, each plug-in had to be updated to handle that new format.

Thus output plug-ins were born. Output plug-ins take the data generated from the analysis plug-ins and place them in a specific output format. In other words, the text output plug-in will place the data into text files, the HTML output plug-in (forthcoming) will format the data into HTML files, etc. This allows the analysis plug-ins to focus solely on performing analysis, and allows new output formats to be quickly added to the framework.

However, a consistent format to place the data into was needed for the output plug-ins to parse and format the data properly. Unfortunately, standard formats such as JSON did not do everything that was required. So, a new "universal" format was created for MASTIFF. This format places analysis plug-in data into what we've termed tables and pages.

The majority of the data extracted by analysis plug-ins can be abstracted into one or more tables of data. For example, the embedded strings plug-in structures data into one table. Each row of the table is the information related to a specific extracted string in the file, and each column is that data field (e.g. location, type, and the string itself).

MASTIFF uses this to its advantage by storing all data in a table-like structure (known oddly enough as a table). Each table contains a header, which describes the data in the table; and multiple rows, which contain the data.

Plug-ins may also generate multiple pieces of information that each need to be stored in their own tables, but still grouped together. Another data structure, known as a page, was created to group multiple tables from a single plug-in together.

Combined, each analysis plug-in output is stored in its own page. Within each page are one of more tables of data. Output plug-ins read each plug-in's page and go through each table, formatting the data within to the format required.

There are advantages to using the universal format outside of the output plug-ins. First, it is now possible to create plug-ins that add to or modify data from other plug-ins. For example, a plug-in that generates a new hash based on a file type does not need to place its data (i.e. a single hash) into its own data structure; it can add its hash to the File Info page data, which already contains all of the hashes. Second, plug-ins can now examine the data from other plug-ins to perform correlation on it. This allows MASTIFF to be extended even further.

From this point forward, all new analysis plug-ins will utilize the output plug-in data structures to take advantage of output plug-ins. We are working on converting all current plug-ins to use output plug-ins. The details on how to use output plug-ins, and create new ones, are contained within the MASTIFF documentation. The analysis plug-in skeleton files, used to quickly generate new plug-ins, have also been updated to utilize output plug-ins.

Currently, there are only two output plug-ins available: raw, which displays the data structures in their raw format (useful for debugging); and text, which puts the data into text files. Additional output plug-ins, including JSON and HTML, will be forthcoming.

MASTIFF Online Plug-in

One last update. We also recently pushed a new plug-in that will submit files to MASTIFF Online Free from MASTIFF if enabled. MASTIFF Online is the service that allows anyone to upload samples and have them analyzed by MASTIFF. The benefit to using MASTIFF Online, instead of a local install of MASTIFF, is that one does not need to worry about keeping MASTIFF up to date or if all of the dependencies are installed.

The plug-in will upload the sample to MASTIFF Online, if the submit option is set to on, and will return the URL where the results may be found.

There are two benefits to using this plug-in. First, it helps increase the size of the MASTIFF Online repository of malware, which will help create new plug-ins and shape the future of MASTIFF. Second, when malware is uploaded to MASTIFF Online, its fuzzy hash is compared to all other malware in the repository. By uploading your malware, you can go to the site and see if there are any samples similar to the sample you are analyzing. This helps your analysis, and MASTIFF in general.

Do bear in mind that MASTIFF Online Free's database is public, so do not enable this when processing files with confidential or proprietary contents.