[Guest Diary: Xavier Mertens] [Integrating VirusTotal within ELK]
Visualisation is a key when you need to keep control of whats happening on networks which carry daily tons of malicious files. virustotal.com is a key player in fighting malwares on a daily basis. Not only, you can submit and search for samples on their website but they also provide an API to integrate virustotal.com in your software or scripts. A few days ago, Didiers Stevens posted some SANS ISC diaries about the Integration of VirusTotal into Microsoft sysinternal tools (here, here and here). The most common API call is to query the database for a hash. If the file was already submitted by someone else and successfilly scanned, youll get back interesting results, the most known being the file score in the form x/y. The goal of my setup is to integrate virustotal.com within my ELK setup. To feed virustotal, hashes of interesting files must be computed. Im getting interesting hashes via my Suricata IDS which inspect all the Internet traffic passing through my network.
The first step is to configure the MD5 hashes support in Suricata. The steps are described here. Suricata logs are processed by a Logstash forwarder and MD5 hashes are stored and indexed via the field fileinfo.md5:
Note: It is mandatory to configure Suricata properly to extract files from network flows. Otherwise, the MD5 hashes wont be correct. Its like using a snaplen of 0">filter {
if ( [event_type] == fileinfo and
[fileinfo][filename] =~ /(?i).(doc|pdf|zip|exe|dll|ps1|xls|ppt)/ ) {
virustotal {
apikey = ">
field = [fileinfo][md5]
lookup_type = hash
target = virustotal
}
}
}
The filter above will query for the MD5 hash stored in fileinfo.md5com if the event contains file information generated by Suricata and if the filename contains an interesting extension. Of course, you can adapt the filter to your own environment and match only specific file format using fileinfo.magic or a minimum file size using fileinfo.size. If conditions match a file, a query will be performed using the virustotal.com API and results stored into a new virustotal field:
Now, its up to you to build your ElasticSearch queries and dashboard to detect suspicious activities in your network. During the implementation, I detected that too many requests sent in parallel to virustotal.com might freeze my Logstash (mine is 1.5.1). Also, keep an eye on your API key consumption to not break your request rate or daily/monthly quota.
(c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.