Windows Event Forwarding & .ETL (ETW)

What’s so useful about ETL (ETW)? 

One of the most useful (.etl) log files is: WMI-Activity/Trace. It is an Event Tracing for Windows (ETW) log which gets written to a ‘.etl’ file. The information inside of this log can be extremely useful to anyone wishing to monitor WMI, as it logs each query, new class, consumer, etc. We touched on this in our recent talk at Bloomcon (https://youtu.be/H3t_kHQG1Js?t=99)
Note: monitor this blog for a future post just about WMI events.

What’s the problem? 

Unfortunately, WEF (Windows Event Forwarding) and many other event forwarding solutions cannot subscribe directly to ‘.etl’ files..). However, the ‘.etl’ file can be read and converted into a channel that WEF can subscribe to. To help facilitate this, I wrote a PowerShell script, and it is available on github (https://github.com/acalarch/ETL-to-EVTX). Use at your own risk 😅!

The script in action!

In summary, the script will query whatever (.etl) file you give it every 15 seconds and write those events to a new channel. Actually, it can do this for any (.etl) file! You just have to configure it to do so.

To prepare your etl file for the script, all you have to do is change some of the channel options. Getting an error in Windows Event Viewer is normal after you make these changes. It doesn't like displaying ETL with "overwrite events as needed". 

Settings for the ETL file/channel.

Other Solutions

My solution is kind of a poor-man’s solution to this problem. Here are some more:

You can read more about ETW here:

You can read a whole lot more about WEF and Windows Events by reading our slide deck or watching our talk:



Malicious [.reg] Files

The Problem

Criminals and red teams have been known to use .hta, .vbs, .vbe, .js, .jse, .html, .bat, .cmd files to break into a computer/network. However, you don't hear too much about [.reg] files, which will be interpreted by RegEdit to make changes to the registry. On a default installation of Windows, the user does not need special admin privileges to add keys to HKCU\Software\Microsoft\Windows\CurrentVersion\Run. So, if they receive an email and are tricked into running the [.reg] file, they could be adding an 'evil' key. Currently, gmail is not blocking these files by default. You may want to check your email provider / gateway!
Here is an example of what a malicious [.reg] key might look like.. this one just launches calc. 
"Malicious" Reg Key, Adds key that will run calc.exe via mshta.exe when the victim logs in 


You can monitor this activity with Windows Event ID 4688 where the command line details for the event contain "regedit.exe" and endswith ".reg". Additionally, you can always monitor runkeys by enabling the "object access" policy or using a tool such as sysmon. Also, you should always monitor events created by mshta; wscript; cscript; regsvr32.exe and scrobj.dll as these (incomplete list I'm sure) can be used to create persistence in run keys. 

Adam Swan / @acalarch
Nate Guagenti / @neu5ron


VBA Obfuscation and Macro Obfuscation

Visual Basic Obfuscation via Line Continuation

Be careful while writing YARA signatures for Microsoft Office Macros. A simple technique used to bypass detection of “sub document_open()” for instance is to break it up with the VBA line continuation character “_” (underscore).

We’ve seen this break a few office malware signatures… so you may wish to check your vendor.

If your vendor is only looking for "document_open" or the equivalent VBA of auto-open then you will be ok. However, if vendor is looking for the surrounding parentheses or preceding "sub" then you may want to double check.

Below are three examples:

**split among 3 lines

**split among several lines

**VirtualProtect (commonly used when executing shellcode) being imported from Kernel32 split among several lines

# Yara Rule
rule VBALineContinuationObfuscation
   Author = "@acalarch, @neu5ron"
   Description = "Identifies potential VBA Obfuscation via empty line continuation, must provide yara an uncompressed vba project”
      $a = {20 5F 0D 0A 20 5F 0D 0A}


Adam Swan / @acalarch
Nate Guagenti / @neu5ron


WEF Server, Add Missing Channels


In collaboration with Adam Swan (@acalarch), in our spare time, we have been setting up Windows Event Forwarding collections and looking at the thousands of windows logs. We then rate their corresponding volume and level of confidence related to information security (as well as for windows system admins and helpdesk). We also have been collaborating this same information with Florian Roth (@cyb3rops) who is working on essentially the same thing.

This post assumes that you have set up the basics of Windows Event Forwarding / Windows Event Framework / Windows Event Collection / Windows Event Subscriptions. (relevant side note: Microsoft apparently hasn’t identified a common lexicon when talking about windows events & subscriptions). 

The Problem
While we were attempting to collect logs from certain clients we would notice that they had software that had windows event log locations which were not on the WEF server. When you create a subscription to a computer and you go to select an event channel to pull from, the list of event channels is populated by what is available on the subscriber (the server collecting the events). Therefore, if a channel exists on a client being collected from but not on the subscriber, the channel will not be available (see figure x). Also, performing registry hack may sometimes cause instability. (ie: In HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Publishers;HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Channels;HKLM\System\CurrentControlSet\Control\WMI\Autologger).
Client on the left, Sysmon channel available. Subscriber on the right, Sysmon Channel Not Available. How do I subscribe to a channel that doesn’t exist locally??

The Solution

The solution, is to add the missing channels to the subscriber. You can do this by installing the manifest for missing windows event channel OR the much simpler way of adding the channel subscription in XML form. Also, https://blogs.technet.microsoft.com/russellt/2016/05/18/creating-custom-windows-event-forwarding-logs/ may be of help. First we’ll dig into a little bit of background knowledge for your sanity and then we’ll provide a step-by-step on how to create a provider for either type.

Background on Windows Events

There are basically two ways to create a channel… Classic and Manifest Based [https://msdn.microsoft.com/en-us/library/windows/desktop/aa364161(v=vs.85).aspx]. This post will describe how to create the channel for either.

Creating a Classic Channel:

This will only work for Logs that are NOT under the“Application and Services Logs” path.
This is an easy one liner in powershell “new-eventlog –Logname “ThelognameIwanttocreate” –Source “TheSourceIwishTocreate”

Creating a Manifest Based Channel:

This Involves Two Steps:
  1. Collect the manifest off of a computer where the channel exists already or from the executable that creates the channel
  2. Install Manifest using wevtutil im manifest.name

Background for Manifest
The manifest for a channel is an XML document that describes a provider. The provider name becomes, the name for your channel as you are used to seeing in the Windows Event log. [https://msdn.microsoft.com/en-us/library/windows/desktop/dd996930(v=vs.85).aspx]

Obtaining the Manifest You Need
There are several ways of going about getting the manifest. It may be published online. In the case of sysmon, simply running “sysmon –m” will install the event manifest and nothing else. Other times it can be quite tricky to find it. Here are some methods we’ve tested and had success with.

Windows PerfView

The easiest way is to try your luck with PerfView by Windows [https://www.microsoft.com/en-us/download/details.aspx?id=28567].
Perfview has a command “dumpRegisteredManifest”. This command will dump the Manifest for the specified channel into the current working directory. This worked for most channels, we tested.
Running “perfview /nogui /accepteula userCommand DumpRegisteredManifest [Channel-Name]” on a host to obtain the desired manifest.
Notepad++ (or any of your favorite IDE/text editor)
Another way to obtain the windows event manifest is to search for it inside the executable you believe contains the manifest. Notepad++ has a decent search utility that will allow you to search for the manifest. Try keywords that should exist in the manifest for each of the executables associate with the channel such as “eventman.xsd” (you may also want to try “e.v.e.n.t.m.a.n.\..x.s.d” as the manifest may be stored in Unicode).
Sysmon Manifest found within the executable.

Installing the Manifest

Luckily installing the manifest is a simple one liner. The resources could not be found error should be expected as we are installing the manifest without installing sysmon, the channel will still appear in the windows event viewer.
“wevtutil im mymanifest.whatever”

Final Thoughts

It’d really be nice if Microsoft would go ahead and just make the manifests exportable without installing additional tools. Additionally, if you are a developer be a scholar like Mark Russinovich (sysmon) and publish your manifest or make it easily installable.
tag:Add Client Log Channel to WEF Server


Download all of Malware-Traffic-Analysis.net PCAPs

Download PCAPs from www.malware-traffic-analysis.net
http://www.malware-traffic-analysis.net/ is an excellent resource that a lot of people in the infosec community use. Hats off to @malware_traffic for creating a valuable resource for the community.

I have always wanted to download all the PCAPs from the site to run locally for different purposes. The PCAPs are useful for a variety of reasons. Including using to replay/re-run in order to check your IPS and/or IDS, passive dns implementation, collecting more malware samples, training exercises, etc..

So I wrote a python script last night to do that. I was going to release the script online, but I thought "wellp if a good amount of people run this script than it will cause a lot of unnecessary traffic to Brad's (@mawlare_traffic) site".
Instead of releasing the script I decided to just create a GitHub repo and upload all the PCAPs there.

Just run the following command to download all of the PCAPs.
git clone https://github.com/neu5ron/malware-traffic-analysis-pcaps.git

If anyone has any comments, expletives, or any other feedback then please comment.


Setup ElasticSearch Logstash and Kibana ELK with Bro

I would not follow this installation process anymore, but you may use it for a few notes. As logstash-forwarder has changed file locations and TLS configuration. Kibana has change ALOT from v3 to v4.

This tutorial will install ELK stack and input Bro HTTP, SSL, Conn, DNS, Files, and DHCP logs with GeoIP and using Kibana over HTTPS.

This documentation is assuming you are using Ubuntu as the server. I was using a 32GB RAM server with 6 cores.

Server Installation:
#Install Java
sudo add-apt-repository -y ppa:webupd8team/java;
sudo apt-get update;
sudo apt-get -y install oracle-java7-installer;

#Install ElasticSearch
wget -O - https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -;
echo 'deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main' | sudo tee /etc/apt/sources.list.d/elasticsearch.list;sudo apt-get update;
sudo apt-get -y install elasticsearch;

sudo vi /etc/elasticsearch/elasticsearch.yml
#Add the following line somewhere in the file, to disable dynamic scripts:
script.disable_dynamic: true
#Find the line that specifies network.host and uncomment it so it looks like this:
network.host: localhost
Save and exit elasticsearch.yml.

#Tune ElasticSearch
#Add to /etc/sysctl.conf
fs.file-max = 65536

#Add to /etc/security/limits.conf
* soft nproc 65535
* hard nproc 65535
* soft nofile 65535
* hard nofile 65535
elasticsearch - nofile 65535
elasticsearch - memlock unlimited

#Uncomment the following lines and change the values in /etc/default/elasticsearch:
#Set ES_HEAP_SIZE to half of your dedidcated RAM max 16GB

# Uncomment the line in "/etc/elasticsearch/elasticsearch.yml"
bootstrap.mlockall: true

sudo swapoff -a
#To disable it permanently, you will need to edit the /etc/fstab file and comment out any lines that contain the word swap.

#Reboot server
sudo shutdown -r now
#start elastic search:
sudo start elasticsearch restart
#autostart elasticsearch:
sudo update-rc.d elasticsearch defaults 95 10
#Install Kibana
cd ~; wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz;
tar -zxvf kibana-3.1.2.tar.gz;
#Open the Kibana configuration file for editing:

vi ~/kibana-3.1.2/config.js
In the Kibana configuration file, find the line that specifies the elasticsearch, and replace the port number (9200 by default) with 80:

elasticsearch: "http://"+window.location.hostname+":80",

sudo mkdir -p /var/www/kibana;
sudo cp -R ~/kibana-3.1.2/* /var/www/kibana/;

#Install Nginx
sudo apt-get -y install nginx;
cd ~; wget https://gist.githubusercontent.com/thisismitch/2205786838a6a5d61f55/raw/f91e06198a7c455925f6e3099e3ea7c186d0b263/nginx.conf
Find and change the values of the server_name to your FQDN (or localhost if you aren't using a domain name) and root to the location where we installed Kibana, so they look like the following entries:
vi nginx.conf
 server_name           localhost;
 root /var/www/kibana;

sudo cp nginx.conf /etc/nginx/sites-available/default;
sudo apt-get install apache2-utils;
#replace $USERNAME with your username you want to use
sudo htpasswd -c /etc/nginx/conf.d/kibana.myhost.org.htpasswd $USERNAME;

#Make Kibana over SSL:
#generate certificate
sudo openssl req -x509 -sha512 -newkey rsa:4096 -keyout /etc/nginx/kibana.key -out /etc/nginx/kibana.pem -days 3560 -nodes

sudo vi /etc/nginx/sites-available/default
#change the listen on port to *:443
#and add to the file under the line that says "access_log            /var/log/nginx/kibana.myhost.org.access.log;":
 #Enable SSL
 ssl on;
 ssl_certificate /etc/nginx/kibana.pem;
 ssl_certificate_key /etc/nginx/kibana.key;
 ssl_session_timeout 30m;
 ssl_protocols TLSv1.2;
 ssl_prefer_server_ciphers on;
 ssl_session_cache shared:SSL:10m;
 ssl_stapling on;
 ssl_stapling_verify on;
 add_header Strict-Transport-Security max-age=63072000;
 add_header X-Frame-Options DENY;
 add_header X-Content-Type-Options nosniff;

#Change the line "elasticsearch: "http://"+window.location.hostname+":80"," in /var/www/kibana3/config.js to 
elasticsearch: "https://"+window.location.hostname+":443",

#Restart nginx
sudo service nginx restart;

#Setup GEOIP
sudo mkdir /usr/share/GeoIP; #Create location that we will use to store the GeoIP databases/information
sudo wget http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz; #IPv4 ASNumber Database
sudo wget http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNumv6.dat.gz; #IPv6 ASNumber Database
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz; #IPv4 GeoIP Country Code Database
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz; #IPv6 GeoIP Country Code Database
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz; #IPv4 GeoIP City Database
sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz; #IPv6 GeoIP City Database
sudo gzip -d Geo*; #Decrompress all the databases
sudo mv Geo*.dat /usr/share/GeoIP/; #Move all the databases to the GeoIP directory

#Install LogStash
sudo apt-get install git;
echo 'deb http://packages.elasticsearch.org/logstash/1.4/debian stable main' | sudo tee /etc/apt/sources.list.d/logstash.list;
sudo apt-get update;
sudo apt-get -y install logstash;
sudo mkdir -p /etc/pki/tls/certs;
sudo mkdir /etc/pki/tls/private;
cd ~/ && git clone https://github.com/logstash-plugins/logstash-filter-translate.git;
sudo cp logstash-filter-translate/lib/logstash/filters/translate.rb /opt/logstash/lib/logstash/filters/translate.rb;
rm -rf logstash-filter-translate/;

#Server Config File
#Clone server config file
sudo apt-get install git;
git clone  https://github.com/neu5ron/siem-and-event-forwarding-configs.git;
sudo mv siem-and-event-forwarding-configs/logstash-server.conf /etc/logstash/conf.d/all_logstash.conf;

Client Installation:
wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -;
echo 'deb http://packages.elasticsearch.org/logstashforwarder/debian stable main' | sudo tee /etc/apt/sources.list.d/logstashforwarder.list;
sudo apt-get update;
sudo apt-get install logstash-forwarder;
cd /etc/init.d/; sudo wget https://raw.github.com/elasticsearch/logstash-forwarder/master/logstash-forwarder.init -O logstash-forwarder;
sudo chmod +x logstash-forwarder;
sudo update-rc.d logstash-forwarder defaults;
sudo mkdir -p /etc/pki/tls/certs;
cd /etc/pki/tls; sudo openssl req -x509 -batch -nodes -days 3650 -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt
#Copy cert to logstash server
scp /etc/pki/tls/certs/logstash-forwarder.crt user@server_private_IP:/etc/pki/tls/certs/
#Copy key to logstash server
scp /etc/pki/tls/certs/logstash-forwarder.key user@server_private_IP:/etc/pki/tls/private/

#Client Logstash configuration file #Clone server config file git clone https://github.com/neu5ron/siem-and-event-forwarding-configs.git
#Make sure you edit the file "logstash-bro_client.conf" to include the location of your bro logs and your $SERVERIP before moving the file.
sudo mv siem-and-event-forwarding-configs/logstash-bro_client.conf  /etc/logstash-forwarder

#Restart logstash
sudo service logstash-forwarder restart

#Now start logstash on the server
sudo service logstash restart


  • errors/logs for logstash
    • for server /var/log/logstash/logstash.log
    • for client /var/log/syslog
  • Logstash troubleshoot client
    • sudo /opt/logstash-forwarder/bin/logstash-forwarder -config=/etc/logstash-forwarder
  • Logstash troubleshoot server
    • sudo /opt/logstash/bin/logstash -f /etc/logstash/conf.d/all_logstash.conf --configtest
  • ElasticSearch
    • get list of indexs
      • curl -XGET 'http://localhost:9200/_aliases'
    • delete a specific index
      • curl -XDELETE 'http://{server}/{index_name}/{type_name}/'
        • example: curl -XDELETE 'http://localhost:9200/logstash-2014.11.18/palo_alto_traffic_log'
      • delete all database
        • curl -XDELETE 'http://localhost:9200/*'

Documentation Followed:
#Palo Alto
#Using Kibana


(Malware,URL,Domain,andIPAnalysisSites &Tools)and(SecurityTools/Software)

For reputations, online sandboxes, etc visit
#TODO:CleanUp Below



FireFox Security Add-Ons:
  • NoScript
    • enable/disable JavaScript (remembers approvals/denials) on a per site basis or temporarily allow/disallow globally
  • QuickJava
    • enable/disable Java, Flash, JavaScript, SilverLight, CSS, Animated Images, Proxy, Images, Cookies globally (meaning 1 for all, does not do site-by-site basis, either on or off for every site) 
  • Ghostery
    • Ad-Blocker
  • Flashblock
    • enable/disablle Flash (also allows click to play) on a per site basis or temporarily allow/disallow globally
  • VTZilla
    • right click on any link and scan the target with VirusTotal.
  • Cookie Monster
    • enable/disable cookies on a per site basis or temporarily allow/disallow globally
  • HTTPS-Everywhere
    • auto HTTPS connection of websites this will automatically redirect to the secure version of the website you request (if the website supports HTTPS) inside your browser.
      For example instead of going to google.com and then Google redirecting you to https://encrypted.google.com/ once you try to hit google.com this Add-On will auto redirect you to https://encrypted.google.com before you ever make a connection to the internet.
  • Defaced websites archives:
  • Attrition  http://attrition.org/mirror/
  • Hack-DB  http://www.hack-db.com/
  • Zone-H  http://www.zone-h.org/

#Chrome Security Extensions:
  • HTTP Switchboard
    • enable/disable JavaScript, cookies, images, etc (remembers approvals/denials) on a per site basis
  • Ad-Blockers
    • Ghostery
    • Ad Block Plus
    • Disconnect
  • HTTPS-Everywhere
    • auto HTTPS connection of websites this will automatically redirect to the secure version of the website you request (if the website supports HTTPS) inside your browser.
      For example instead of going to google.com and then Google redirecting you to https://encrypted.google.com/ once you try to hit google.com this Add-On will auto redirect you to https://encrypted.google.com before you ever make a connection to the internet.
  • Mailvelope
    • Secure email with OpenPGP encryption for Webmail (supports Gmail)
  • NotScripts ScriptSafe
    • enable/disable JavaScript (remembers approvals/denials) on a per site basis or temporarily allow/disallow globally
  • Vanilla Cookie Manager
    • Auto clear cookies, enable/disable auto clear of cookies on a per site basis.