Live Capture Procedures
As we move to a world of cloud based systems, we are increasingly finding that we are required to capture and analyse data over networks. Once, analysing a disk drive was a source of incident
analysis and forensic material. Now we find that we cannot access the disk in an increasingly cloud based and remote world requiring the use of network captures. This is not a problem however. The
tools that are freely available in both Windows and Linux offer a means to capture traffic and carve out the evidence we require
As we move to a world of cloud based systems,we are increasingly finding that we are required to capture and analyse data over networks. To do this, we need to become familiar with the various tools that are available for these purposes. In this article, we look at a few of the more common free tools that will enable you to capture traffic for analysis within your organisation.
Once, analysing a disk drive was a source of incident analysis and forensic material. Now we find
that we cannot access the disk in an increasingly cloud based and remote world requiring the use
of network captures. This is not a problem however.The tools that are freely available in both Windows and Linux offer a means to capture traffic and carve out the evidence we require.
For this reason alone we would require the ability o0 capture and analyse data over networks, but
when we start to add all of th4e other benefits, we need to ask, why are you not already doing this?
Live Capture Procedures
In the event that a live network capture is warranted, we can easily run a network sniffer to capture
communication flows to and from the compromised or otherwise suspect system. There are many tools that can be used (such as WireShark, SNORT and others) to capture network traffic, but
tcpdump is generally the best capture program when set to capture raw traffic. The primary benefit
is that this tool will minimize any performance issues while allowing the data to be captured in a
format that can be loaded into more advanced protocol analysers for review.
That stated there are only minor differences between Tcpdump and Windump and most of what
you can do in one is the same on the other (some flags do vary).
Tcpdump
Tcpdump uses the libpcap library. This can capture traffic from a file or an interface. This means that
you can save a capture and analyse it later. This is a great aid in incident response and network forensics. With a file such as, capture.pcap, we can read and display the data using the -r flag. For instance:
tcpdump -r capture.pcap
Will replay the data saved in the file, capture. pcap. By default, this will display the output to the screen. In reality, the data is sent to SDTOut (Standard Out), but for most purposes the console
and SDTOut are one and the same thing. Using BPF (Berkley Packet Filters), you can also
restrict the output – both collected and saved. In this way, you can collect all data to and from
a host and then strip selected ports (or services) from this saved file. Some of the options that apply
to tcpdump include (quoted with alterations from the Redhat tcpdump MAN file):
-A Print each packet (minus its link level header) in ASCII.
-c Exit after receiving a set number of packets (defined after c).
-C Before writing a raw packet to a savefile, check whether the file is currently larger than
a given file_size. Where this is the case, close the current savefile and open a new one.
-d Dump the compiled packet-matching code in a human readable form to standard output
and stop.
-dd Dump packet-matching code as a C program fragment.
-ddd Dump packet-matching code as decimal numbers (preceded with a count).
-D Print the list of the network interfaces available on the system and on which tcpdump can
capture packets.
This can be useful on systems that do not support the use of the ifconfig -a command.
-e Print the link-level header on each dump line.
-F Use file as input for the filter expression. An additional expression given on the command
line is ignored.
-i Listen on interface. This specifies the system interface to listen for traffic on.
-L List the known data link types for the interface and exit.
-n Don’t convert host addresses to names.This can be used to avoid DNS lookups.
-nn Don’t convert protocol and port numbers etc. to names either.
-N Don’t print domain name qualification of host names.
-p Don’t put the interface into promiscuous mode.
-q Quick (quiet?) output. Print less protocol information so output lines are shorter.
-r Read packets from file that has been created using the “-w” option.
-S Print absolute, rather than relative, TCP sequence numbers.
-s Snarf snaplen bytes of data from each packet rather than the default of 68 bytes.
-T Force packets selected by “expression” to be interpreted the specified type.
-t Don’t print a timestamp on each dump line.
-tt Print an unformatted timestamp on each dump line.
-ttt Print a delta (in micro-seconds) between current and previous line on each dump line.
-tttt Print a timestamp in default format proceeded by date on each dump line.
-v When parsing and printing, produce (slightly more) verbose output.
-vv Even more verbose output.
-vvv Even more verbose output.
-w Write the raw packets to file rather than parsing and printing them out.
-x Print each packet (minus its link level header) in hex.
-xx Print each packet, including its link level header, in hex.
-X Print each packet (minus its link level header) in hex and ASCII.
-XX Print each packet, including its link level header, in hex and ASCII.
-y Set the data link type to use while capturing packets to datalinktype.
-Z Drops privileges (if root) and changes user ID to user and the group ID to the primary group of user.
This is by no means the complete list of options for tcpdump and I recommend that the reader looks
over the man page to learn more. When we are capturing data for incident handling and forensic purposes, remember NOT to notify the source hosts of your captures. Use the (-n) flags where possible to ensure that you are not looking up IP addresses from a remote DNS Server. This can flag that you are investigating. In nearly all cases packets will flow over the screen far quicker than you can hope to read them. This is why we should always try to capture data to file even if we read it immediately (as noted using the r option from above). To capture data to a file, we should always specify the interface that we intend to capture on (this is using the -i flag). This makes our life a little easier by ensuring that tcpdump does not inadvertently start on an interface other than the one we want it to start on. In *NIX (Unix/Linux), interface names will take the form of one of the following:
• eth0, eth1, ... Ethernet
• ppp4, PPP
• le0. BSD Ethernet interface
• l0. The loopback interface.
Loopback is a special “virtualised” interface for the host to send packets to itself. In *NIX, you can
listen on the loopback. By issuing the following command, we can obtain a list of the interfaces
that tcpdump may use tcpdump –D. We also see this in Figure 1.
In this case, we have the options to use the Ethernet (eth0), Loopback(lo) or “any” option to listen
on all configured interfaces (any). The following command is a fairly standard way to initiate tcpdump:
tcpdump -nqtp
As noted, the -n option is essential in incident response work. Using the -p option stops the host entering promiscuous mode. This does not make a great deal of difference on modern switched (other than span ports) networks as the traffic is isolated with each host becoming its own collision domain. It does stop rogue broadcast traffic to some extent from being logged and unless your desire is
actually to capture all traffic (such as for an IDS), the -p option can be advantageous.
The -q and -t options are used to limit the volume of information returned (see above for details). Other options include using -v to -vvv to make a more verbose output. This will depend on your desired result (and disk space).
An interface in promiscuous mode will accept and store/display all packets that are received by the host. This is whether or not they were intended for this current host. That is, packets for any destination, not just the host you are on. When listening or “sniffing” whilst connected to either a hub
or a switch SPAN port, you will be able to listen for packets for any machine on a particular Ethernet segment when in promiscuous mode.
To listen for a selected port or protocol we can simply note what we are filtering for. As an example,
if we want to watch TCP 80 (usually HTTP or web traffic – although not always) we can specify
this using the terminology, “tcp port 80”. We can see this in the image below where the -X option
has been used to capture the payload as well (Figure 2)
If we now increase the SNAPLEN value (this is the maximum size of the packets we capture using
the -s option, we can see the full payload (Figure 3).
Ideally, we could capture all of the traffic to a file such as will result from the following command:
tcpdump -npXA “tcp port 80” -s 1500 -w www.pcap
In this instance, we are saving the complete web traffic for the host to a file called www.pcap. We can
then analyse this file using tcpdump, or we can access it with any other program that supports
the pcap format (such as WireShark or NGrep).
Remember...
It is possible to capture and save all packets and traffic to and from a host to a ‘pcap’ format file and
then to extract selected information at a later time.
BPF (Berkeley Packet Filter)
Berkeley Packet Filters (BPFs) allow you to create detailed fine grain filters in Tcpdump and other libpcap based programs. As BPFs run at the libpcap level, they are fast. They can be used to extract and filter date for display or to save far quicker than many other methods. BPFs can also be made into a file for processing if there are many options. Using Snort, the -F filter is used to load such a file – e.g.:
snort {some options} -F filter_file.bpf
Some of the primary terms used in creating BPFs are listed below:
• dst host [host name] – fi lter o n t he d estination host address.
• src host [host name] – filter on the source host address
• gateway [host name] – the packet used the selected host as a gateway.
• dst net [network identifier] – In this example, the destination address of the packet belongs to the selected network. This can be from /etc/networks or a network address.
• src net [network identifier] – In this instance, the source address of the packet is selected.
• dst port [Port Number] – In this example, the packet is either IP/TCP or IP/UDP with a destination
port value that we have selected.
• src port [Port Number] – Similarly, we can filter on packets based on a source port.
• tcp src [Port Number] – Or we can match only TCP packets whose source port is port.
• less [length] – Is used to select packets less than a certain length
These are a SMALL selection of the many options. In addition we can use logical operators to
refine our values:
• Negation (`!' or `not').
• Concatenation (`&&' or `and').
• Alternation (`||' or `or').
As with all tools, the best way to get to know these is to start using them. You will discover that
you can filter on IP or TCP options, and you can go right down to the individual flags and options in
a packet.
WireShark
With a captured file, we can use other tools to visualise and view the data. WireShark (Ethereal) is one such tool. WireShark (Figure 4) can capture and analyze network traffic. Configuring WireShark as a sniffer on the interface where an attack is originating from will allow you to capture information related to the router. This can be saved in the pcap format to be used as evidence or analyzed further offline.
Using the “Follow TCP Streams” feature of Wire- Shark, you can isolate individual communications
of interest. This feature even allows you to extract (carve) files that have been set to (or through) the
router. The PCAP format allows you to save the network capture and to replay or investigate this
later in a forensically sound manner.
WireShark can even use the information in the scan to build a set of Cisco access filters (see Figure
6). Filtering the pcap capture file will allow you to create specific filters for the attack that has been
recorded. This allows us to simplify the process of taking captures related to an incident and to making a specific set of filters to block known bad traffic. In the same way, we can capture a set of traffic associated with an unknown and even proprietary service and create allow filters to allow this traffic through our filtering router when blocking all other traffic. When conducting the analysis always:
• Document the collection procedure
• Record both the commands run during data gathering evidence and their results. When possible, send any digital data to a remote host or save it to external media.
• Many devices can be accessed using a command line. These systems can generally be
scripted to minimize the interaction required.
NGrep
NGrep is a network (pcap format) based search tool. Just like its file based compatriot, Grep, it is
used when searching for strings in the payload (or the header for that matter). Ngrep supports:
• Basic ASCII strings (e.g. "ASCII"),
• Regex (this is Regular Expressions such as GET.*php"), and
• Hex strings (e.g. 0029A)
The optimum results can be achieved using RegEX. These are a little arcane, so I will recommend a few tools to help use and learn these (there are also many good books and web sites for this). Hex also proves good if you know exactly what you are looking for, this can include using sections for captured data. Just like any good command line program, we have options. Some of these are listed below:
-i Ignore case for regular expression
-d Read data from a specific live interface
-I Read data from the specified pcap-file
-O Save matching packets to a pcap format file
-x Dump packet contents as hexadecimal and ASCII
-X Dump packet contents as hexadecimal and ASCII and then treat the matched expression
as a hexadecimal string
As a result, we can use NGrep as a filter and poor-man’s scanner to find traffic on a network.
It is best to start with capturing a file using Tcpdump into a PCAP format and then reading this
into NGrep to extract the information we seek. The reason, Tcpdump is fast and comprehensive
with few of the errors and issues associated with a richer platform (such as WireShark).
Step 1
The first stage is to capture all of the data we need into a PCAP file. For this we will use TCPDump.
To do this, we are going to capture all traffic (with the complete payload, -s 1500) to a file called capture.pcap.
tcpdump -nn -i eth0 -s 1500 -w capture.pcap
The command above and in the image below is what we have used for this. See the posts on Tcpdump for more details on the TCPDump command settings. In the Figure 7, I have used the date command before and after the capture so that you can see this has run for a number of minutes.
Note that we are not translating the IP address and protocols. This means that we have to learn what the protocols usually are and also we are less likely to make assumptions (for instance, TCP
80 is not always web traffic).
Step 2
Now that we have our PCAP capture, we can start an examination. It is always a good idea (in
incident response and forensic situations) to make a hash and a copy of the file. You can see how we have done this and validated the copy as well in the Figure 8.
Step 3
We can see the contents of the entire capture with NGrep by looking at our capture file.
ngrep -xX -I working.pcap
The command above and in the Figure 9 as output displays the packet capture in Hex and ASCII.
This is of course far too detailed and we need to filter the results to make any detail from this.
In this instance, the packet output displayed shows a HTTP GET request and associated data. The Proxy used is clear inside the packet as is the web site called (http:/ /www.msnbc. com). From the GET request data we can see that the user is making a search request (the page is – /search?).
Step 4
Now, say we are looking for a specific incident that occured using HTTP on TCP 80 (the default) to the server http://www.msnbc.com, we could first restrict the output to HTTP on TCP port 80 only (port 80).
ngrep -W byline -I working.pcap port 80
The output for this command is displayed in Figure 10. Notice that the output contains the details
of the HTML recieved from the web server. This is from any web server however and we wish to restrict this to a single host. We will also save this output to a file using the following command (where we have the server, www. montrealgazette.com):
ngrep -I working.pcap -O /tmp/traffic.pcap “GET” host www. montrealgazette.com and tcp port 80
The command and output is displayed in Figure 11.We have now saved the selected traffic we wish
to investigate. We can verify that we can read this using another PCAP compatible program (in the
instance in step 5 Tcpdump).
Step 5
Here we are validating that we have saved the data capture we wanted using TCPDump to read the
extract capture file (/tmp/traffic.pcap):
/tmp/traffic.pcap
The output is displayed in Figure 12 showing us that the packet capture has worked.
Here you can see we have a limited set of traffic to a specific host with a specific request (HTTP
GET). NGrep is a powerful tool and well worth taking the time to learn how to use. It can be far easier to have a small extract of a capture to work with or even to determine if a file contains information work investigating before you go into details.
NGrep can do this for us.
NGrep will either treat the search input as case sensitive (the default) or with the use of the -i option
will treat it as being case insensitive. This is, you can search with or without capitalisation in a
standard string. Of course, this does not matter if you are using a well formed RegEx. Tcpdump is
simpler and as a result makes a better capture engine then NGrep (not that you cannot do this with
it). As such, it is better to use Tcpdump to capture to a PCAP format file, and then to analyse
the saved data with NGREP. The ‚-I’ flag is used to select the PCAP format file that you want to read.
By default, NGrep will display the ASCII of a file and insert a . in place of unprintable characters. The -x parameter is used to print both Hex and ASCII in the output (this is recommended).
To Conclude…
Live data capture is an essential skill in required for both Incident Handlers as well as Forensic
practitioners and it is one that is becoming more, not less, important over time as we move towards networked and cloud based systems. This article has introduced a few tools that, although free, can be used together to create a powerful network forensics and incident response toolkit. Like all of these tools, the secret comes to practice. The more you use them, the better you well become at not only doing the basics, but in innovating and experimenting. This will allow you to extend the use of even these simple tools.
As we move to a world of cloud based systems, we are increasingly finding that we are required to capture and analyse data over networks. Once, analysing a disk drive was a source of incident
analysis and forensic material. Now we find that we cannot access the disk in an increasingly cloud based and remote world requiring the use of network captures. This is not a problem however. The
tools that are freely available in both Windows and Linux offer a means to capture traffic and carve out the evidence we require
As we move to a world of cloud based systems,we are increasingly finding that we are required to capture and analyse data over networks. To do this, we need to become familiar with the various tools that are available for these purposes. In this article, we look at a few of the more common free tools that will enable you to capture traffic for analysis within your organisation.
Once, analysing a disk drive was a source of incident analysis and forensic material. Now we find
that we cannot access the disk in an increasingly cloud based and remote world requiring the use
of network captures. This is not a problem however.The tools that are freely available in both Windows and Linux offer a means to capture traffic and carve out the evidence we require.
For this reason alone we would require the ability o0 capture and analyse data over networks, but
when we start to add all of th4e other benefits, we need to ask, why are you not already doing this?
Live Capture Procedures
In the event that a live network capture is warranted, we can easily run a network sniffer to capture
communication flows to and from the compromised or otherwise suspect system. There are many tools that can be used (such as WireShark, SNORT and others) to capture network traffic, but
tcpdump is generally the best capture program when set to capture raw traffic. The primary benefit
is that this tool will minimize any performance issues while allowing the data to be captured in a
format that can be loaded into more advanced protocol analysers for review.
That stated there are only minor differences between Tcpdump and Windump and most of what
you can do in one is the same on the other (some flags do vary).
Tcpdump
Tcpdump uses the libpcap library. This can capture traffic from a file or an interface. This means that
you can save a capture and analyse it later. This is a great aid in incident response and network forensics. With a file such as, capture.pcap, we can read and display the data using the -r flag. For instance:
tcpdump -r capture.pcap
Will replay the data saved in the file, capture. pcap. By default, this will display the output to the screen. In reality, the data is sent to SDTOut (Standard Out), but for most purposes the console
and SDTOut are one and the same thing. Using BPF (Berkley Packet Filters), you can also
restrict the output – both collected and saved. In this way, you can collect all data to and from
a host and then strip selected ports (or services) from this saved file. Some of the options that apply
to tcpdump include (quoted with alterations from the Redhat tcpdump MAN file):
-A Print each packet (minus its link level header) in ASCII.
-c Exit after receiving a set number of packets (defined after c).
-C Before writing a raw packet to a savefile, check whether the file is currently larger than
a given file_size. Where this is the case, close the current savefile and open a new one.
-d Dump the compiled packet-matching code in a human readable form to standard output
and stop.
-dd Dump packet-matching code as a C program fragment.
-ddd Dump packet-matching code as decimal numbers (preceded with a count).
-D Print the list of the network interfaces available on the system and on which tcpdump can
capture packets.
This can be useful on systems that do not support the use of the ifconfig -a command.
-e Print the link-level header on each dump line.
-F Use file as input for the filter expression. An additional expression given on the command
line is ignored.
-i Listen on interface. This specifies the system interface to listen for traffic on.
-L List the known data link types for the interface and exit.
-n Don’t convert host addresses to names.This can be used to avoid DNS lookups.
-nn Don’t convert protocol and port numbers etc. to names either.
-N Don’t print domain name qualification of host names.
-p Don’t put the interface into promiscuous mode.
-q Quick (quiet?) output. Print less protocol information so output lines are shorter.
-r Read packets from file that has been created using the “-w” option.
-S Print absolute, rather than relative, TCP sequence numbers.
-s Snarf snaplen bytes of data from each packet rather than the default of 68 bytes.
-T Force packets selected by “expression” to be interpreted the specified type.
-t Don’t print a timestamp on each dump line.
-tt Print an unformatted timestamp on each dump line.
-ttt Print a delta (in micro-seconds) between current and previous line on each dump line.
-tttt Print a timestamp in default format proceeded by date on each dump line.
-v When parsing and printing, produce (slightly more) verbose output.
-vv Even more verbose output.
-vvv Even more verbose output.
-w Write the raw packets to file rather than parsing and printing them out.
-x Print each packet (minus its link level header) in hex.
-xx Print each packet, including its link level header, in hex.
-X Print each packet (minus its link level header) in hex and ASCII.
-XX Print each packet, including its link level header, in hex and ASCII.
-y Set the data link type to use while capturing packets to datalinktype.
-Z Drops privileges (if root) and changes user ID to user and the group ID to the primary group of user.
This is by no means the complete list of options for tcpdump and I recommend that the reader looks
over the man page to learn more. When we are capturing data for incident handling and forensic purposes, remember NOT to notify the source hosts of your captures. Use the (-n) flags where possible to ensure that you are not looking up IP addresses from a remote DNS Server. This can flag that you are investigating. In nearly all cases packets will flow over the screen far quicker than you can hope to read them. This is why we should always try to capture data to file even if we read it immediately (as noted using the r option from above). To capture data to a file, we should always specify the interface that we intend to capture on (this is using the -i flag). This makes our life a little easier by ensuring that tcpdump does not inadvertently start on an interface other than the one we want it to start on. In *NIX (Unix/Linux), interface names will take the form of one of the following:
• eth0, eth1, ... Ethernet
• ppp4, PPP
• le0. BSD Ethernet interface
• l0. The loopback interface.
Loopback is a special “virtualised” interface for the host to send packets to itself. In *NIX, you can
listen on the loopback. By issuing the following command, we can obtain a list of the interfaces
that tcpdump may use tcpdump –D. We also see this in Figure 1.
In this case, we have the options to use the Ethernet (eth0), Loopback(lo) or “any” option to listen
on all configured interfaces (any). The following command is a fairly standard way to initiate tcpdump:
tcpdump -nqtp
As noted, the -n option is essential in incident response work. Using the -p option stops the host entering promiscuous mode. This does not make a great deal of difference on modern switched (other than span ports) networks as the traffic is isolated with each host becoming its own collision domain. It does stop rogue broadcast traffic to some extent from being logged and unless your desire is
actually to capture all traffic (such as for an IDS), the -p option can be advantageous.
The -q and -t options are used to limit the volume of information returned (see above for details). Other options include using -v to -vvv to make a more verbose output. This will depend on your desired result (and disk space).
An interface in promiscuous mode will accept and store/display all packets that are received by the host. This is whether or not they were intended for this current host. That is, packets for any destination, not just the host you are on. When listening or “sniffing” whilst connected to either a hub
or a switch SPAN port, you will be able to listen for packets for any machine on a particular Ethernet segment when in promiscuous mode.
To listen for a selected port or protocol we can simply note what we are filtering for. As an example,
if we want to watch TCP 80 (usually HTTP or web traffic – although not always) we can specify
this using the terminology, “tcp port 80”. We can see this in the image below where the -X option
has been used to capture the payload as well (Figure 2)
If we now increase the SNAPLEN value (this is the maximum size of the packets we capture using
the -s option, we can see the full payload (Figure 3).
Ideally, we could capture all of the traffic to a file such as will result from the following command:
tcpdump -npXA “tcp port 80” -s 1500 -w www.pcap
In this instance, we are saving the complete web traffic for the host to a file called www.pcap. We can
then analyse this file using tcpdump, or we can access it with any other program that supports
the pcap format (such as WireShark or NGrep).
Remember...
It is possible to capture and save all packets and traffic to and from a host to a ‘pcap’ format file and
then to extract selected information at a later time.
BPF (Berkeley Packet Filter)
Berkeley Packet Filters (BPFs) allow you to create detailed fine grain filters in Tcpdump and other libpcap based programs. As BPFs run at the libpcap level, they are fast. They can be used to extract and filter date for display or to save far quicker than many other methods. BPFs can also be made into a file for processing if there are many options. Using Snort, the -F filter is used to load such a file – e.g.:
snort {some options} -F filter_file.bpf
Some of the primary terms used in creating BPFs are listed below:
• dst host [host name] – fi lter o n t he d estination host address.
• src host [host name] – filter on the source host address
• gateway [host name] – the packet used the selected host as a gateway.
• dst net [network identifier] – In this example, the destination address of the packet belongs to the selected network. This can be from /etc/networks or a network address.
• src net [network identifier] – In this instance, the source address of the packet is selected.
• dst port [Port Number] – In this example, the packet is either IP/TCP or IP/UDP with a destination
port value that we have selected.
• src port [Port Number] – Similarly, we can filter on packets based on a source port.
• tcp src [Port Number] – Or we can match only TCP packets whose source port is port.
• less [length] – Is used to select packets less than a certain length
These are a SMALL selection of the many options. In addition we can use logical operators to
refine our values:
• Negation (`!' or `not').
• Concatenation (`&&' or `and').
• Alternation (`||' or `or').
As with all tools, the best way to get to know these is to start using them. You will discover that
you can filter on IP or TCP options, and you can go right down to the individual flags and options in
a packet.
WireShark
With a captured file, we can use other tools to visualise and view the data. WireShark (Ethereal) is one such tool. WireShark (Figure 4) can capture and analyze network traffic. Configuring WireShark as a sniffer on the interface where an attack is originating from will allow you to capture information related to the router. This can be saved in the pcap format to be used as evidence or analyzed further offline.
Using the “Follow TCP Streams” feature of Wire- Shark, you can isolate individual communications
of interest. This feature even allows you to extract (carve) files that have been set to (or through) the
router. The PCAP format allows you to save the network capture and to replay or investigate this
later in a forensically sound manner.
WireShark can even use the information in the scan to build a set of Cisco access filters (see Figure
6). Filtering the pcap capture file will allow you to create specific filters for the attack that has been
recorded. This allows us to simplify the process of taking captures related to an incident and to making a specific set of filters to block known bad traffic. In the same way, we can capture a set of traffic associated with an unknown and even proprietary service and create allow filters to allow this traffic through our filtering router when blocking all other traffic. When conducting the analysis always:
• Document the collection procedure
• Record both the commands run during data gathering evidence and their results. When possible, send any digital data to a remote host or save it to external media.
• Many devices can be accessed using a command line. These systems can generally be
scripted to minimize the interaction required.
NGrep
NGrep is a network (pcap format) based search tool. Just like its file based compatriot, Grep, it is
used when searching for strings in the payload (or the header for that matter). Ngrep supports:
• Basic ASCII strings (e.g. "ASCII"),
• Regex (this is Regular Expressions such as GET.*php"), and
• Hex strings (e.g. 0029A)
The optimum results can be achieved using RegEX. These are a little arcane, so I will recommend a few tools to help use and learn these (there are also many good books and web sites for this). Hex also proves good if you know exactly what you are looking for, this can include using sections for captured data. Just like any good command line program, we have options. Some of these are listed below:
-i Ignore case for regular expression
-d Read data from a specific live interface
-I Read data from the specified pcap-file
-O Save matching packets to a pcap format file
-x Dump packet contents as hexadecimal and ASCII
-X Dump packet contents as hexadecimal and ASCII and then treat the matched expression
as a hexadecimal string
As a result, we can use NGrep as a filter and poor-man’s scanner to find traffic on a network.
It is best to start with capturing a file using Tcpdump into a PCAP format and then reading this
into NGrep to extract the information we seek. The reason, Tcpdump is fast and comprehensive
with few of the errors and issues associated with a richer platform (such as WireShark).
Step 1
The first stage is to capture all of the data we need into a PCAP file. For this we will use TCPDump.
To do this, we are going to capture all traffic (with the complete payload, -s 1500) to a file called capture.pcap.
tcpdump -nn -i eth0 -s 1500 -w capture.pcap
The command above and in the image below is what we have used for this. See the posts on Tcpdump for more details on the TCPDump command settings. In the Figure 7, I have used the date command before and after the capture so that you can see this has run for a number of minutes.
Note that we are not translating the IP address and protocols. This means that we have to learn what the protocols usually are and also we are less likely to make assumptions (for instance, TCP
80 is not always web traffic).
Step 2
Now that we have our PCAP capture, we can start an examination. It is always a good idea (in
incident response and forensic situations) to make a hash and a copy of the file. You can see how we have done this and validated the copy as well in the Figure 8.
Step 3
We can see the contents of the entire capture with NGrep by looking at our capture file.
ngrep -xX -I working.pcap
The command above and in the Figure 9 as output displays the packet capture in Hex and ASCII.
This is of course far too detailed and we need to filter the results to make any detail from this.
In this instance, the packet output displayed shows a HTTP GET request and associated data. The Proxy used is clear inside the packet as is the web site called (http:/ /www.msnbc. com). From the GET request data we can see that the user is making a search request (the page is – /search?).
Step 4
Now, say we are looking for a specific incident that occured using HTTP on TCP 80 (the default) to the server http://www.msnbc.com, we could first restrict the output to HTTP on TCP port 80 only (port 80).
ngrep -W byline -I working.pcap port 80
The output for this command is displayed in Figure 10. Notice that the output contains the details
of the HTML recieved from the web server. This is from any web server however and we wish to restrict this to a single host. We will also save this output to a file using the following command (where we have the server, www. montrealgazette.com):
ngrep -I working.pcap -O /tmp/traffic.pcap “GET” host www. montrealgazette.com and tcp port 80
The command and output is displayed in Figure 11.We have now saved the selected traffic we wish
to investigate. We can verify that we can read this using another PCAP compatible program (in the
instance in step 5 Tcpdump).
Step 5
Here we are validating that we have saved the data capture we wanted using TCPDump to read the
extract capture file (/tmp/traffic.pcap):
/tmp/traffic.pcap
The output is displayed in Figure 12 showing us that the packet capture has worked.
Here you can see we have a limited set of traffic to a specific host with a specific request (HTTP
GET). NGrep is a powerful tool and well worth taking the time to learn how to use. It can be far easier to have a small extract of a capture to work with or even to determine if a file contains information work investigating before you go into details.
NGrep can do this for us.
NGrep will either treat the search input as case sensitive (the default) or with the use of the -i option
will treat it as being case insensitive. This is, you can search with or without capitalisation in a
standard string. Of course, this does not matter if you are using a well formed RegEx. Tcpdump is
simpler and as a result makes a better capture engine then NGrep (not that you cannot do this with
it). As such, it is better to use Tcpdump to capture to a PCAP format file, and then to analyse
the saved data with NGREP. The ‚-I’ flag is used to select the PCAP format file that you want to read.
By default, NGrep will display the ASCII of a file and insert a . in place of unprintable characters. The -x parameter is used to print both Hex and ASCII in the output (this is recommended).
To Conclude…
Live data capture is an essential skill in required for both Incident Handlers as well as Forensic
practitioners and it is one that is becoming more, not less, important over time as we move towards networked and cloud based systems. This article has introduced a few tools that, although free, can be used together to create a powerful network forensics and incident response toolkit. Like all of these tools, the secret comes to practice. The more you use them, the better you well become at not only doing the basics, but in innovating and experimenting. This will allow you to extend the use of even these simple tools.