Nitpicker

WARNING! Doku is still BETA!

Nitpicker

Nitpicker is designed to collect Ipflow informations

What is an Ipflow?

An Ipflow is an connection between two end points. Following informations are stored inside a flow:

Why?

The major reason for nitpicker is, to generate comprehensible traffic logfiles and to run 24/7/365.

How?

It keeps track about past and current traffic and stores it into flowfiles in the most sophisticated way.

"Design" or "How it works"

It listens to the specified network interface. If nitpicker recieves a packet (e.g. via the BPF) it try's to find the corresponding flow. If succesfull it increases the flowcounters (e.g. Source to Destination Bytes and Packets, Destination to Source Bytes and Packets and the duration). If not it generates a new flow in timequeue.

Design of the "TimeQueue" or "WTH are the queues for?"

First the flowes are stored in a timequeue which is accessable via a hash function. Flows are kept for FLOWTIMEOUT seconds (per default 600sec.) in the timequeue until they move to the savequeue. The savequeue grows up to MAXFLOWSPERFILE (per default 10000) until its dumped into a flowfile.

The Flowfile

Nitpicker saves the accounted data in flowfiles. In thouse files the accounting data is subsequent accessible e.g. for evaluation reasons. The filmename format is: flow.<unixtimestamp>.u At the end of a dumping period, the .u files will be renamed to *.s, and new .u files will be created

Structure of a flowfile


A flowfile begins with the header followed by the flows until the end of the file.

Flowfileheader (256 byte)

Magic Number: 32bit

The Magic Number - actual { 'N', 'P', 'F', 'F' }

Major Version: 8bit

Major Version - actual '4'

Minor Version: 8bit

Minor Version - actual '0'

Starttime: 32bit

Date of the first flow (seconds since the Epoch (00:00:00 UTC, January 1, 1970; see time(3)))

Stoptime: 32bit

Date of the last flow (seconds since the Epoch (00:00:00 UTC, January 1, 1970; see time(3)))

Bytes: 64bit

Total bytes of all flows in file 0

Flows: 32bit

Overall number of flows in file

Packets: 32bit

Overall number of packets of all flows in file

Unused: 1808bit

Spare for further use...
 

Flow (33 byte)

Protocol Type: 8bit

Protocol Type ('1' = ICMP, '6' = TCP, '17' = UDP...)

Startoffset: 16bit

Startoffset of the flow (seconds since the Epoch (00:00:00 UTC, January 1, 1970; see time(3)))

Duration: 16bit

Duration of the flow in seconds

SDPackets: 32bit

Overall packets from source to destination

SDBytes: 32bit

Overall bytes from source to destination

DSPackets: 32bit

Overall packets from destination to source

DSBytes: 32bit

Overall bytes from destination to source

Src. Ip: 32bit

Source IPv4 Address

Dst. Ip: 32bit

Destination IPv4 Address

Protocol Specific: 32bit

Protocol specific information (see below)

TCP and UDP protocol specific information (4 byte)

Src. Port: 16bit

TPC or UDP Source Port

Dst. Port: 16bit

TCP or UDP Destination Port

ICMP protocol specific information (4 byte)

Type: 16bit

ICMP Type

Code: 16bit

ICMP Code

Running Nitpicker

The Tools or "Possibilities to interact with the flowfiles"

*We should place here a link for each tool to get in detail how they work*

What?

	#include "brain.h"
		
	int
	main()
	{
		short readcount=0;
	
		while !(understand_text)
		{
			read_again("http://www.nitpicker.de/new/");
			readcount++;
			
			if (readcount > 2)
				exit(1); // No hope left...
					
			if !(like_nitpicker) // Impossible
				exit(1); 
		}
			if (yourOS_is_not_able_to_run_nitpicker)  
				install_true_os(FreeBSD);
		
		install_nitpicker();
		return(0);
	}
Nitpicker accesses the raw ethernet frames and reads the header informations.
Nitpicker will write the flows into a file when there are n flows in the savequeue. They are moved there af ter n seconds passed since the flowstart (This prevents never-ending flows like news to be written too)

Last changes:

09.06.2003: M. Steiner, Ch. Jachmann (1st Release)
19.06.2003: Soeren Todt (Rumfummeling)
10.09.2003: M.Steiner (Added flowimages and "The Flowfile" subsection)

We are still waiting for Humbert to finish the documentation