Apache Module mod_firehose

Enabling a Firehose ¶

To enable the module, it should be compiled and loaded in to your running Apache configuration, and the directives below used to record the data you are interested in.

It is possible to record both incoming and outgoing data to the same filename if desired, as the direction of flow is recorded within each fragment.

It is possible to write to both normal files and fifos (pipes). In the case of fifos, mod_firehose ensures that the packet size is no larger than PIPE_BUF to ensure writes are atomic.

If a pipe is being used, something must be reading from the pipe before httpd is started for the pipe to be successfully opened for write. If the request to open the pipe fails, mod_firehose will silently stand down and not record anything, and the server will keep running as normal.

By default, all attempts to write will block the server. If the webserver has been built against APR v2.0 or later, and an optional "nonblock" parameter is specified all file writes will be non blocking, and buffer overflows will cause debugging data to be lost. In this case it is possible to prioritise the running of the server over the recording of firehose data.

Stream Format ¶

The server typically serves multiple connections simultaneously, and as a result requests and responses need to be multiplexed before being written to the firehose.

The fragment format is designed as clear text, so that a firehose can be opened with and inspected by a normal text editor. Alternatively, the firehose tool can be used to demultiplex the firehose back into individual requests or connections.

The size of the multiplexed fragments is governed by PIPE_BUF, the maximum size of write the system is prepared to perform atomically. By keeping the multiplexed fragments below PIPE_BUF in size, the module guarantees that data from different fragments does not interleave. The size of PIPE_BUF varies on different operating systems.

The BNF for the fragment format is as follows:

 stream = 0*(fragment)

 fragment = header CRLF body CRLF

 header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count

 length = <up to 16 byte hex fragment length>
 timestamp = <up to 16 byte hex timestamp microseconds since 1970>
 request = "<"
 response = ">"
 uuid = <formatted uuid of the connection>
 count = <hex fragment number in the connection>

 body = <the binary content of the fragment>

 SPC = <a single space>
 CRLF = <a carriage return, followed by a line feed>

All fragments for a connection or a request will share the same UUID, depending on whether connections or requests are being recorded. If connections are being recorded, multiple requests may appear within a connection. A fragment with a zero length indicates the end of the connection.

Fragments may go missing or be dropped if the process reading the fragments is too slow. If this happens, gaps will exist in the connection counter numbering. A warning will be logged in the error log to indicate the UUID and counter of the dropped fragment, so it can be confirmed the fragment was dropped.

It is possible that the terminating empty fragment may not appear, caused by the httpd process crashing, or being terminated ungracefully. The terminating fragment may be dropped if the process reading the fragments is not fast enough.