README - mozsearch

Logalloc is a replace-malloc library for Firefox (see

memory/build/replace_malloc.h) that dumps a log of memory allocations to a

given file descriptor or file name. That log can then be replayed against

Firefox's default memory allocator independently or through another

replace-malloc library, allowing the testing of other allocators under the

exact same workload.

To get an allocation log the following environment variable when starting

Firefox:

  MALLOC_LOG=/path/to/log-file

or

  MALLOC_LOG=number

When MALLOC_LOG is a number below 10000, it is considered as a file

descriptor number that is fed to Firefox when it is started. Otherwise,

it is considered as a file name.

As those allocation logs can grow large quite quickly, it can be useful

to pipe the output to a compression tool.

MALLOC_LOG=1 would send to Firefox's stdout, MALLOC_LOG=2 would send to

its stderr. Since in both cases that could be mixed with other output

from Firefox, it is usually better to use another file descriptor

by shell redirections, such as:

  MALLOC_LOG=3 firefox 3>&1 1>&2 | gzip -c > log.gz

(3>&1 copies the `| gzip` pipe file descriptor to file descriptor #3, 1>&2

then copies stderr to stdout. This leads to: fd1 and fd2 sending to stderr

of the parent process (the shell), and fd3 sending to gzip.)

Each line of the allocations log is formatted as follows:

  <pid> <tid> <function>([<args>])[=<result>]

where <args> is a comma separated list of values. The number of <args> and

the presence of <result> depend on the <function>.

Example log:

  18545 18545 malloc(32)=0x7f90495120e0

  18545 18545 calloc(1,148)=0x7f9049537480

  18545 18545 realloc(0x7f90495120e0,64)=0x7f9049536680

  18545 18545 posix_memalign(256,240)=0x7f9049583300

  18545 18545 jemalloc_stats()

  18545 18545 free(0x7f9049536680)

This log can be replayed with the logalloc-replay tool in

memory/replace/logalloc/replay. However, as the goal of that tool is to

reproduce the recorded memory allocations, it needs to avoid as much as

possible doing its own allocations for bookkeeping. Reading the logs as

they are would require data structures and memory allocations. As a

consequence, the logs need to be preprocessed beforehand.

The logalloc_munge.py script is responsible for that preprocessing. It simply

takes a raw log on its stdin, and outputs the preprocessed log on its stdout.

It replaces pointer addresses with indexes the logalloc-replay tool can use

in a large (almost) linear array of allocation tracking slots (prefixed with

'#'). It also replaces the pids with numbers starting from 1 (such as the

first seen pid number is 1, the second is 2, etc.).

The above example log would become the following, once preprocessed:

  1 1 malloc(32)=#1

  1 1 calloc(1,148)=#2

  1 1 realloc(#1,64)=#1

  1 1 posix_memalign(256,240)=#3

  1 1 jemalloc_stats()

  1 1 free(#1)

The logalloc-replay tool then takes the preprocessed log on its stdin and

replays the allocations printed there, but will only replay those with the

same process id as the first line (which normally is 1).

As the log files are simple text files, though, it is easy to separate out

the different processes log with e.g. grep, and feed the separate processes

logs to logalloc-replay.

The logalloc-replay program won't output anything unless jemalloc_stats

records appears in the log. You can expect those to be recorded when going

to about:memory in Firefox, but they can also be added after preprocessing.

Here is an example of what one can do:

  gunzip -c log.gz | python logalloc_munge.py | \

  awk '$1 == "2" { print $0 } !(NR % 10000) { print "2 1 jemalloc_stats()" }' | \

    ./logalloc-replay

The above command replays the allocations of process #2, with some stats

output every 10000 records.

The logalloc-replay tool itself being hooked with replace-malloc, it is possible

to set LD_PRELOAD/DYLD_INSERT_LIBRARIES/MOZ_REPLACE_MALLOC_LIB and replay a log

through a different allocator. For example:

  LD_PRELOAD=libreplace_jemalloc.so logalloc-replay < log

Will replay the log against jemalloc4 (which is, as of writing, what

libreplace_jemalloc.so contains).