Skip to main content

Integrating Apache Traffic Server

This document provides instructions on configuring Apache Traffic Server (ATS) to use Backtrace's Invoker as a crash log helper. The workflow upon ATS crash is as follows:

  1. ATS launches the invoker on startup, which monitors the traffic_server process.
  2. Upon the traffic_server crash, the invoker launches ptrace via a script that provides metadata.
  3. This script, in turn, submits the crash file generated by ptrace to the coroner object store.

What You'll Need

  • A Backtrace account (log in or sign up for a free trial license).
  • Your subdomain name (used to connect to your Backtrace instance). For example,
  • A Backtrace project and a submission token.
  • Apache Traffic Server (tested with version 6.2.0)
  • Coroner Server running and accessible from the ATS machine. (See Coroner Server installation)
  • Coroner Client package on the ATS machine. (See Coroner Client Installation)
  • Backtrace Ptrace package installed on the ATS machine. (See Ptrace Installation)
  • Backtrace Invoker package installed on the ATS machine: $ sudo apt-get install backtrace-invoker.
  • (Optional but recommended) - Install the morgue command-line queryingtool: (See: Morgue Installation)

Set Up Helper Scripts

exec /opt/backtrace/bin/invoker -t "/home/ats/ %p" $@

This script calls ptrace with any metadata you wish to gather and then submits it to the coroner. Here, we're extracting the version number using traffic_ctl, but feel free to add any additional metadata that you need.


## Change the following as needed


# Add commands to gather any metadata that you wish to pass to the ptrace invocation
VERSION=`/opt/ts/bin/traffic_ctl metric get proxy.node.version.manager.short|cut -d ' ' -f 2`

mkdir -p ${DM}/
ptrace --kv="version:$VERSION" $1 -O ${DM}/ats
if test "$?" == "0"; then
coroner -c $CF put -u ${PROJECT} ${TOKEN} ${DM}/*.btt


Ensure that your traffic_server processes can run these scripts and have permission to write to the destination folder specified in (referred to by the variable DM).

$ chmod +x
$ chmod +x

Configure ATS Crash Log Helper

To configure the ATS Crash Log Helper, add or edit the following line in /etc/trafficserver/records.config:

CONFIG proxy.config.crash_log_helper STRING /home/ats/

Verify Invoker Launch

After making the configuration changes, restart ATS and verify that the invoker is running by executing the following commands:

$ sudo /bin/trafficserver restart
$ ps aux | grep invoker
nobody 11616 0.0 0.1 4336 1396 ? T 16:43 0:00 /opt/backtrace/bin/invoker -t /home/ats/ %p --syslog --wait --host x86_64-unknown-linux-gnu


To test the setup, send a SIGABRT signal to traffic_server:

$ sudo kill -SIGABRT

Then, verify via the morgue tool that the number of crashes for the project has increased:

morgue list myproj

Occurrences: 1 (100.00%)


Launching Manually

To check for any errors, you can manually launch

$ ./ 2291
7750a16bfbb8667ca39de9a568c86897944a476143074275631cd13c842ce74d => 607dcbc50c634580951c0fdf832b2fc9

If there are any errors, continue with the troubleshooting steps below to verify the correct installation of Backtrace components and proper permissions.

Verifying ptrace Installation

Running /opt/backtrace/bin/ptrace on a process you own should generate a .btt file:

$ ps
2291 pts/7 00:00:00 bash
11709 pts/7 00:00:00 ps

$ /opt/backtrace/bin/ptrace 2291

$ ls *.btt

If the file is not generated, verify your ptrace installation. Refer to Ptrace Installation for assistance.

Verifying Coroner Installation

To confirm that you can send snapshots to the server using coroner, perform the following steps:

$ coroner -c put project1 project1_token bash.2291.1461078102.btt
7750a16bfbb8667ca39de9a568c86897944a476143074275631cd13c842ce74d => 607dcbc50c634580951c0fdf832b2fc9

If you encounter issues, verify your coroner client installation. See Coroner Client Installation for more information.

Launching Manually

Try launching manually to ensure there are no errors:

$ ./ 2291
7750a16bfbb8667ca39de9a568c86897944a476143074275631cd13c842ce74d => 607dcbc50c634580951c0fdf832b2fc9

Ensuring ATS Has Permission to Run

Make sure that ATS has the necessary permissions to execute

$ chmod +x /path/to/

Verifying Invocation of and Generation of .btt File

You can verify if is being invoked and generating the .btt file in the desired location by commenting out the last 3 lines of