Tag Archives: dtrace

Installing and configuring DTRACE on Oracle Linux SPARC

DTRACE is one of the killer features of Solaris, and allows you to programmatically monitor system statistics and diagnose performance issues.  See https://github.com/opendtrace/toolkit for toolkit scripts so you do not have to write your own.

Dtrace is not shipped with the install media. You need to manually download the rpms from




Dtrace is very kernel version dependent. Do not yum update your kernel without checking that dtrace is available for that release or you will have problems!

You can use yum to install the rpms

[root@host-8-161 sfw]# yum localinstall dtrace*
Loaded plugins: downloadonly, ulninfo
Setting up Local Package Process
Examining dtrace-utils-0.6.0-3.el6.sparc64.rpm: dtrace-utils-0.6.0-3.el6.sparc64
Marking dtrace-utils-0.6.0-3.el6.sparc64.rpm to be installed
Examining dtrace-utils-devel-0.6.0-3.el6.sparc64.rpm: dtrace-utils-devel-0.6.0-3.el6.sparc64
Marking dtrace-utils-devel-0.6.0-3.el6.sparc64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package dtrace-utils.sparc64 0:0.6.0-3.el6 will be installed
---> Package dtrace-utils-devel.sparc64 0:0.6.0-3.el6 will be installed
--> Processing Dependency: libdtrace-ctf-devel > 0.4.0 for package: dtrace-utils-devel-0.6.0-3.el6.sparc64
--> Running transaction check
---> Package libdtrace-ctf-devel.sparc64 0:0.5.0-3.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

 Package Arch Version Repository Size
 dtrace-utils sparc64 0.6.0-3.el6 /dtrace-utils-0.6.0-3.el6.sparc64 766 k
 sparc64 0.6.0-3.el6 /dtrace-utils-devel-0.6.0-3.el6.sparc64 77 k
Installing for dependencies:
 sparc64 0.5.0-3.el6 public_ol6_latest 15 k

Transaction Summary
Install 2 Packages (+1 Dependent package)

Total size: 857 k
Total download size: 15 k
Installed size: 877 k
Is this ok [y/N]: y
Downloading Packages:
libdtrace-ctf-devel-0.5.0-3.el6.sparc64.rpm | 15 kB 00:00 
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Public key for libdtrace-ctf-devel-0.5.0-3.el6.sparc64.rpm is not installed
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
 Userid : "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>"
 Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03
 Package : 6:oraclelinux-release-6Server-7.0.8.sparc64 (@anaconda-OracleLinuxServer-201705232044.sparc64/6.7)
 From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
 Installing : dtrace-utils-0.6.0-3.el6.sparc64 1/3 
 Installing : libdtrace-ctf-devel-0.5.0-3.el6.sparc64 2/3 
 Installing : dtrace-utils-devel-0.6.0-3.el6.sparc64 3/3 
 Verifying : dtrace-utils-devel-0.6.0-3.el6.sparc64 1/3 
 Verifying : libdtrace-ctf-devel-0.5.0-3.el6.sparc64 2/3 
 Verifying : dtrace-utils-0.6.0-3.el6.sparc64 3/3

 dtrace-utils.sparc64 0:0.6.0-3.el6 dtrace-utils-devel.sparc64 0:0.6.0-3.el6

Dependency Installed:
 libdtrace-ctf-devel.sparc64 0:0.5.0-3.el6




At this point when you run dtrace it doesn’t show anything useful and has no probes available.

[root@host-8-160 sfw]# dtrace -l
dtrace: module license 'CDDL' taints kernel.
Disabling lock debugging due to kernel taint
 1  dtrace                   BEGIN
 2  dtrace                   END
 3  dtrace                   ERROR


You need to manually load the kernel modules for the probes and providers you want to use.  There is a list of providers in the the Oracle Linux Dtrace Tutorial manual and the Oracle Linux Dtrace Guide

A summary of what is available at the time of writing (June 2017) is below.

Provider Kernel Module Description
dtrace dtrace Provides probes that relate to DTrace itself, such as BEGIN, ERROR, and END. You can use these probes to initialize DTrace’s state before tracing begins, process its state after tracing has completed, and handle unexpected execution errors in other probes.
fasttrap fasttrap Supports user-space tracing of DTrace-enabled applications.
io sdt Provides probes that relate to data input and output. The io provider enables quick exploration of behavior observed through I/O monitoring tools such as iostat.
proc sdt Provides probes for monitoring process creation and termination, LWP creation and termination, execution of new programs, and signal handling.
profile profile Provides probes associated with an interrupt that fires at a fixed, specified time interval. These probes are associated with the asynchronous interrupt event rather than with any particular point of execution. You can use these probes to sample some aspect of a system’s state.
sched sdt Provides probes related to CPU scheduling. Because CPUs are the one resource that all threads must consume, the sched provider is very useful for understanding systemic behavior.
syscall systrace Provides probes at the entry to and return from every system call. Because system calls are the primary interface between user-level applications and the operating system kernel, these probes can offer you an insight into the interaction between applications and the system.

You can manually load the probes

[root@host-8-160 log]# modprobe -a dtrace profile systrace sdt dt_test fasttrap

However, you may want to write  startup script to automatically load the probes at boot time if the dtrace device exists.

[root@host-8-160 sfw]# cat /etc/sysconfig/modules/dtrace.modules

if [ ! -c /dev/dtrace/dtrace ] ; then
         exec /sbin/modprobe -a dtrace profile systrace sdt dt_test

[root@host-8-160 sfw]# chmod 755 /etc/sysconfig/modules/dtrace.modules


Once the module has been loaded into the kernel, you can list all probes using

[root@host-8-160 sfw]# dtrace -l

or for just a single provider

[root@host-8-160 etc]# dtrace -l -P io
 266 io       vmlinux   end_bio_bh_io_sync done
 267 io       vmlinux   _submit_bh start
 269 io       vmlinux   __wait_on_buffer wait-start
 270 io       vmlinux   __wait_on_buffer wait-done

You may also want to look at the information in the manual about setting the permissions on the dtrace helper device to allow code that runs as a user other than root to be recorded.