DTrace vs SystemTap, Redux

Share via Twitter Share via Facebook Share via Linkedin Share via Reddit

Three years ago come August, O’Reilly’s Nat Torkington, interviewing Sun’s Jonathan Schwartz, pressed the CEO on the issues of patents generally and DTrace patents specifically.

Torkington’s question? “So if the Linux kernel were to implement DTrace, Sun wouldn’t employ the patents against them?”

Schwartz’ answer? “Knock yourself out.”

That was 2005. Fast forward to 2008. As one of the DTrace engineers has noted, Paul Fox is taking Schwartz up on that challenge.

As the FreeBSD and OS X engineers can tell you, the task of porting DTrace to a platform other than Solaris is non-trivial in the extreme. On the other hand, as they’ve both been successful (or in progress, in the case of FreeBSD) – with the kind assistance of the DTrace team – it clearly can be done. Where there’s a will, there’s a way and so on.

Making Fox’s work more compelling is the code. Or rather, the fact that there is code. That alone elevates this from an interesting but ambitious idea to potentially fascinating project.

Merely potentially because the appetite for DTrace within the Linux ecosystem would be, how do I put this, uncertain. Some view the technology as marketing coolaid, some argue it’s far ahead, and others, well, they’re undecided.

Undecided, at least in part, because there’s already a project within the Linux ecosystem to provide DTrace like functionality. SystemTap is a project we’ve discussed in this space before: a little over two years ago, to be precise. At that time, I said the following:

Anyhow, still undetermined for SystemTap is when/if a.) SystemTap makes its way into mainstream commercial and non-commercial distributions and b.) the required kernel modules, KProbes and RELAYFS (both of which I had to add to my kernel, necessitating two separate compiles) become standard. Until that happens, DTrace has an advantage over its Linux counterpart.

How does SystemTap score on those metrics? Well, on the Ubuntu Hardy machine I’m writing this on now, SystemTap is available for install. Which is good. Less so is the fact that kernel developers called the technology, on list, “klunky and prone to break in unexpected ways” (see Frank Ch. Eigler’s response to that charge here) just four days ago. As a related aside, I was interested to read that with respect to user space probing – an ability I’d believed to have been added – is not there yet (“We’re finally getting very close in this“) two years later.

What does this tell us? That the problem that DTrace and SystemTap both are trying to solve is hard; exceedingly so. Further, that DTrace merely by its production availability and accessibility to mere systems administrators owns a significant edge over its Linux based competition. Whatever the marketers on either side might say.

What we don’t yet know, however, is where things go from here: there are too many variables to make predicting a reasonable exercise. Can DTrace be successfully and completely ported to Linux, technically? Assuming the licensing issues cannot be resolved via technical workarounds, are Schwartz’ 2005 words sufficient legal grounds to stand on? If not, what does that mean for Fox’s project?

And even if DTrace can overcome the daunting technical and legal obstacles to find a new home in Linux, would anyone use it? Or would NIH preempt its adoption?

One answer, at least, we have. While many would probably argue that a DTrace port to Linux would be disastrous for Solaris and OpenSolaris – that DTrace was one the reasons the CDDL license was originally selected for the latter project, in face – the engineers that built it appear to be standing ready to help, should anyone be inclined to ask.

Technically, at least, the decision between DTrace and SystemTap would appear to be straightforward. While I know SystemTap advocates champion the greater reach of SystemTap’s guru mode among other differentiating factors, even the Linux kernel developers are admitting – implicitly and otherwise – that DTrace is the better choice at present.

When it comes to Linux and Solaris technologies, however, decisions are rarely if ever straightforward, and even more rarely made strictly according to the technology.

Which means that this should be an interesting space to watch.

Disclaimer: Many of the principals on both sides of the DTrace v SystemTap divide are clients, including IBM, The Linux Foundation, Red Hat, and Sun. On a personal level, I know many of the DTrace developers, and the SystemTap guys were kind enough to send me both a sticker and a mug after I complained that the DTrace guys didn’t have any.