{"id":163,"date":"2006-02-02T23:58:58","date_gmt":"2006-02-02T23:58:58","guid":{"rendered":"http:\/\/www.lucas-nussbaum.net\/blog\/?p=163"},"modified":"2006-02-02T23:58:58","modified_gmt":"2006-02-02T23:58:58","slug":"played-with-dtrace","status":"publish","type":"post","link":"https:\/\/www.lucas-nussbaum.net\/blog\/?p=163","title":{"rendered":"Played with DTrace"},"content":{"rendered":"<p>Today, I installed <a href=\"http:\/\/www.gnusolaris.org\/\">NexentaOS<\/a> on a <a href=\"http:\/\/www.grid5000.fr\">Grid5000<\/a> cluster node. NexentaOS is basically Debian GNU\/kOpenSolaris (Debian userland, with an OpenSolaris kernel. <a href=\"http:\/\/apt.gnusolaris.org\/dists\/\">APT repository here<\/a>). It works very well (good hardware support &amp; detection, nice GNOME desktop). And I got the chance to play with <a href=\"http:\/\/www.opensolaris.org\/os\/community\/dtrace\/\">DTrace<\/a>.<\/p>\n<p>Before writing this blog entry, I was considering writing a LinuxFR article about DTrace, when I came across <a href=\"http:\/\/linuxfr.org\/~itstimetogo\/16398.html\">this LinuxFR &#8220;journal&#8221; about Solaris 10<\/a>, which gave me a good laugh. The best part is :<\/p>\n<p><i>&#8211; DTrace : Sorte de surcouche de strace\/ltrace. C&#8217;est peu int\u00e9ressant. En d\u00e9monstration, un type utilise DTrace pour &#8220;d\u00e9couvrir&#8221; que lancer une xterm \u00e9crit dans ~\/.bash_history. C&#8217;est presque comique.<\/i> (In approximate english: <i>DTrace : a sort of layer above ltrace\/strace. Not really interesting. In the demo, the developer used DTrace to &#8220;discover&#8221; that running xterm writes in ~\/.bash_history. It&#8217;s nearly funny.<\/i>)<\/p>\n<p>It&#8217;s funny how people continue to compare DTrace to strace. It&#8217;s like comparing the GNOME project with fvwm. Yeah, both of them can display windows, and there are still people thinking that fvwm is enough for everybody.<\/p>\n<p>OK, back to DTrace. DTrace is a tracing framework, which allows a <i>consumer<\/i> application (generally a D script) to register with some <i>DTrace providers<\/i> (probes) and get some data. This nice graph from the <a href=\"http:\/\/www.sun.com\/software\/solaris\/howtoguides\/dtracehowto.jsp\">DTrace howto<\/a> explains it much better than I do :<\/p>\n<p><img><\/p>\n<p>Most system monitoring tools on Linux use polling : they retrieve some data from the system at regular intervals (think of top, &#8220;vmstat 1&#8221;, &#8230;). DTrace changes this and uses <i>push<\/i> instead, which allows to monitor events that you wouldn&#8217;t notice on Linux. It also allows to monitor much more stuff than current Linux tools, in a very easy and clean way.<\/p>\n<p>With DTrace, you can monitor a lot of stuff and find the answer to a lot of questions, like :<\/p>\n<ul>\n<li>Monitor process creation, even the short ones. The execsnoop script (which would be a one-liner if you remove the output formatting, and is available in the DTrace Toolkit) shows that logging in by ssh and running <code>for i in $(seq 1 3); do \/bin\/echo $i; done<\/code> runs the following processes :\n<pre>     0   1172   1171 sh -c \/usr\/bin\/locale -a     0   1172   1171 \/usr\/bin\/locale -a     0   1175   1173 -bash     0   1177   1176 id -u     0   1179   1178 dircolors -b     0   1174   1173 pt_chmod 9     0   1180   1175 mesg n     0   1181   1175 seq 1 3     0   1182   1175 \/bin\/echo 1     0   1183   1175 \/bin\/echo 2     0   1184   1175 \/bin\/echo 3 <\/pre>\n<\/li>\n<li>Monitor user and library function calls, and profile them like gprof (yeah, DTrace can replace gprof)<\/li>\n<li>Monitor system calls for the whole system or a specific app (yeah, DTrace can replace strace, but you already knew that ;). And you don&#8217;t need to restart the app before monitoring it.<\/li>\n<li>Replace vmstat. Of course, you can also get the usual vmstat results, but only for events caused by a specific process.<\/li>\n<li>Mesure the average latency between GET requests and their result when you browse the web using mozilla. DTrace does this by monitor <i>write<\/i> syscalls issued by your browser containing a GET and mesuring the delay before the subsequent <code>read<\/code> returns.<\/li>\n<li>Monitor all <code>open<\/code> syscalls issued on the whole system<\/li>\n<li>Monitor all TCP connections received by the system<\/li>\n<li>Analyze disk I\/O : how much data was written\/read to\/from the disk, by which process. Nice way to understand buffering and I\/O scheduling.<\/li>\n<\/ul>\n<p>Another example combining rwsnoop and iosnoop :<\/p>\n<ol>\n<li>I start rwsnoop -n bash (read\/write monitor, only on processes named bash) and iosnoop -m \/ (I\/O monitor, only on the root partition)<\/li>\n<li>I run : <code>echo blop &gt; t<\/code><\/li>\n<li>rwsnoop shows all the read\/write calls issued to write to my pseudo-terminal, and the write call to \/root\/t :\n<pre>   UID    PID CMD          D   BYTES FILE [...]     0   1175 bash         R       1 \/devices\/pseudo\/pts@0:1     0   1175 bash         W       1 \/devices\/pseudo\/pts@0:1     0   1175 bash         W       5 \/root\/t     0   1175 bash         W      32 \/devices\/pseudo\/pts@0:1     0   1175 bash         W      17 \/devices\/pseudo\/pts@0:1 <\/pre>\n<\/li>\n<li>iosnoop doesn&#8217;t display anything. But when I run <code>sync<\/code> :\n<pre>   UID   PID D    BLOCK   SIZE       COMM PATHNAME     0  1264 W    96640   1024       sync \/root\/t <\/pre>\n<\/li>\n<li>We can see that the write is buffered in the kernel. And that I the 5 chars to my file were transformed in 1024 bytes written to the disk. (Question for the reader: why 5 ? Yeah, it&#8217;s easy)<\/li>\n<\/ol>\n<p>Short conclusion: DTrace looks fantastic. As a toy, it allows to demonstrate\/understand the inner workings of Solaris. As a tool, it can probably provide A LOT of useful info, especially since writing DTrace providers seems quite easy (<a href=\"http:\/\/blogs.sun.com\/roller\/page\/bmc?entry=dtrace_and_ruby\">Ruby provider<\/a>, <a href=\"http:\/\/blogs.sun.com\/roller\/page\/bmc?entry=dtrace_and_php_demonstrated\">PHP provider<\/a>.<\/p>\n<p>Second short conclusion for those who really haven&#8217;t understand anything (some LinuxFR readers ;-) : as you can see, when you run <code>echo blop &gt; t<\/code>, &#8220;blop&#8221; is actually written to disk in \/root\/t. Fabulous, isn&#8217;t it ?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today, I installed NexentaOS on a Grid5000 cluster node. NexentaOS is basically Debian GNU\/kOpenSolaris (Debian userland, with an OpenSolaris kernel. APT repository here). It works very well (good hardware support &amp; detection, nice GNOME desktop). And I got the chance to play with DTrace. Before writing this blog entry, I was considering writing a LinuxFR [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"0","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-163","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=\/wp\/v2\/posts\/163","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=163"}],"version-history":[{"count":0,"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=\/wp\/v2\/posts\/163\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=163"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=163"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lucas-nussbaum.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}