{"id":6129,"date":"2017-01-12T08:59:02","date_gmt":"2017-01-12T13:59:02","guid":{"rendered":"http:\/\/appcrawler.com\/wordpress\/?p=6129"},"modified":"2017-01-12T08:59:02","modified_gmt":"2017-01-12T13:59:02","slug":"does-a-command-retain-open-file-handles-when-the-output-of-the-pipe-is-aggregated","status":"publish","type":"post","link":"http:\/\/appcrawler.com\/wordpress\/2017\/01\/12\/does-a-command-retain-open-file-handles-when-the-output-of-the-pipe-is-aggregated\/","title":{"rendered":"Does a command retain open file handles when the output of the pipe is aggregated?"},"content":{"rendered":"<p>We had 90,000 open awk file handles owned by a single shell process, and didn&#8217;t know if a command such as what is below would result in each file handle being retained even after its output was processed and the results passed to the pipe for the &#8220;wc -l&#8221; command shown.<\/p>\n<pre>\r\n[esb@cmhlcarchapp01 ~]$ find . -type f | xargs awk '{system(\"sleep 1\");print $0}' | wc -l\r\n<\/pre>\n<p>We found they are not kept open.  The GetCertDates.java file below is one of many I saw as I continually ran the lsof command shown.  At any one time, only the one in the loop showed up as open.  As such, we had another script bug.  However, I wanted to post this for future reference.<\/p>\n<pre>\r\n[esb@cmhlcarchapp01 ~]$ lsof | grep awk\r\nawk       30133       esb  cwd       DIR  253,0     4096 1179654 \/homelocal\/esb\r\nawk       30133       esb  rtd       DIR  253,0     4096       2 \/\r\nawk       30133       esb  txt       REG  253,0   382752 1048651 \/bin\/gawk\r\nawk       30133       esb  mem       REG  253,0   599392  264766 \/lib64\/libm-2.12.so\r\nawk       30133       esb  mem       REG  253,0   157072  262201 \/lib64\/ld-2.12.so\r\nawk       30133       esb  mem       REG  253,0    22536  262281 \/lib64\/libdl-2.12.so\r\nawk       30133       esb  mem       REG  253,0  1926520  262264 \/lib64\/libc-2.12.so\r\nawk       30133       esb  mem       REG  253,0 99158576  400766 \/usr\/lib\/locale\/locale-archive\r\nawk       30133       esb  mem       REG  253,0    26060  394258 \/usr\/lib64\/gconv\/gconv-modules.cache\r\nawk       30133       esb    0r      CHR    1,3      0t0    4025 \/dev\/null\r\nawk       30133       esb    1w     FIFO    0,8      0t0 8426830 pipe\r\nawk       30133       esb    2u      CHR  136,0      0t0       3 \/dev\/pts\/0\r\nawk       30133       esb    3r      REG  253,0      527 1184079 \/homelocal\/esb\/GetCertDates.java\r\n[esb@cmhlcarchapp01 ~]$\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>We had 90,000 open awk file handles owned by a single shell process, and didn&#8217;t know if a command such as what is below would result in each file handle being retained even after its output was processed and the&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/appcrawler.com\/wordpress\/2017\/01\/12\/does-a-command-retain-open-file-handles-when-the-output-of-the-pipe-is-aggregated\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[14,16],"tags":[],"_links":{"self":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6129"}],"collection":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/comments?post=6129"}],"version-history":[{"count":2,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6129\/revisions"}],"predecessor-version":[{"id":6132,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6129\/revisions\/6132"}],"wp:attachment":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/media?parent=6129"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/categories?post=6129"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/tags?post=6129"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}