Does a command retain open file handles when the output of the pipe is aggregated?

We had 90,000 open awk file handles owned by a single shell process, and didn’t know if a command such as what is below would result in each file handle being retained even after its output was processed and the results passed to the pipe for the “wc -l” command shown.

[esb@cmhlcarchapp01 ~]$ find . -type f | xargs awk '{system("sleep 1");print $0}' | wc -l

We found they are not kept open. The GetCertDates.java file below is one of many I saw as I continually ran the lsof command shown. At any one time, only the one in the loop showed up as open. As such, we had another script bug. However, I wanted to post this for future reference.

[esb@cmhlcarchapp01 ~]$ lsof | grep awk
awk       30133       esb  cwd       DIR  253,0     4096 1179654 /homelocal/esb
awk       30133       esb  rtd       DIR  253,0     4096       2 /
awk       30133       esb  txt       REG  253,0   382752 1048651 /bin/gawk
awk       30133       esb  mem       REG  253,0   599392  264766 /lib64/libm-2.12.so
awk       30133       esb  mem       REG  253,0   157072  262201 /lib64/ld-2.12.so
awk       30133       esb  mem       REG  253,0    22536  262281 /lib64/libdl-2.12.so
awk       30133       esb  mem       REG  253,0  1926520  262264 /lib64/libc-2.12.so
awk       30133       esb  mem       REG  253,0 99158576  400766 /usr/lib/locale/locale-archive
awk       30133       esb  mem       REG  253,0    26060  394258 /usr/lib64/gconv/gconv-modules.cache
awk       30133       esb    0r      CHR    1,3      0t0    4025 /dev/null
awk       30133       esb    1w     FIFO    0,8      0t0 8426830 pipe
awk       30133       esb    2u      CHR  136,0      0t0       3 /dev/pts/0
awk       30133       esb    3r      REG  253,0      527 1184079 /homelocal/esb/GetCertDates.java
[esb@cmhlcarchapp01 ~]$

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.