Using xargs with tar archives The following line does not work as each set resets the tar file: find . -type f -name "*.java" | xargs tar cvf myfile.tar The following line does work by replacing the "c" with an "r" in tar args: find . -type f -name "*.java" | xargs tar rvf myfile.tar -------------------------------------------------------------------------------- Using the xargs Command http://www.unixreview.com/documents/s=8274/sam0306g/ Many UNIX professionals think the xargs command, construct and execute argument lists, is only useful for processing long lists of files generated by the find command. While xargs dutifully serves this purpose, xargs has other uses. In this article, I describe xargs and the historical "Too many arguments" problem, and present eight xargs "one-liners": -------------------------------------------------------------------------------- Examining the "Too Many Arguments" Problem In the early days of UNIX/xenix, it was easy to overflow the command-line buffer, causing a "Too many arguments" failure. Finding a large number of files and piping them to another command was enough to cause the failure. Executing the following command, from Unix Power Tools, first edition (O'Reilly & Associates): pr -n 'find . -type f -mtime -1 -print' | lpr will potentially overflow the command line given enough files. This command provides a list of all the files edited today to pr, and pipes pr's output to the printer. We can solve this problem with xargs: find . -type f -mtime -1 -print|xargs pr -n | lp With no options, xargs reads standard input, but only writes enough arguments to standard output as to not overflow the command-line buffer. Thus, if needed, xargs forces multiple executions of pr -n|lp. While xargs controls overflowing the command-line buffer, the command xargs services may overflow. I've witnessed the following mv command fail -- not the command-line buffer -- with an argument list too long error: find ./ -type f -print | xargs -i mv -f {} ./newdir Limit the number of files sent to mv at a time by using the xargs -l option. The following command sets a limit of 56 files at time, which mv receives: find ./ -type f -print | xargs -l56 -i mv -f {} ./newdir The modern UNIX OS seems to have solved the problem of the find command overflowing the command-line buffer. However, using the find -exec command is still troublesome. It's better to do this: # remove all files with a txt extension find . -type f -name "*.txt" -print | xargs rm than this: find . -type f -name "*.txt" -exec rm {} \; -print Controlling the call to rm with xargs is more efficient than having the find command execute rm for each object found. -------------------------------------------------------------------------------- xargs One-Liners The find-xargs command combination is a powerful tool. The following example finds the unique owners of all the files in the /bin directory: find /bin -type f -follow | xargs ls -al | awk ' NF==9 { print $3 } ' | sort -u If /bin is a soft link, as it is with Solaris, the -follow option forces find to follow the link. The xargs command feeds the ls -al command, which pipes to awk. If the output of the ls -al command is 9 fields, print field 3 -- the file owner. Sorting the awk output and piping to the uniq command ensures unique owners. You can use xargs options to build extremely powerful commands. Expanding the xargs/rm example, let's assume the requirement exists to echo each file to standard output as it deletes: find . -type f -name "*.txt" | xargs -i ksh -c "echo deleting {}; rm {}" The xargs -i option replaces instances of {} in a command (i.e., echo and rm are commands). Conversely, instead of using the -i option with {}, the xargs -I option replaces instances of a string. The above command can be written as: find . -type f -name "*.txt" | xargs -I {} ksh -c "echo deleting {}; rm {}" The new, third edition of Unix Power Tools by Powers et al. provides an xargs "one-liner" that duplicates a directory tree. The following command creates in the usr/project directory, a copy of the current working directory structure: find . -type d -print|sed 's@^@/usr/project/@' | xargs mkdir The /usr/project directory must exist. When executing, note the error: mkdir: Failed to make directory "/usr/project/"; File exists which doesn't prevent the directory structure creation. Ignore it. To learn how the above command works, you can read more in Unix Power Tools, third edition, Chapter 9.17 (O'Reilly & Associates). In addition to serving the find command, xargs can be a slave to other commands. Suppose the requirement is to group the output of UNIX commands on one line. Executing: logname; date displays the logname and date on two separate lines. Placing commands in parentheses and piping to xargs places the output of both commands on one line: (logname; date)|xargs Executing the following command places all the file names in the current directory on one line, and redirects to file "file.ls": ls | xargs echo > file.ls Use the xargs number of arguments option, -n, to display the contents of "file.ls" to standard output, one name per line: cat file.ls|xargs -n1 # from unix in a nutshell In the current directory, use the xargs -p option to prompt the user to remove each file individually: ls|xargs -p -n1 rm Without the -n option, the user is prompted to delete all the files in the current directory. Concatenate the contents of all the files whose names are contained in file into file.contents: xargs cat < file > file.contents Move all files from directory $1 to directory $2, and use the xargs -t option to echo each move as it happens: ls $1 | xargs -I {} -t mv $1/{} $2/{} The xargs -I argument replaces each {} in the string with each object piped to xargs. -------------------------------------------------------------------------------- Conclusion When should you use xargs? When the output of a command is the command-line options of another command, use xargs in conjunction with pipes. When the output of a command is the input of another command, use pipes. -------------------------------------------------------------------------------- When using the duplicate directory tree "one-liner", reader Peter Ludemann suggests using the mkdir -p option: find . -type d -print|sed 's@^@/usr/project/@'|xargs mkdir -p instead of: find . -type d -print|sed 's@^@/usr/project/@'|xargs mkdir mkdir's "-p" option creates parent directories as needed, and doesn't error out if one exists. Additionally, /usr/project does not have to exist. -------------------------------------------------------------------------------- Ludemann, in addition to reader Christer Jansson, commented that spaces in directory names renders the duplicate directory tree completely useless. Although I'm unable to salvage the duplicate directory command, for those find and xargs versions that support -0 (probably GNU versions only), you might try experimenting with: find ... -print0 | xargs -0 ... Using Ludemann's email example, suppose your current directory structure contains: foo bar foo bar find . -type f -print | xargs -n 1 incorrectly produces: foo bar foo bar while find . -type f -print0 | xargs -0 -n 1 delivers the correct results: foo bar foo bar According to the 7.1 Red Hat Linux man page for xargs and find, the -0 doesn't use the null terminator for file names disabling the special meaning of white space. -------------------------------------------------------------------------------- Reader Peter Simpkin asks the question, "Does the use of xargs only operate after the find command has completed? find. -type f -name "*.txt" -print | xargs rm If not, I was under the impression that the above was a bad idea as it is modifying the current directory that find is working from, or at least this is what people have told me, and, thus the results of find are then undefined." My response is "no". Any Unix command that supports command-line arguments is an xargs candidate. The results of the find command are as valid as the output of the ls command: # remove files ending with .txt in current directory ls *.txt|xargs rm If a command such as this is valid: chmod 444 1.txt 2.txt 3.txt then: find . \( -name 1.txt -o -name 2.txt -o -name 3.txt \) -print|xargs chmod 444 is valid.