[Trilug-ontopic] [bash] copying `find`ed files across network
Tom Roche
Tom_Roche at pobox.com
Wed May 23 17:19:04 EDT 2012
http://www.trilug.org/pipermail/trilug-ontopic/2012-May/000315.html
Tom Roche Wed, 23 May 2012 15:36:07 -0400
>> I'm trying to copy a buncha files across a network. Something is
>> horribly wrong, improvements or alternatives appreciated. I've got
>> me at remote:~ $ find /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$' | sort | xargs du -ch | tail -n 1
>> > 36M total
>> and I want to copy only those files.
Alan Porter Wed May 23 15:45:15 EDT 2012
> How about this?
> tar -zcf - $(find /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$') | ssh root at remote "tar -zxvf -"
I don't have root on remote, and I don't wanna restore there ...
but making a similar adjustment adds a datapoint (starred below):
REMOT_PATH='/work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc'
LEAF="$(basename ${REMOT_PATH})"
LOCAL_ROOT="${HOME}/code/CMAQ/${LEAF}"
LOCAL_ARCHIVE_FP="${LOCAL_ROOT}/${LEAF}.tgz"
REMOT_FIND="find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$'"
REMOT_TAR="pushd ${REMOT_PATH} ; tar cfvz - \$(${REMOT_FIND})"
for CMD in \
"mkdir -p ${LOCAL_ROOT}" \
"pushd ${LOCAL_ROOT}" \
"ssh t \"pushd ${REMOT_PATH} ; ${REMOT_FIND} | wc -l\"" \
"ssh t \"${REMOT_TAR}\" | tar xvfz -" \
"find . -type f | wc -l" \
"popd" \
; do
echo -e "$ ${CMD}"
eval "${CMD}"
done
> $ mkdir -p /home/me/code/CMAQ/BLD_ddm_saprc07tc
> $ pushd /home/me/code/CMAQ/BLD_ddm_saprc07tc
> ~/code/CMAQ/BLD_ddm_saprc07tc ~
# case 1: `raw find`
* $ ssh t "pushd /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ; find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$' | wc -l"
* /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ~
* 266
# case 2: `find` inside `tar`
* $ ssh t "pushd /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ; tar cfvz - $(find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$')" | tar xvfz -
* tar: Cowardly refusing to create an empty archive
* Try `tar --help' or `tar --usage' for more information.
>
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> $ find . -type f | wc -l
> 0
> $ popd
Why is the raw `find` working, but the same `find` inside `tar` draws
the 'empty archive' warning (presumably due to *not* finding the files)?
More information about the Trilug-ontopic
mailing list