[Trilug-ontopic] [bash] copying `find`ed files across network

[Tom Roche]

I'm trying to copy a buncha files across a network. Something is
horribly wrong, improvements or alternatives appreciated. I've got

me at remote:~ $ find /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$' | sort | xargs du -ch | tail -n 1
> 36M   total

and I want to copy only those files. Usually in situations like this
I do something like

tar+gzip | untar+gunzip

so I tried the following bash scriptlet (note 'remote' is defined in
my .ssh/config):

REMOT_PATH='/work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc'
LEAF="$(basename ${REMOT_PATH})"
LOCAL_ROOT="${HOME}/code/CMAQ/${LEAF}"
LOCAL_ARCHIVE_FP="${LOCAL_ROOT}/${LEAF}.tgz"
#echo -e "REMOT_PATH='${REMOT_PATH}'"
#echo -e "LOCAL_ROOT='${LOCAL_ROOT}'"

REMOT_FIND="find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$'"
REMOT_TAR="pushd ${REMOT_PATH} ; tar cfz - \$(${REMOT_FIND})"
LOCAL_TAR_FP="/tmp/junk.sh"
REMOT_TAR_FP="./$(basename ${LOCAL_TAR_FP})" # in remote ${HOME}
#echo -e "REMOT_FIND='${REMOT_FIND}'"
#echo -e "REMOT_TAR='${REMOT_TAR}'"
#echo -e "REMOT_TAR_FP='${REMOT_TAR_FP}'"
echo -e "${REMOT_TAR}\n" > "${LOCAL_TAR_FP}"
#cat "${LOCAL_TAR_FP}"

for CMD in \
  "date" \
  "rm ${LOCAL_ARCHIVE_FP}" \
  "ls -al ${LOCAL_TAR_FP}" \
  "scp ${LOCAL_TAR_FP} t:" \
  "ssh remote \"ls -al ${REMOT_TAR_FP}\"" \
  "ssh remote \"cat ${REMOT_TAR_FP}\"" \
  "ssh remote \"chmod u+x ${REMOT_TAR_FP}\"" \
  "ssh remote \"${REMOT_TAR_FP}\" > \"${LOCAL_ARCHIVE_FP}\"" \
  "ssh remote \"chmod u-x ${REMOT_TAR_FP}\"" \
  "ls -al ${LOCAL_ARCHIVE_FP}" \
  "tar tfz ${LOCAL_ARCHIVE_FP} | wc -l" \
  "date" \
; do
  echo -e "$ ${CMD}"
  eval "${CMD}"
done

> $ date
> Wed May 23 15:07:12 EDT 2012
> $ rm /home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz
> rm: cannot remove `/home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz': No such file or directory
> $ ls -al /tmp/junk.sh
> -rw-r--r-- 1 me me 116 May 23 15:07 /tmp/junk.sh
> $ scp /tmp/junk.sh t:
...
> $ ssh t "ls -al ./junk.sh"
> -rw-r--r-- 1 rtd mod3app 116 May 23 15:00 ./junk.sh
> $ ssh t "cat ./junk.sh"
> pushd /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ; tar cfz - $(find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$')
>
> $ ssh t "chmod u+x ./junk.sh"
> $ ssh t "./junk.sh" > "/home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz"
> $ ssh t "chmod u-x ./junk.sh"
> $ ls -al /home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz
> -rw-r--r-- 1 me me 1095728 May 23 15:07 /home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz
> $ tar tfz /home/me/code/CMAQ/BLD_ddm_saprc07tc/BLD_ddm_saprc07tc.tgz | wc -l
>
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> 0

What am I doing wrong? I could swear I've previously successfully
scripted this sort of thing in much the same way.

One thing that's not at fault is the for-eval loop above: I get the
same results if I just run the commands at the commandline.

TIA, Tom Roche <Tom_Roche at pobox.com>