[TriLUG] FTP (Again...)

[Mark Freeze]

I'd like to dredge up the old ftp problem I've discussed several times
on this list to see if there are any new ideas, or really just to
waste everyone's time...

(Preface - We use vsftpd with SSL)

We typically receive anywhere between ten and fifty files per day on
our ftp site from our customers.  I have some scripts that monitor the
incoming ftp directory  that take action on the uploaded files (based
on the filename) when the upload is complete.

My question is simply this: How do you know the upload is complete?

For example, we receive five files from $LOCAL_HOSPITAL every evening
at 10pm.  At 11pm we cron up the ol perl script to parse these files
into databases, etc...   One evening last week, $LOCAL_HOSPITAL was
late in transferring the files and the cron job fired in the middle of
the upload, grabbed a few incomplete files, and crashed the rest of
the upload.  The short(-sighted) answer that I get is, "Move the cron
job to later in the night."  While that would work for
$LOCAL_HOSPITAL, we receive many other files from various sources at
differing (and unannounced) times during the day and night.   I really
want my process to  watch this directory for me, but only get
completed uploads.

I know I will suffer the smack-down for saying this, but WS_FTP for
windows used to accomplish this by naming the uploaded file with a
".tmp" extension until the upload completes, then renaming the file
back to its original name.  I think later versions used a temp
directory then moved the file after completion.  I may be confusing
the product with something else, but if it didn't work that way it
should have.

The solutions I hate and that are against my religionization:

* - Sending a 'token' or 'semaphore' after the file has completed.  I
can't ask my customers to do this.  It's enough to ask them to
remember their passwords.

* - Checking the file size over and over and over and over. Anyway,
even if the file size stays the same, how do you know that the upload
completed and wasn't interrupted? (That one maybe unsolveable with
FTP.)

Does anyone have anything to offer on this matter?  A bone to throw
out? A perl of wisdom? (He! He!)

Would anyone like to help me edit the source of vsftpd to accomplish
the same stuff as in the windows examples?

Is there another solution?

Can I quit adding stuff to this message?

Thanks in advance!

Regards,
Mark.