[TriLUG] stale NFS problem with RHEL 4.5 & NetApp Data ONTAP 7.0.6

[Cristóbal Palmer]

Hi folks,

We've got a cluster of web servers, some of which seem to have an
interesting problem:

[~]$ cd /path/to/docroot/of/vhost/html
[html]$ ls -alh|grep for
drwxrwxr-x  21 2197 20164 4.0K Sep 26 02:19 forum
-rw-r--r--   1 2197 20164 3.8K May 14  2005 oeforums.gif
-rw-r--r--   1 2197 20164 7.6K Jan 16  2005 oeforums.png
drwxr-xr-x  18 2197 20164 4.0K Sep 26 02:14 oforums
[html]$ ls -dalh forums
drwxr-xr-x  18 2197 20164 4.0K Sep 26 02:14 forums
[html]$ ls -alh forums
total 287M
drwxr-xr-x  18 2197 20164 4.0K Sep 26 02:14 .
drwxrwxr-x  29 2197 20164 4.0K Sep 26 15:37 ..
-rwxr-xr-x   1 2197 20164  29K Jan 15  2005 admin.php
[about 20 more lines redacted]

So to parse the above for you:

I log into a web node and...
(1) I cd to the docroot of one of our vhosts
(2) I grep for anything in the docroot matching "for" and see several
files and a directory called "forum" there. I do NOT see a directory
called "forums" in this output. This is correct.
(3) I explicitly ask to see the directory "forums" and I DO see
something. This is NOT correct.
(4) I explicitly ask to see "forums" and its contents and I DO see
something. This is NOT correct.

The docroot is NFS (version 3) mounted. The NFS server is a NetApp. If
I take a web node out of service, unmount and remount the
filesystem... the problem goes away. An 'ls' will then correctly show
no directory named "forums" in the docroot.

I'm told that changing our mount options could fix this. Proposals
included setting "noac" or "actime=0" to either disable attribute
caching completely or set attribute caches to have a validity of 0
seconds, respectively. I'm worried that'll tank our performance. This
little NetApp is already plenty busy.

Here are our current mount options for that filesystem:

rsize=8192,wsize=8192,intr,tcp

We have a ticket open with NetApp, but I'm hoping to get additional
feedback from people who might be using CentOS or RHEL with a
NetApp... people who might've run into this before and found an
elegant workaround. :)

Cheers,
-- 
Cristóbal M. Palmer
celebrating 15 years of sunsite/metalab/ibiblio:
http://tinyurl.com/2o8hj4