[TriLUG] SAN file locking

[bak]

SAN storage is not meant to be shared in the way that you describe, at
the file level. Individual files are not shared in SAN setups, so file
locking is not a relevant concept.

Think of SAN as presenting some blocks on a disk somewhere to a server
such that it believes those blocks are on a directly connected disk of
an arbitrary size.

Just like you wouldn't have two PCs connected to the same USB hard drive
trying to access its blocks at the same time, as a rule, two servers
won't access the same set of blocks on a SAN pretending to be a disk at
the same time. Filesystem layout is just too complex to coordinate in
this way and keep things running at low latencies. (A side effect of
this is that usually a SAN system can't see 'into' a disk used by a
server, and wouldn't for example know how many files are in there, how
full it is, etc. All it sees are blocks.)

As a rule, a SAN setup will use various kinds of masking to keep hosts
from seeing blocks that don't "belong" to them in the closet of disk
drives that makes up a SAN array.

Now, there are exceptions to the "only one at a time rule", usually made
for clustered systems. They often have a disk that they share in order
to coordinate cluster activities. In the case of a Microsoft Exchange
cluster, or a Solaris cluster, data is not served from there.

VMWare clusters also share disks, but they have a very specialized
cluster-aware filesystem that lets them pass the "ownership" of
individual block ranges back and forth to cluster members as you do
things like VMotion.

The use cases you describe are not reasons why you would deploy a SAN.
Sometimes-access to one or more files at a time is exactly why you use a
NAS -- let some other system handle the filesystem and serve things out
as needed, one file at a time.

SANs are for low-latency, high-throughput situations. In an Oracle or
OLTP situation, getting to the data fast and crunching through it to
return a result is the goal, so fast access to a relatively small
working set is necessary. Contrast this with a NAS situation, where you
might have a music file that gets read slowly every day or two.

Another traditional place for SAN are things like Exchange where
everyone has to talk to one or two relatively beefy servers to begin
with, so you don't need a whole farm of them sharing files.

Although you can run VMWare over NFS these days, often SANs are deployed
to run ESX clusters, because you want your VM guests to have fast,
low-latency access to their virtual disks so that they don't bog down
waiting for I/O, and because a shared storage backend enables all kinds
of nifty VMWare magic.

SAN is all about 'limited space, one server at a time, but very fast' --
databases, virtual disks... whereas NAS is about monster buckets of data
that are used more rarely -- home directories, pictures.

--bak


On 12/16/11 7:42 PM, Joseph Mack NA3T wrote:
> I haven't setup a SAN so don't know how people handle file locking. The
> problem is that any host on the SAN network can in principle access a
> file. Here's the only thing I've found about SAN file locking, a
> metadata broker
> 
> http://fusesource.com/docs/broker/5.4/clustering/Failover-MasterSlave-Shared.html
> 
> 
> I assume the SAN file locking problem is the same as for other shared
> file systems eg. for multiple hosts mounting disks via NFS.
> 
> This wiki page
> 
> http://en.wikipedia.org/wiki/File_locking
> 
> shows that some implementations of NFS have file locking and
> some don't.
> 
> Is everyone just mounting /usr (ro) and sharing that, while mounting
> home directories (rw) expecting the user to handle the exclusive writing
> manually? In this case there is no real sharing. If there's no real
> sharing, what are SANs being used for? - stupendously large amounts of
> home directories (eg cloud storage of pictures, music, movies)? What
> about all this data storage we're always hearing about that is storing
> more data than all the National Geographics in dusty boxes in people's
> creeking attics - is this all (ro) to make webpages etc?
> 
> How are large companies managing their inventories, production reports,
> financial statements, stocks, puts/calls? How does Apple, GM,
> Goldman-Sachs store their data and handle filelocking. I assume they're
> all on SANs. Is the data all handled by the database (Oracle?) which
> handles the locking for the user and the user never sees the shared blocks?
> 
> Joe
>