[TriLUG] docker swarm, rancheros, persistent storage

Tue Jan 2 16:53:12 EST 2018

I'm a little worried this is nonsequitur (not swarm, not Rancher) and maybe
not even helpful but it's really interesting and related using a completely
different docker technology using resource scheduling (like Slurm).
Shifter at NERSC gives scientists a way to deploy their docker batch sets
(mpi style or embarrassingly parallel) in a secure way using loop devices.
I think you can still share them from your image (tricky since they only
allow user permissions) or use their available filesystems.  This is how
you would use it in their environment and API

http://www.nersc.gov/users/software/using-shifter-and-
docker/using-shifter-at-nersc/#Volume_Mounting

And looking under the hood are the linked papers

http://www.nersc.gov/research-and-development/user-defined-images/

It gives me some ideas though. Unfortunately the Shifter spec doesn't go
into file locking and worse it prereqs Lustre, a parallel file system.
POSIX locking can be very tricky under the best conditions and your
software might be the best place to conduct locking.  I know putting that
further up the application stack is burdensome but I think that is the
future direction as data becomes so very large POSIX just falls over.

Still, POSIX over NFS is just so *handy* and it's weird that it doesn't
work.  Have you tried v4.1 vs v4 with later patching?   I know it's been a
little volatile of late.

sean

On Sat, Dec 16, 2017 at 1:52 PM, Dewey Hylton via TriLUG <trilug at trilug.org>
wrote:

> Hi all!
>
> What do you use for replicated or shared persistant volumes for your
> docker swarm containers?
>
> Most folks who know me know that I'm a minimalist, sometimes (or most
> times) to a fault. I'm currently working with RancherOS for my Docker-based
> projects. I like it very much because it is very simple and stripped down -
> thus it fits me. I can install via PXE, it seems to run Docker very well,
> etc. It also works great in swarm mode. I happen to like swarm mode because
> it is baked into Docker, secure by default (as in its management traffic is
> secured by TLS), and it is very simple to get going - particularly when
> compared with Kubernetes.
>
> Recently I have begun looking at moving some of my stateful containers
> into swarm mode for redundancy. The theory is pretty simple; move the
> stateful data into a named Docker volume which is accessible by all cluster
> nodes, and therefore all containers. For external databases and such this
> is not a big deal, but I have found this to be a pain point for simple
> shared files. For example, a moinmoin wiki leverages plain files instead of
> a database; another is the fossil cms, which can serve an entire directory
> of fossil repositories, which are stored on the filesystem as sqlite
> database files. All this works great in a Docker container, with data in a
> named volume - but I have yet to figure out how to make those directories
> and files available to containers on different hosts.
>
> I have read that Kubernetes can provide those shared volumes somehow, but
> I'd really like to find a decent way to do this without having to add so
> much complexity. I've attempted NFS mounts (docker volume create --opt
> type=nfs) and while the creation does not error out, and all swarm
> containers see the volume, the volume data does not seem to reflect the
> data on the nfs share at all. I've also attempted portworx, which sounds
> fantastic aside from its price tag for enterprise users, but I have failed
> to get it installed properly. It may be that both of these failures are
> somewhat due to the stripped-down nature of RancherOS. If anyone has any
> experience with this, particularly with RancherOS (not necessarily Rancher,
> though), I'd love to hear from you.
>
> Even if you do not use RancherOS, and instead have a full installation of
> something else (eg. Ubuntu/CentOS) with docker installed atop of that - and
> you have figured out how to provide shared volumes across your swarm nodes
> - I would appreciate hearing from you as well.
>
> Thanks!
> --
> This message was sent to: Sean Korb <spkorb at gmail.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> TriLUG mailing list : https://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  : https://www.trilug.org/
> mailman/options/trilug/spkorb%40gmail.com
> Welcome to TriLUG: http://trilug.org/welcome

-- 
Sean Korb spkorb at spkorb.org http://www.spkorb.org
'65 Suprang,'68 Cougar,'78 R100/7,'60 Metro,'59 A35,'71 Pantera #1382
"The more you drive, the less intelligent you get" --Miller
"Computers are useless.  They can only give you answers." -P. Picasso