[TriLUG] The sad state of sysadmin in the age of containers

[Igor Partola via TriLUG]

Ah, well there's the real issue then. I really don't see Puppet as
something evil. If the NSA wanted to backdoor my servers they could do it
much simpler. Any script, and package, any piece of software that runs as
root may download and install a backdoor; why target just puppet? If I was
tasked with backdooring as many boxes as possible, I would target something
much more ubiquitous that runs as root, perhaps systemd, the Linux kernel,
some closed source blob. Puppet just happens to install stuff, often times
from signed and trusted repositories only. In fact, you have to explicitly
ask it to install stuff from elsewhere, since the built-in package
directive will only do what your package manager will do. Blaming puppet
for these security problems is like blaming bash because you can write a
script that wget's a shell script from an unsecured location and then runs
it as root. It's not bash's job to catch problems like this.

Now, onto the real problem: HTTP and lack of signatures. HTTP is easy to
fix: it needs to die. We should all move to HTTPS, and this summer most
sites will (see https://letsencrypt.org/). I strongly advocate that
browsers should stop supporting raw HTTP altogether. It'll be a painful
transition but it needs to happen. If you don't advocate the use of Telnet
instead of SSH, then you wouldn't advocate HTTP instead of HTTPS (remember,
your browser may download and run arbitrary code on your computer. Don't
you want that code to come from where you think it comes from and not the
NSA?)

The second part is much tricker: signing code. Whose signatures do you
trust? How do you know that the person isn't doing something nefarious? Or
one of their collaborators? For example, I publish a number of open source
libraries and projects on GitHub. You can download them there, and I can
even personally send you a signed tarball using my private PGP key. You can
be 100% sure that the code comes from me. But that doesn't tell you whether
the code contains a backdoor or not. Without reading any source code you
won't know it, and most people do not read 100% of the code they use
(indeed, it would take ages to do it, and every update would mean that it
becomes necessary to do it again).

In other words, you have to trust me personally, if you want to use my
code. My signature on it is necessary, but not sufficient to ensure you
aren't downloading a backdoor. This is why distro-independent repositories
(such as PyPI) don't require authors to sign their packages: it doesn't
matter. The download is secured since it comes via HTTPS, but whether it
comes from author A or B is often not an issue: you as the user of the
package cannot tell them apart anyways.

So why do distros use signed packages, while lots of other sources don't?
The answer is that it's a good idea and distros have the resources to read
all the code. Moreover, they place huge time constraints on publishing the
package. It will take months or years for a new package to appear in
Debian. It will take days or weeks for an update to make it in. PyPI and
the like don't do this: updates are instant. This gives more authors the
freedom to publish source and binary packages, at the cost of security.
Seems like a bad tradeoff, until you try using both methods and see just
how slow publishing a package for a distro is. As the author of the
software, I don't want to wait years, especially since I'm likely trying to
use the software right away.

Having said all that, any time you see a situation where code is downloaded
over HTTP and without a signature, shame the involved parties publicly!

Igor