[TriLUG] essential Linux skills?

[David McDowell]

I think Aaron just set a length record, can anyone verify?  ;)

As always, a wealth of knowledge from this guy!  thanks!


On 8/17/05, Aaron S. Joyner <aaron at joyner.ws> wrote:
> Douglass Davis wrote:
> 
> > I am teaching Linux this semester.   If some one told you that they
> > knew about Linux, what would be some essential skills that you would
> > expect them to have, or things you would expect them to know.  I don't
> > mean expert level Linux knowledge but just intermediate.
> >
> > This message is especially for those of you who have ever hired some
> > one for a position involving Linux.
> >
> > Thanks for your help.
> >
> >
> > --
> > http://www.douglassdavis.com/
> >
> Some basic things I would suggest:
> - Man pages are perhaps the most important thing you can teach.  Show
> them how to read a man page, and explain that every word in a man page
> is important, and meant to be meditated on.  They're usually extremely
> dense, but that's because every word caries some meaning.  If you can
> get them to read man pages to solve they're problem, I'd bet they'll
> learn everything twice as fast, and 4 times as well.
> - File system permissions, how they work, both modal (755) and verbose
> (o+rwx), how to read the output of ls, how to set permissions with chmod
> (late in the class, touch on sticky bit, setuid, setgid, and how they
> affect both files *and* directories)
> - Basic UNIX filesystem concepts (everything is a file, hard drives are
> files (/dev/hda), serial ports are files (/dev/ttyS0), etc, maybe block
> device vs char device)
> - Basic and important directories found in /, such as /etc, /home, /var,
> /usr, /proc, /tmp - and what's usually stored under each, and why
> - Cover some important and useful sub-dirs, such as /usr/share,
> /usr/share/doc/, /usr/src, /var/log, /var/run...
> - Cover the standard unix user setup, /etc/passwd and /etc/group,
> /etc/shadow, and perhaps how /etc/skel is used in account setup.
> Possibly mention that in a modern distribution, through pam and
> nsswitch, these concepts are extensible and can be centralized to allow
> centralized control and management of users, much like active directory
> does in windows.  You won't have time to teach ldap and kerberos
> thoroughly, but at least make the point that it's possible.
> - Cover man pages again
> - Basic familiarity with a shell.  Just pick one (I'd suggest bash), and
> explain to them some of the basics of commands, while being sure to
> explain the difference between shell built-ins, vs system commands, i.e.
> cd, export and set vs. ls, cat and grep.  Explain how echo can be
> tricky, because traditionally they're commands but bash also has a
> built-in version.  Explain shell variables vs environment variables, at
> least at a cursory level, and also touch on inheritance of environment
> variables.  Be sure to mention "special" variables, with some thoroughly
> explained, such as $PATH, $CWD, etc.  Teach them some neat tricks with
> the shell that save them time, like a few good aliases, or one-liner
> simple scripts (while read line; do foo; done; # this is often a
> favorite).  Explain output redirection, STDIN vs STDOUT vs STDERR, and
> how to chain these together in the shell to take input from one place
> (cat) and pipe it through another place (sed) and dump it out stdout
> into a file (/tmp/foo).  Once you've shown them the basics of using the
> shell as an interactive tool, then show them how to build on that
> briefly to build very simple scripts to automate repetitive tasks.  Be
> sure you at least gloss the primary programming constructs for
> conditionals, loops, etc (if, for, while, etc - depending on how far
> towards programming you want to go).  A really good combo exercise might
> have them do something moderately tedious in the process of learning an
> editor (vi), and teach them how to improve on that task in the editor
> (more on that in a moment), then show them how to do the entire same
> task in shell as an automated command, and call it from their ~/bin
> directory.  If you can teach them to understand the shell, and show them
> that it is good, you'll make life-long *NIX users out of them.
> - Cover man pages again
> - Teach them an editor, preferably one that is universally available.
> That pretty much means you teach vi, or you teach ed.  Most people
> consider that an easy decision.  :)  Then again, there is merit in
> teaching the full feature set of vim, or perhaps even in teaching
> emacs.  Generally though, as far as basic skill set goes, if you can't
> at least *use* vi, you're kinda in trouble on a minimal unix box.
> Teaching basic vi also goes very well with the rest of the unix
> philosophy, in that it's very-much designed for the super-efficiency of
> lazy typists.  Bonus points for being, or bringing in someone, who can
> edit things *ridiculously* fast in vim, and wowing the students with the
> tricks that can be done in the hands of an experienced vim user.
> Consult the list when searching for this person, if you can not fill the
> role.
> - Be sure to cover the basic suite of *NIX command line tools.  Those
> classics we all take for granted, that will allow them to construct the
> shell scripts in the previous example.  Some must-haves, cat, grep, cut,
> wc, sort, uniq, tr, sed, mktmp, (this list could go on forever) and some
> more advanced bits such as awk, or even perl one-liners, depending on
> time and quality of students.
> - Teach them about dot-files in their home directory, and how some files
> share intelligent defaults from a common dotfile (.netrc, for example)
> and how most daemons have their own dotfile or directory (.ssh is a good
> one)
> - Ensure they understand the myriad of remote access and file copy
> options.  Touch on telnet, ftp, rcp, rlogin, .rlogin (that's dot
> rlogin), and /etc/hosts.equiv - explain why they are bad.  Then
> introduce ssh, scp, and ssh host keys.  Be sure to explain the rsa key
> functionality of ssh, at least in terms of what it can allow you to do
> with un-prompted logins, and how this can be used to do things securely
> while unattended, and allow for secure authentication of hosts, not just
> users.
> - Be sure they understand the linux init process.  Try to give them at
> least a cursory understanding of how the system goes from ground zero
> (the bootloader), through the kernel, which starts init, which starts a
> single shell script, which (in most distributions, at least) fans out
> into a group of shell scripts and config files, eventually launching all
> of the programs started at bootup, including things like the login
> gettys.  Explain how all processes are children of init, and that they
> all inherit certain things set early in the boot process (such as path,
> umask, etc).  Show them that if they get lost, if they can't understand
> where something is coming from, starting at, etc it's only a matter of
> walking through the shell scripts, from beginning to end (you probably
> can't do it in it's entirety in the time alloted for something as large
> as RedHat, but the principal should at least be understood of how the
> system works).  This lays the ground work for being able to troubleshoot
> _all_manner_ of system start up problems, which anyone tinkering with
> Linux anywhere beyond the cursory user level will inevitably run across
> one day.
> - Explain daemons.  Show their basic config files, explain a few of the
> common linux config file formats (key=value, apache configs, etc) and go
> into some more depth for some of the more common daemons they might deal
> with (at least a cursory understanding of Apache, Samba, and maybe BIND
> would be helpful).  Explain cron, probably in some detail, and how it's
> commonly used.
> - Explain the TCP/IP stack.  You don't have to teach a networking class,
> but expose them to ifconfig and route, explain how to do the common
> things such as set an IP on an interface, set a default route, list
> those back out from ifconfig and route -- maybe even teach the ip tool
> in addition to (not instead of) ifconfig and route.  Explain network
> interfaces, how they're named, and the fact that they break the usual
> convention of being a file (there is no /dev/eth0...).
> - Here you should make sure you've touched on man pages
> - Explain the "special" directory structures, specifically /proc, maybe
> devfs, but only in the context of broader dev devices and "how it used
> to be" / is on most unices vs. how it is on a very modern linux.
> - Maybe explain the resolver structures, and everything keyed out of
> nsswitch.conf.  Specifically, how is a dns lookup handled (the machine
> looks in nsswitch.conf, which probably says "hosts dns", then looks at
> /etc/hosts, then looks at /etc/resolv.conf, then talks to your
> nameserver based on resolv.conf's config.  Explain how the same is done
> for /etc/passwd lookups, by looking at nsswitch, and then possibly to an
> external source such as ldap, nis, etc.
> - Maybe explain PAM, not in intricate detail, but gloss it and provide
> references.
> - Explain ps, and how to look at running processes.  This is good to do
> in conjunction with basic shell scripting, ps ax | grep.  Explain
> signals, and the kill command.  Explain how signals are handled, and at
> least SIG TERM, KILL, HUP, and why all of them but KILL and STOP are
> handled by the process, and the other two are handled by the kernel.
> - Maybe gloss over the concepts of selinux, and role-based access
> controls.  I wouldn't go too deep into this, but explain how "root is
> god" doesn't really go as far in an selinux system.
> 
> After you've hit the above topics, with sufficient depth, any thing else
> is gravy.  You can take the class in one of several directions, towards
> more programming (show more shell scripts, do more advanced shell
> tricks, tackle perl or python), or towards more sysadmin directions
> (setup daemons, configure them, make them do your bidding) or towards
> more userland eye candy (setup gnome or kde, change the window manager
> to something exotic, enlightenment perhaps, setup configs for the window
> manager, etc).  But don't get too lulled into this part of the class too
> fast.  Do note that I leave userland eye candy off my list of essential
> unix features, because being able to configure gnome or open OpenOffice
> and type up a document are no-brainer activities that any windows user
> can manage.  Exploring the GUI tools that come with <your distro of
> choice> is an easy, one afternoon process, and isn't likely to provide
> you any career opportunities, or help you later in life.  These aspects
> of the Linux GUI are still highly in flux, and likely to be out-dated by
> the time your students finish the class.  Most of the points from
> earlier in my message haven't changed in any fundamental way in the last
> 25 years.
> 
> As a professor, you should probably prepare by reading a few key books,
> I'd specifically recommend The Art of UNIX Programming.  This will
> provide you with the background (if you don't already possess it), to be
> able to give insights into *why* the unix systems you're teaching work
> the way they do, as well as a whole host of interesting historical
> anecdotes that can make the class interesting while putting things in
> proper context.  Note that this book is freely available under the
> creative commons license, and can be pulled by a simple google search,
> or picked up at your local Barnes and Nobles for under $40 conveniently
> printed on dead trees.
> 
> Let me wrap all this up by saying that I've recently put a lot of
> thought into all of this, as I've been doing phone screens of system
> administrators for Google.  I'd say what I've described above
> constitutes the basics of a person who would make a good Linux user,
> who's got a good foundation laid to work towards being a sys admin or a
> software engineer.
> 
> Best of luck with your class, I hope this has been worth the time spent
> reading it,
> Aaron S. Joyner
> System Administrator
> Google, Inc.
> 
> PS - If anyone has time to throw this knowledge into the wiki, please
> do!  Shoot an email reminder when you've got it up there and I'll try to
> pick a few sub-sections to write up details for.
> --
> TriLUG mailing list        : http://www.trilug.org/mailman/listinfo/trilug
> TriLUG Organizational FAQ  : http://trilug.org/faq/
> TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
> TriLUG PGP Keyring         : http://trilug.org/~chrish/trilug.asc
>