[TriLUG] Apache: Segmentation fault errors

[Paul G. Szabady]

Greetings,

I finally have a chance to go heads down on this again.  But I think I'm
still missing something.  I downloaded the src rpm for httpd and added the
following into my httpd.spec file before compiling it.

EXTRA_CFLAGS='-g'
export EXTRA_CFLAGS

I upgraded to the latest version of rpms:
rpm -Uvh /usr/src/redhat/RPMS/i386/httpd-2.0.46-61.ent.i386.rpm
/usr/src/redhat/RPMS/i386/httpd-debuginfo-2.0.46-61.ent.i386.rpm
/usr/src/redhat/RPMS/i386/httpd-devel-2.0.46-61.ent.i386.rpm
/usr/src/redhat/RPMS/i386/mod_ssl-2.0.46-61.ent.i386.rpm  --nodeps --force

I then went to the phpBB admin panel which causes the seg fault.  It
dumped a core file which I tried to backtrace here:

# gdb /usr/sbin/httpd /var/tmp/core.53045
GNU gdb Red Hat Linux (6.3.0.0-1.62rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host
libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libssl.so.4...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/libcrypto.so.4...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2
Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done.
Loaded symbols for /usr/kerberos/lib/libkrb5.so.3
Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done.
Loaded symbols for /usr/kerberos/lib/libcom_err.so.3
Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/libpcre.so.0...done.
Loaded symbols for /lib/libpcre.so.0
Reading symbols from /usr/lib/libpcreposix.so.0...done.
Loaded symbols for /usr/lib/libpcreposix.so.0
Reading symbols from /usr/lib/libaprutil-0.so.0...Reading symbols from
/usr/lib/debug/usr/lib/libaprutil-0.so.0.9.4.debug...done.
done.
Loaded symbols for /usr/lib/libaprutil-0.so.0
Reading symbols from /lib/libdb-4.1.so...done.
Loaded symbols for /lib/libdb-4.1.so
Reading symbols from /usr/lib/libexpat.so.0...Reading symbols from
/usr/lib/debug/usr/lib/libexpat.so.0.4.0.debug...done.
done.
Loaded symbols for /usr/lib/libexpat.so.0
Reading symbols from /usr/local/lib/libiconv.so.2...done.
Loaded symbols for /usr/local/lib/libiconv.so.2
Reading symbols from /usr/lib/libapr-0.so.0...Reading symbols from
/usr/lib/debug/usr/lib/libapr-0.so.0.9.4.debug...done.
done.
Loaded symbols for /usr/lib/libapr-0.so.0
Reading symbols from /lib/tls/librt.so.1...done.
Loaded symbols for /lib/tls/librt.so.1
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/tls/libpthread.so.0...done.
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /usr/lib/httpd/modules/mod_access.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_access.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_access.so
Reading symbols from /usr/lib/httpd/modules/mod_auth.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_auth.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth.so
Reading symbols from /usr/lib/httpd/modules/mod_auth_anon.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_auth_anon.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth_anon.so
Reading symbols from /usr/lib/httpd/modules/mod_auth_dbm.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_auth_dbm.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth_dbm.so
Reading symbols from /usr/lib/httpd/modules/mod_auth_digest.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_auth_digest.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth_digest.so
Reading symbols from /usr/lib/httpd/modules/mod_include.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_include.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_include.so
Reading symbols from /usr/lib/httpd/modules/mod_log_config.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_log_config.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_log_config.so
Reading symbols from /usr/lib/httpd/modules/mod_env.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_env.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_env.so
Reading symbols from /usr/lib/httpd/modules/mod_mime_magic.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_mime_magic.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_mime_magic.so
Reading symbols from /usr/lib/httpd/modules/mod_cern_meta.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_cern_meta.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_cern_meta.so
Reading symbols from /usr/lib/httpd/modules/mod_expires.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_expires.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_expires.so
Reading symbols from /usr/lib/httpd/modules/mod_deflate.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_deflate.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_deflate.so
Reading symbols from /usr/lib/httpd/modules/mod_headers.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_headers.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_headers.so
Reading symbols from /usr/lib/httpd/modules/mod_usertrack.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_usertrack.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_usertrack.so
Reading symbols from /usr/lib/httpd/modules/mod_unique_id.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_unique_id.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_unique_id.so
Reading symbols from /usr/lib/httpd/modules/mod_setenvif.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_setenvif.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_setenvif.so
Reading symbols from /usr/lib/httpd/modules/mod_mime.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_mime.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_mime.so
Reading symbols from /usr/lib/httpd/modules/mod_dav.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_dav.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dav.so
Reading symbols from /usr/lib/httpd/modules/mod_status.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_status.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_status.so
Reading symbols from /usr/lib/httpd/modules/mod_autoindex.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_autoindex.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_autoindex.so
Reading symbols from /usr/lib/httpd/modules/mod_asis.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_asis.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_asis.so
Reading symbols from /usr/lib/httpd/modules/mod_info.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_info.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_info.so
Reading symbols from /usr/lib/httpd/modules/mod_dav_fs.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_dav_fs.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dav_fs.so
Reading symbols from /usr/lib/httpd/modules/mod_vhost_alias.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_vhost_alias.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_vhost_alias.so
Reading symbols from /usr/lib/httpd/modules/mod_negotiation.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_negotiation.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_negotiation.so
Reading symbols from /usr/lib/httpd/modules/mod_dir.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_dir.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dir.so
Reading symbols from /usr/lib/httpd/modules/mod_imap.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_imap.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_imap.so
Reading symbols from /usr/lib/httpd/modules/mod_actions.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_actions.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_actions.so
Reading symbols from /usr/lib/httpd/modules/mod_speling.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_speling.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_speling.so
Reading symbols from /usr/lib/httpd/modules/mod_alias.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_alias.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_alias.so
Reading symbols from /usr/lib/httpd/modules/mod_rewrite.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_rewrite.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_rewrite.so
Reading symbols from /usr/lib/httpd/modules/mod_proxy.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_proxy.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy.so
Reading symbols from /usr/lib/httpd/modules/mod_proxy_ftp.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_proxy_ftp.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_ftp.so
Reading symbols from /usr/lib/httpd/modules/mod_proxy_http.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_proxy_http.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_http.so
Reading symbols from /usr/lib/httpd/modules/mod_proxy_connect.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_proxy_connect.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_connect.so
Reading symbols from /usr/lib/httpd/modules/mod_cache.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_cache.so
Reading symbols from /usr/lib/httpd/modules/mod_suexec.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_suexec.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_suexec.so
Reading symbols from /usr/lib/httpd/modules/mod_disk_cache.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_disk_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_disk_cache.so
Reading symbols from /usr/lib/httpd/modules/mod_file_cache.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_file_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_file_cache.so
Reading symbols from /usr/lib/httpd/modules/mod_mem_cache.so...Reading
symbols from
/usr/lib/debug/usr/lib/httpd/modules/mod_mem_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_mem_cache.so
Reading symbols from /usr/lib/httpd/modules/mod_cgi.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_cgi.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_cgi.so
Reading symbols from /usr/lib/httpd/modules/mod_auth_mysql.so...done.
Loaded symbols for /etc/httpd/modules/mod_auth_mysql.so
Reading symbols from /usr/lib/mysql/libmysqlclient.so.10...done.
Loaded symbols for /usr/lib/mysql/libmysqlclient.so.10
Reading symbols from /usr/lib/httpd/modules/mod_authz_ldap.so...done.
Loaded symbols for /etc/httpd/modules/mod_authz_ldap.so
Reading symbols from /usr/lib/libldap.so.2...done.
Loaded symbols for /usr/lib/libldap.so.2
Reading symbols from /usr/lib/liblber.so.2...done.
Loaded symbols for /usr/lib/liblber.so.2
Reading symbols from /usr/lib/libsasl.so.7...done.
Loaded symbols for /usr/lib/libsasl.so.7
Reading symbols from /usr/lib/libgdbm.so.2...done.
Loaded symbols for /usr/lib/libgdbm.so.2
Reading symbols from /lib/libpam.so.0...done.
Loaded symbols for /lib/libpam.so.0
Reading symbols from /lib/liblaus.so.1...done.
Loaded symbols for /lib/liblaus.so.1
Reading symbols from /usr/lib/httpd/modules/mod_perl.so...done.
Loaded symbols for /etc/httpd/modules/mod_perl.so
Reading symbols from
/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE/libperl.so...done.
Loaded symbols for
/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE/libperl.so
Reading symbols from /lib/libutil.so.1...done.
Loaded symbols for /lib/libutil.so.1
Reading symbols from /usr/lib/httpd/modules/libphp4.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/libphp4.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/libphp4.so
Reading symbols from /usr/local/lib/libsablot.so.0...done.
Loaded symbols for /usr/local/lib/libsablot.so.0
Reading symbols from /usr/lib/libjs.so...done.
Loaded symbols for /usr/lib/libjs.so
Reading symbols from /usr/lib/libpspell.so.4...done.
Loaded symbols for /usr/lib/libpspell.so.4
Reading symbols from /usr/lib/libpanel.so.5...done.
Loaded symbols for /usr/lib/libpanel.so.5
Reading symbols from /usr/lib/libncurses.so.5...done.
Loaded symbols for /usr/lib/libncurses.so.5
Reading symbols from /usr/lib/libgmp.so.3...done.
Loaded symbols for /usr/lib/libgmp.so.3
Reading symbols from /usr/lib/libpng12.so.0...done.
Loaded symbols for /usr/lib/libpng12.so.0
Reading symbols from /usr/lib/libcurl.so.2...done.
Loaded symbols for /usr/lib/libcurl.so.2
Reading symbols from /usr/lib/libbz2.so.1...done.
Loaded symbols for /usr/lib/libbz2.so.1
Reading symbols from /usr/lib/libttf.so.2...done.
Loaded symbols for /usr/lib/libttf.so.2
Reading symbols from /usr/lib/libfreetype.so.6...done.
Loaded symbols for /usr/lib/libfreetype.so.6
Reading symbols from /usr/lib/libjpeg.so.62...done.
Loaded symbols for /usr/lib/libjpeg.so.62
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libxml2.so.2...done.
Loaded symbols for /usr/lib/libxml2.so.2
Reading symbols from /usr/lib/libstdc++.so.5...done.
Loaded symbols for /usr/lib/libstdc++.so.5
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /usr/lib/libltdl.so.3...done.
Loaded symbols for /usr/lib/libltdl.so.3
Reading symbols from /usr/lib/libpspell-modules.so.1...done.
Loaded symbols for /usr/lib/libpspell-modules.so.1
Reading symbols from /usr/lib/libgpm.so.1...done.
Loaded symbols for /usr/lib/libgpm.so.1
Reading symbols from /usr/lib/httpd/modules/mod_python.so...done.
Loaded symbols for /etc/httpd/modules/mod_python.so
Reading symbols from /usr/lib/httpd/modules/mod_ssl.so...Reading symbols
from /usr/lib/debug/usr/lib/httpd/modules/mod_ssl.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_ssl.so
Reading symbols from /usr/lib/php4/imap.so...Reading symbols from
/usr/lib/debug/usr/lib/php4/imap.so.debug...done.
done.
Loaded symbols for /usr/lib/php4/imap.so
Reading symbols from /usr/lib/php4/ldap.so...Reading symbols from
/usr/lib/debug/usr/lib/php4/ldap.so.debug...done.
done.
Loaded symbols for /usr/lib/php4/ldap.so
Reading symbols from /usr/lib/php4/mysql.so...Reading symbols from
/usr/lib/debug/usr/lib/php4/mysql.so.debug...done.
done.
Loaded symbols for /usr/lib/php4/mysql.so
Reading symbols from /usr/lib/php4/odbc.so...Reading symbols from
/usr/lib/debug/usr/lib/php4/odbc.so.debug...done.
done.
Loaded symbols for /usr/lib/php4/odbc.so
Reading symbols from /usr/lib/libodbc.so.1...done.
Loaded symbols for /usr/lib/libodbc.so.1
Reading symbols from /usr/lib/libodbcpsql.so.2...done.
Loaded symbols for /usr/lib/libodbcpsql.so.2
Reading symbols from /usr/lib/php4/pgsql.so...Reading symbols from
/usr/lib/debug/usr/lib/php4/pgsql.so.debug...done.
done.
Loaded symbols for /usr/lib/php4/pgsql.so
Reading symbols from /usr/lib/libpq.so.3...done.
Loaded symbols for /usr/lib/libpq.so.3
Reading symbols from /usr/lib/python2.2/lib-dynload/structmodule.so...done.
Loaded symbols for /usr/lib/python2.2/lib-dynload/structmodule.so
Reading symbols from /usr/lib/python2.2/lib-dynload/_codecsmodule.so...done.
Loaded symbols for /usr/lib/python2.2/lib-dynload/_codecsmodule.so
#0  0x00000029 in ?? ()
(gdb) thread apply all bt full

Thread 1 (process 53045):
#0  0x00000029 in ?? ()
No symbol table info available.
#1  0x000008e6 in ?? ()
No symbol table info available.
#2  0x0cd3f01c in ?? ()
No symbol table info available.
#3  0x00000010 in ?? ()
No symbol table info available.
#4  0xbfff0a28 in ?? ()
No symbol table info available.
#5  0xbfff0a28 in ?? ()
No symbol table info available.
#6  0x00005000 in ?? ()
No symbol table info available.
#7  0x00005000 in ?? ()
No symbol table info available.
#8  0x00000000 in ?? ()
No symbol table info available.
(gdb) bt full
#0  0x00000029 in ?? ()
No symbol table info available.
#1  0x000008e6 in ?? ()
No symbol table info available.
#2  0x0cd3f01c in ?? ()
No symbol table info available.
#3  0x00000010 in ?? ()
No symbol table info available.
#4  0xbfff0a28 in ?? ()
No symbol table info available.
#5  0xbfff0a28 in ?? ()
No symbol table info available.
#6  0x00005000 in ?? ()
No symbol table info available.
#7  0x00005000 in ?? ()
No symbol table info available.
#8  0x00000000 in ?? ()
No symbol table info available.
(gdb) quit
#

Yet I'm still not seeing anything useful.  Any idea what I'm missing?

-- 
Paul
@ Thy Service

> On 3/8/07, Robert Dale <robdale at gmail.com> wrote:
>> "Before running gdb, be sure that the server is compiled with the -g
>> option in EXTRA_CFLAGS to include the symbol information in the object
>> files."
>>
>> You'll have to recompile and don't let it get 'strip'ped.
>>
>> There is also the off chance that you have bad ram.
>>
>> Do you have another machine to test the additional VHs?
>>
>> --
>> Robert Dale
>>
>>
>> On 3/8/07, Paul G. Szabady <Paul at thyservice.com> wrote:
>> > Thanks!
>> >
>> > So, here's my core dump trace.  Is the "No symbol table info
>> available."
>> > output normal or am I missing one or more parameters?
> You may not need to re-compile, provided your on RedHat or CentOS
> there should be a debuginfo package corresponding to apache.   I've
> never used one, before but I think all you have to do is install it
> and then you have all of apaches symbols.
>
> Robert's right though, in that it lookis like your version of Apache
> has been stripped of all symbols such that all you have are memory
> addresses (which really skilled individuals can do something with, but
> us mere mortals find, well, opaque at best).
>
> Good Luck...james
> --
> TriLUG mailing list        : http://www.trilug.org/mailman/listinfo/trilug
> TriLUG Organizational FAQ  : http://trilug.org/faq/
> TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
>