[TriLUG] help with fuser/ssh reporting lots of processes

Matt Flyer via TriLUG trilug at trilug.org
Mon Jul 18 10:00:13 EDT 2016


Unless you have a reason to want connections from folks in that part of
the world, I would consider adding the whole 221.224.0.0 -
221.231.255.255 range to your blacklist.

Unfortunately, the IPV4 space is Swiss Cheese and it changes
frequently, but finding some of the big blocks of the offenders space
can help cut down on some of the noise.
 
On Mon, 2016-07-18 at 09:47 -0400, Ken MacKenzie via TriLUG wrote:
> I get lots of those, I hope it goes without saying you should use
> fail2ban
> to catch repeat brute force offenders.
> 
> I have 2 jail setups, a quick ban and a repeat offender ban.  On
> occasion I
> check the logs and I add problem IP addresses to the blrules file (I
> use
> shorewall for my firewall setup).
> 
> /etc/shorewall/blrules is the black list file for shorewall.  Once in
> there
> it is a permanent ban on the offending IP address.  I eventually need
> to
> automate that process but for now the occasional continued trouble
> maker is
> not a big deal to handle manually.
> 
> Ken
> 
> On Mon, Jul 18, 2016 at 8:55 AM, Tim Jowers via TriLUG <trilug at trilug
> .org>
> wrote:
> 
> > 
> > Thank you William and Roy,
> > 
> >   I see lots of things in /var/log/secure now! Tons of filed logins
> > from
> > China as I changed password and removed root from ssh login access.
> > Duh! I
> > know, slacker for not doing that before. Now my fuser ssh/tcp only
> > shows
> > one process and no others are showing up. Using an ssh key for ssh
> > from now
> > on and realize I can't be sloppy as crackers are out there.  (I
> > wonder if
> > there is some lawsuit which could be filed against this outfit as I
> > am sure
> > they have US customers.)
> > E.g. /var/log/secure has:
> > 
> > Jul 18 10:02:12 test1 sshd[29252]: Failed password for root from
> > 221.229.172.99 port 31660 ssh2
> > 
> > Jul 18 10:02:14 test1 sshd[29254]: Failed password for root from
> > 112.85.42.99 port 22571 ssh2
> > 
> > Jul 18 10:02:14 test1 sshd[29252]: Failed password for root from
> > 221.229.172.99 port 31660 ssh2
> > 
> > Have a great week everyone,
> > 
> > Tim
> > 
> > 
> > 
> > 
> > On Mon, Jul 18, 2016 at 8:36 AM, Roy Vestal <rvestal at trilug.org>
> > wrote:
> > 
> > > 
> > > Hi Tim,
> > > 
> > > I did a quick whois on the 221.229.172.99 and found that is is a
> > > Chinese
> > > IP:
> > > 
> > > inetnum:        221.224.0.0 - 221.231.255.255
> > > netname:        CHINANET-JS
> > > descr:          CHINANET jiangsu province network
> > > descr:          China Telecom
> > > descr:          A12,Xin-Jie-Kou-Wai Street
> > > descr:          Beijing 100088
> > > country:        CN
> > > admin-c:        CH93-AP
> > > tech-c:         CJ186-AP
> > > mnt-by:         APNIC-HM
> > > mnt-lower:      MAINT-CHINANET-JS
> > > mnt-routes:     MAINT-CHINANET-JS
> > > remarks:        This object can only modify by APNIC hostmaster
> > > remarks:        If you wish to modify this object details please
> > > remarks:        send email to hostmaster at apnic.net with your
> > > remarks:        organisation account name in the subject line.
> > > status:         ALLOCATED PORTABLE
> > > source:         APNIC
> > > mnt-irt:        IRT-CHINANET-CN
> > > changed:        hm-changed at apnic.net 20030626
> > > 
> > > irt:            IRT-CHINANET-CN
> > > address:        No.31 ,jingrong street,beijing
> > > address:        100032
> > > e-mail:         anti-spam at ns.chinanet.cn.net
> > > abuse-mailbox:  anti-spam at ns.chinanet.cn.net
> > > admin-c:        CH93-AP
> > > tech-c:         CH93-AP
> > > auth:           # Filtered
> > > mnt-by:         MAINT-CHINANET
> > > changed:        anti-spam at ns.chinanet.cn.net 20101115
> > > source:         APNIC
> > > 
> > > role:           CHINANET JIANGSU
> > > address:        260 Zhongyang Road,Nanjing 210037
> > > country:        CN
> > > phone:          +86-25-86588231
> > > phone:          +86-25-86588745
> > > fax-no:         +86-25-86588104
> > > e-mail:         ip at jsinfo.net
> > > remarks:        send anti-spam reports to spam at jsinfo.net
> > > remarks:        send abuse reports to abuse at jsinfo.net
> > > remarks:        times in GMT+8
> > > admin-c:        CH360-AP
> > > tech-c:         CS306-AP
> > > tech-c:         CN142-AP
> > > nic-hdl:        CJ186-AP
> > > remarks:        www.jsinfo.net
> > > notify:         ip at jsinfo.net
> > > mnt-by:         MAINT-CHINANET-JS
> > > changed:        dns at jsinfo.net 20090831
> > > changed:        ip at jsinfo.net 20090831
> > > changed:        hm-changed at apnic.net 20090901
> > > source:         APNIC
> > > changed:        hm-changed at apnic.net 20111114
> > > 
> > > person:         Chinanet Hostmaster
> > > nic-hdl:        CH93-AP
> > > e-mail:         anti-spam at ns.chinanet.cn.net
> > > address:        No.31 ,jingrong street,beijing
> > > address:        100032
> > > phone:          +86-10-58501724
> > > fax-no:         +86-10-58501724
> > > country:        CN
> > > changed:        dingsy at cndata.com 20070416
> > > changed:        zhengzm at gsta.com 20140227
> > > mnt-by:         MAINT-CHINANET
> > > source:         APNIC
> > > 
> > > % Information related to '221.228.0.0/14AS23650'
> > > 
> > > route:          221.228.0.0/14
> > > descr:          CHINANET jiangsu province network
> > > country:        CN
> > > origin:         AS23650
> > > mnt-by:         MAINT-CHINANET-JS
> > > changed:        ip at jsinfo.net 20030630
> > > source:         APNIC
> > > 
> > > 
> > > Try an lsof of each port and see what is using the tcp
> > > connections:
> > > 
> > > $> lsof -i :48079
> > > 
> > > You should see the command, pid, and user that is using that
> > > port. From
> > > there you could use the lsof command again to see what spawned
> > > the
> > session:
> > > 
> > > 
> > > lsof -p PID (replace PID with the actual PID in the response)
> > > 
> > > 
> > > From there you might be able to determine what is creating the
> > > ssh
> > > connection.
> > > 
> > > HTH,
> > > 
> > > -Roy
> > > 
> > > 
> > > 
> > > On 7/18/16 8:19 AM, Tim Jowers via TriLUG wrote:
> > > 
> > > > 
> > > > Hi,
> > > > 
> > > >    I run these two less than a second apart:
> > > > 
> > > > [root at test1 log]# fuser ssh/tcp
> > > > 
> > > > ssh/tcp:               685  5066  5283  5284  5289  5290  5291 
> > > >  5292
> > 5293
> > > 
> > > > 
> > > > 5294
> > > > 
> > > > [root at test1 log]# fuser ssh/tcp
> > > > 
> > > > ssh/tcp:               685  5066  5289  5290  5293  5294  5296 
> > > >  5297
> > 5298
> > > 
> > > > 
> > > > 5299
> > > > 
> > > > 
> > > >    Any ideas how to troubleshoot?   I think I have some Chinese
> > > > search
> > bot
> > > 
> > > > 
> > > > malware based on this:
> > > > 
> > > > [root at test1 log]# lsof -i
> > > > 
> > > > COMMAND   PID   USER   FD   TYPE     DEVICE SIZE/OFF NODE NAME
> > > > 
> > > > sshd      685   root    3u  IPv6  350221175      0t0  TCP *:ssh
> > > > (LISTEN)
> > > > 
> > > > sshd      685   root    4u  IPv4  350221177      0t0  TCP *:ssh
> > > > (LISTEN)
> > > > 
> > > > mysqld    811  mysql   10u  IPv4  350221673      0t0  TCP
> > > > *:mysql
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > sshd     5066   root    3r  IPv4 4054471422      0t0  TCP
> > > > 198-20-184-56-host.colocrossing.com:ssh->
> > > > cpe-45-37-198-154.nc.res.rr.com:59006 (ESTABLISHED)
> > > > 
> > > > sshd     5361   root    3r  IPv4 4054875967      0t0  TCP
> > > > 198-20-184-57-host.colocrossing.com:ssh->112.85.42.99:41796
> > (ESTABLISHED)
> > > 
> > > > 
> > > > 
> > > > sshd     5362   sshd    3u  IPv4 4054875967      0t0  TCP
> > > > 198-20-184-57-host.colocrossing.com:ssh->112.85.42.99:41796
> > (ESTABLISHED)
> > > 
> > > > 
> > > > 
> > > > sshd     5365   root    3r  IPv4 4054877149      0t0  TCP
> > > > 198-20-184-56-host.colocrossing.com:ssh-
> > > > >112.85.42.99:openmailpxy
> > > > (ESTABLISHED)
> > > > 
> > > > sshd     5366   sshd    3u  IPv4 4054877149      0t0  TCP
> > > > 198-20-184-56-host.colocrossing.com:ssh-
> > > > >112.85.42.99:openmailpxy
> > > > (ESTABLISHED)
> > > > 
> > > > sshd     5369   root    3r  IPv4 4054886185      0t0  TCP
> > > > 198-20-184-56-host.colocrossing.com:ssh->221.229.172.99:36122
> > > > (ESTABLISHED)
> > > > 
> > > > sshd     5370   sshd    3u  IPv4 4054886185      0t0  TCP
> > > > 198-20-184-56-host.colocrossing.com:ssh->221.229.172.99:36122
> > > > (ESTABLISHED)
> > > > 
> > > > sshd     5371   root    3r  IPv4 4054886747      0t0  TCP
> > > > 198-20-184-57-host.colocrossing.com:ssh->221.229.172.99:15096
> > > > (ESTABLISHED)
> > > > 
> > > > sshd     5372   sshd    3u  IPv4 4054886747      0t0  TCP
> > > > 198-20-184-57-host.colocrossing.com:ssh->221.229.172.99:15096
> > > > (ESTABLISHED)
> > > > 
> > > > java    18216   root   43u  IPv6 3405192816      0t0  TCP
> > > > *:webcache
> > > > (LISTEN)
> > > > 
> > > > java    18216   root   48u  IPv6 3405192820      0t0  TCP
> > > > *:8009
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > java    18216   root   72u  IPv6 3405192937      0t0  TCP
> > > > localhost.localdomain:8005 (LISTEN)
> > > > 
> > > > httpd   26003 apache    3u  IPv6 3253453758      0t0  TCP
> > > > *:http
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > httpd   26361 apache    3u  IPv6 3253453758      0t0  TCP
> > > > *:http
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > httpd   27165 apache    3u  IPv6 3253453758      0t0  TCP
> > > > *:http
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > httpd   27818   root    3u  IPv6 3253453758      0t0  TCP
> > > > *:http
> > (LISTEN)
> > > 
> > > > 
> > > > 
> > > > and
> > > > 
> > > > [root at test1 log]# netstat -a
> > > > 
> > > > Active Internet connections (servers and established)
> > > > 
> > > > Proto Recv-Q Send-Q Local Address               Foreign Address
> > > > State
> > > > 
> > > > tcp        0      0 *:ssh                       *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 *:mysql                     *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 198-20-184-57-host.colo:ssh
> > > > 112.85.42.99:15265
> > > > ESTABLISHED
> > > > 
> > > > tcp        0      0 198-20-184-56-host.colo:ssh
> > > > 221.229.172.99:48079
> > > > TIME_WAIT
> > > > 
> > > > tcp        0      0 198-20-184-56-host.colo:ssh
> > > > 221.229.172.99:33195
> > > > ESTABLISHED
> > > > 
> > > > tcp        0      0 198-20-184-57-host.colo:ssh
> > > > 221.229.172.99:44556
> > > > ESTABLISHED
> > > > 
> > > > tcp        0      0 198-20-184-57-host.colo:ssh
> > > > 221.229.172.99:15096
> > > > TIME_WAIT
> > > > 
> > > > tcp        0    608 198-20-184-56-host.colo:ssh
> > > > cpe-45-37-198-154.nc.:59006
> > > > ESTABLISHED
> > > > 
> > > > tcp        0      0 198-20-184-56-host.colo:ssh
> > > > 112.85.42.99:42180
> > > > ESTABLISHED
> > > > 
> > > > tcp        0      0 *:webcache                  *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 *:http                      *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 *:ssh                       *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 localhost.localdomain:8005  *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 *:8009                      *:*
> > > > LISTEN
> > > > 
> > > > tcp        0      0 198-20-184-56-host.col:http
> > > > ns336619.ip-37-187-16:18286
> > > > TIME_WAIT
> > > > 
> > > > tcp        0      0 198-20-184-56-host.col:http
> > > > hydrogen081.a.ahrefs.:30831
> > > > TIME_WAIT
> > > > 
> > > > and some StackOverflow article where someone posted that
> > *221.229.172.99*
> > > 
> > > > 
> > > > is a Chinese search botnet.
> > > > 
> > > > last and lastlog don't show anything. There is no
> > > > /var/log/auth.log
> > > > present. Not sure if there should be. Just tried things based
> > > > on
> > Internet
> > > 
> > > > 
> > > > searching.
> > > > 
> > > > I guess there is no easy way to kill this?  Sounds like I
> > > > should just
> > ask
> > > 
> > > > 
> > > > for a new server instance (ChicagoVPS)? I use SVN to back up my
> > > > files
> > > > there.
> > > > 
> > > > 
> > > > Thanks for any ideas.
> > > > 
> > > > Tim
> > > > 
> > > 
> > --
> > This message was sent to: Ken M. <ken at mack-z.com>
> > To unsubscribe, send a blank message to trilug-leave at trilug.org
> > from that
> > address.
> > TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> > Unsubscribe or edit options on the web  :
> > http://www.trilug.org/mailman/options/trilug/ken%40mack-z.com
> > Welcome to TriLUG: http://trilug.org/welcome
> > 


More information about the TriLUG mailing list