[ale] Ouch, dang it.

[DJPfulio at jdpfu.com]

If the Twiki is just for you, just allow localhost and use an ssh SOCKS proxy to access it, just like we all do for DBMS server management. Just put ssh on a high port or only allow your static IP from home access into it, assuming you have a static IP at home, of course.

I'm pretty ruthless about bots and about 150 user-agents or any user-agent that claims to be an old version of any browser or referrers that try to directly access content remotely using an iframe.

Often, it is easier to just throttle requests from a single IP. Nginx makes it easy.
https://nginx.org/en/docs/http/ngx_http_limit_req_module.html  That's a failsafe for any "bad requests" that don't get blocked before this point.

I hate the idea of needing to block access, but there are practicalities that cannot be ignored, especially with all the abuse these days.


On 5/29/25 18:31, lollipopman691 via Ale wrote:
> I run a small TWiki server which is in robots.txt on an aws instance.
> Recently that VM started to become unstable.  Today I logged on and
> found that the disk was completely full up. It normally runs about
> 85% full. After poking around a bit I found that the TWiki access
> logs for the last few days were multiple gigabytes in size. Further,
> someone or something was requesting a single page on my TWiki over
> and over at a prodigious rate.  I use that instance as a forwarding
> email server, so it's critical that it stays on line. So I took the
> simplest course and shut httpd off, removing all my web content from
> view for now, including a bunch of recipes I use weekly. Dang it.
> 
> I grabbed today's log file and did some simple shell scripting on it
> to try to figure out what was going on.  It looks like the requests
> are coming at over 200 times a minute from a variety of addresses in
> the far east, at least according to https://www.iplocation.net/ .
> 
> My last TWiki log has requests from about 70,000 ip addresses for
> that one TWiki page. About 90% of them are hitting the page only
> once. Most of the rest are hitting it twice. A handful are over 100,
> with the largest at around 700.  I nmap(1) ed a couple of them for
> fun. The one which appeared to be up ( 47.239.152.3 ) showed:
> 
> PORT     STATE  SERVICE 80/tcp   closed http 443/tcp  closed https 
> 3389/tcp closed ms-wbt-server
> 
> Mildly interesting.  The Net of 10,000 lies claims that
> "ms-wbt-server" is a Microsoft remote desktop server, so at a guess
> I'd say this was a compromised Windows machine.
> 
> Has anyone seen this kind of thing before?  I currently plan to leave
> httpd down for a few days and then restart it and see if this trouble
> has gone away. I reckon the long-term solution is to move my mail
> server off the web machine and then just let it do its thing?
> 
> -- CHS