[Sks-devel] Dealing with abusive clients

Post by Pete Stephenson
Hi all,
I've been receiving some queries that, while not stressing my server,
appear to be abusive in nature...though perhaps accidentally so.

This is a good reminder for people writing scripts to always include a
user-agent with a contact email address in it!

Post by Pete Stephenson
216.241.59.205 - - [20/Jul/2017:14:46:51 +0000] "GET / HTTP/1.1" 200
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:53 +0000] "GET / HTTP/1.1" 200
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:56 +0000] "GET / HTTP/1.1" 200
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:58 +0000] "GET / HTTP/1.1" 200
5285 "-" "-"
This particular client is making continuous requests for the main page
of my server every 2-3 seconds. They're not making any queries for keys,
submitting keys, etc., but are only requesting the main page.
This has been going on since at least the 15th of July.
I haven't observed any other odd traffic, so it seems unlikely that a
botnet is involved. Maybe a script that has gone awry?

There might be a clue in the host header if you could log that? I use
this nginx config to do that (and not log the client IP)

```
log_format anonymized '[ip_not_logged] - $http_host [$time_local] '

'"$request" $status $bytes_sent '

'"$http_referer" "$http_user_agent" "$gzip_ratio"';
```

Post by Pete Stephenson
Although slightly annoying, it doesn't consume much resources. Any
suggestions on how to deal with this client? For example, should I
continue to serve them normally, firewall their IP address, etc.?

Hopefully you'll learn something about the intent behind this - maybe
it's an over-zealous keyserver monitoring bot?

I'd be inclined to do that before doing any firewalling or whatever.

Post by Pete Stephenson
Any suggestions on how to deal with more serious abuse in the future?

I'm a big fan of fail2ban if there's a particular log pattern you'd like
to pick up on and temporarily firewall.

On a sillier, more evil note, if you're really sure something's
malicious (for example posting credentials to non-existent phpmyadmin
login pages) you could be a real bastard and have some fun... For the
non-existent Wordpress login page on one website I run, I serve a
ZIP-bomb - a small file which expands to a very large file - and
typically crashes the script / bot that accessed the URL.

I don't really recommend that but I find it quite fun to think of the
script kiddies scratching their heads wondering why their l33t hacker
tool isn't working.

Finally: I personally think it's a nice move not to log client IP
addresses. I got scared looking through my access log in the past about
how much those pks queries reveal about our users.

Paul

Pete Stephenson

2017-07-20 17:07:14 UTC

Post by Pete Stephenson
Hi all,
I've been receiving some queries that, while not stressing my server,
appear to be abusive in nature...though perhaps accidentally so.

This is a good reminder for people writing scripts to always include a
user-agent with a contact email address in it!

Agreed. Or at least a URL describing the bot or script. Ideally, it'd
have contact information for the operator (not just the developer), but
I'll take what I can get.

There might be a clue in the host header if you could log that? I use
this nginx config to do that (and not log the client IP)

Good idea. I'll see if I can tweak the logs.

Post by Paul M Furley
Hopefully you'll learn something about the intent behind this - maybe
it's an over-zealous keyserver monitoring bot?

I contacted the abuse point-of-contact for the ISP that provides their
connectivity. I emphasized that I don't suspect foul play or mailicious
intent, but just a script that is doing something unintended. They were
extremely responsive and are investigating with their client. If they
provide any useful information, I will let the list know (sans personal
information, which they shouldn't be sending me anyway).

Post by Paul M Furley
I'd be inclined to do that before doing any firewalling or whatever.

Me too.

Post by Pete Stephenson
Any suggestions on how to deal with more serious abuse in the future?

I'm a big fan of fail2ban if there's a particular log pattern you'd like
to pick up on and temporarily firewall.

Good idea. I already use that for stopping SSH attacks (though I use
public keys, so they're not getting in anyway).

Post by Paul M Furley
On a sillier, more evil note, if you're really sure something's
malicious (for example posting credentials to non-existent phpmyadmin
login pages) you could be a real bastard and have some fun... For the
non-existent Wordpress login page on one website I run, I serve a
ZIP-bomb - a small file which expands to a very large file - and
typically crashes the script / bot that accessed the URL.
I don't really recommend that but I find it quite fun to think of the
script kiddies scratching their heads wondering why their l33t hacker
tool isn't working.

That is a bit evil, and not something I intend to do while providing
service for the SKS pool. If it were my own private server, maybe.

Post by Paul M Furley
Finally: I personally think it's a nice move not to log client IP
addresses. I got scared looking through my access log in the past about
how much those pks queries reveal about our users.

I agree, in general. I keep logs for only a few days specifically to
identify abuse. I rarely check them (only when other stuff, like
bandwidth statistics or general patterns of access frequency, large
numbers of keys getting submitted, etc. indicates something unusual) and
they're automatically deleted after a few days. To me, this strikes a
reasonable balance between user privacy and detection and prevention of
abuse, though I've considered rotating logs after one day vs. five days.
We'll see.

I figure that if users are particularly concerned about their privacy
when querying SKS servers, they can use Tor. My server is also
accessible directly over tor, and I also log Tor accesses to my server
to detect abuse, but obviously, IP addresses are not logged (they only
show up as localhost). I make no efforts at all to combine or analyze
the data other than getting a sense for what percentage of total traffic
is Tor vs. non-Tor.

Cheers!
-Pete

Valentin Sundermann

2017-07-20 17:18:52 UTC

I see these requests too, but from a different IP. I noticed them 1-2
months ago but wasn't able to find the origin of these requests (they
got sorted into a general logfile because of the "missing" Host field).

The IP that is querying my server belongs to Amazon's AWS. Requests look
the same, every 2 seconds a "GET /".

Post by Paul M Furley
There might be a clue in the host header if you could log that? I use
this nginx config to do that (and not log the client IP)

Good idea. I'll see if I can tweak the logs.

I log HTTP Host headers and it uses localhost in each requests. Still no
idea what this could be.

Best regards,
Valentin Sundermann

Pete Stephenson

2017-07-20 17:59:31 UTC

Post by Valentin Sundermann
I see these requests too, but from a different IP. I noticed them 1-2
months ago but wasn't able to find the origin of these requests (they
got sorted into a general logfile because of the "missing" Host field).

Interesting. That suggests something a bit more unusual than a simple
misconfigured script, though it's oddly tailored for SKS since it's
using port 11371 rather than port 80 (I log requests for 80 and 11371,
as well as HTTPS on 443, and Tor separately).

Post by Valentin Sundermann
I log HTTP Host headers and it uses localhost in each requests. Still no
idea what this could be.

I also started logging the Host header. It's also using "localhost" as
the host. Very strange.

Kristian Fiskerstrand

2017-07-20 18:05:24 UTC

Post by Valentin Sundermann

Post by Pete Stephenson
216.241.59.205 - - [20/Jul/2017:14:46:51 +0000] "GET / HTTP/1.1"

200

Post by Pete Stephenson
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:53 +0000] "GET / HTTP/1.1"

200

Post by Pete Stephenson
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:56 +0000] "GET / HTTP/1.1"

200

Post by Pete Stephenson
5285 "-" "-"
216.241.59.205 - - [20/Jul/2017:14:46:58 +0000] "GET / HTTP/1.1"

200

Post by Pete Stephenson
5285 "-" "-"
This particular client is making continuous requests for the main

page

Post by Pete Stephenson
of my server every 2-3 seconds. They're not making any queries for

keys,

Post by Pete Stephenson
submitting keys, etc., but are only requesting the main page.
This has been going on since at least the 15th of July.
I haven't observed any other odd traffic, so it seems unlikely that

Post by Pete Stephenson
botnet is involved. Maybe a script that has gone awry?

I see these requests too, but from a different IP. I noticed them 1-2
months ago but wasn't able to find the origin of these requests (they
got sorted into a general logfile because of the "missing" Host field).
The IP that is querying my server belongs to Amazon's AWS. Requests look
the same, every 2 seconds a "GET /".

Post by Paul M Furley
There might be a clue in the host header if you could log that? I

use

Post by Paul M Furley
this nginx config to do that (and not log the client IP)

Good idea. I'll see if I can tweak the logs.

I log HTTP Host headers and it uses localhost in each requests. Still no
idea what this could be.
Best regards,
Valentin Sundermann

Ditto, I'm also seeing similar requests from amazon ec2
--
----------------------------
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
----------------------------
Public OpenPGP certificate at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3

Pascal Levasseur

2017-07-20 16:33:38 UTC