[irc/freenode/web-7.0.git] / content / news / 2018-09-24-spamfilter.md

---
author: edk
date: 2018-09-24 17:47:00+00:00
slug: spamfilter
title: Spam filtering
category: freenode
category: announcements
---

Hi,

As most of you are aware, we've been experiencing significant spam over the
past few weeks. As a result, we have decided to roll out a server-side spam
filter. Unlike our current spam-mitigating techniques, this system applies to
private messages and does not let the first matching line get through.

Various ethical concerns have been raised over the course of introducing this
feature. They'll be addressed below. The short version, though, is that the
system has various limitations built in designed to prevent operator abuse.
Only a tiny bit of information can get out of the filters, and they do not have
access to much information themselves, to the extent that we believe the
obvious ways to abuse such a feature are impractical.

We've historically been reluctant to take steps like this, and we remain so,
but we believe the disruption has reached such a level that this is necessary
to allow the communities using freenode to collaborate effectively. The prior
complement of anti-spam measures represents our preferred approach, and we
intend to employ this only when they prove insufficient to minimize disruption.

For the technically inclined, you can view the changes
[here](https://github.com/freenode/ircd-seven/compare/000aa02980b812a18652fc2cc21041eb7adc980b...6ffdaf49400643a922fc9086f7a645ddd07013a6).


- _Could this be used to spy on users?_ Which filter a user matched is not
  reported to staff, only that one did. This limits the theoretical maximum
  rate of passive monitoring to one bit per message, far less than the
  information content of conversations.

    Recipients of private messages are not included in the line that filters
    match on, so staff cannot use spam filters to see who is talking to whom.

- _Could this be used to shadowban users?_ No. If a filter blocks a message,
  its sender is either disconnected from the network or sent an error message.

    Currently, the filter system is configured not to use the nick, username, or
    hostname for filter matching, so it can't discriminate against particular
    users at all.


The exact information filters "see" is as follows:

- The type of message (PRIVMSG/NOTICE)
- The target of the message, if that target is a channel. For private
  messages, filters can see that they are PMs but not who their target is.
- Whether or not the sender is identified (but not their account name)
- The full contents of the message

The code can be configured to filter on the `nick!user@host` of the sender. We
haven't enabled this, and have no current plans to, but this is subject to
change should the nature of the spam demand it.

Filtering is always performed on the server originating a message, and inside
the ircd process. This system will never cause a message to be distributed
more widely than before.

Staff can, as always, answer your questions about this change, and we welcome
constructive feedback. Private messages to staff are not subject to filtering.
Commit	Line	Data
99ffeb40 EK	1	---
	2	author: edk
	3	date: 2018-09-24 17:47:00+00:00
	4	slug: spamfilter
	5	title: Spam filtering
	6	category: freenode
	7	category: announcements
	8	---
	9
	10	Hi,
	11
	12	As most of you are aware, we've been experiencing significant spam over the
	13	past few weeks. As a result, we have decided to roll out a server-side spam
	14	filter. Unlike our current spam-mitigating techniques, this system applies to
	15	private messages and does not let the first matching line get through.
	16
	17	Various ethical concerns have been raised over the course of introducing this
	18	feature. They'll be addressed below. The short version, though, is that the
	19	system has various limitations built in designed to prevent operator abuse.
	20	Only a tiny bit of information can get out of the filters, and they do not have
	21	access to much information themselves, to the extent that we believe the
	22	obvious ways to abuse such a feature are impractical.
	23
	24	We've historically been reluctant to take steps like this, and we remain so,
	25	but we believe the disruption has reached such a level that this is necessary
a498e01e EK	26	to allow the communities using freenode to collaborate effectively. The prior
a498e01e EK	27	complement of anti-spam measures represents our preferred approach, and we
f7f6f77c	28	intend to employ this only when they prove insufficient to minimize disruption.
99ffeb40	29
9c6b18df EK	30	For the technically inclined, you can view the changes
9c6b18df EK	31	[here](https://github.com/freenode/ircd-seven/compare/000aa02980b812a18652fc2cc21041eb7adc980b...6ffdaf49400643a922fc9086f7a645ddd07013a6).
3090c82a	32
99ffeb40 EK	33
	34	- _Could this be used to spy on users?_ Which filter a user matched is not
	35	reported to staff, only that one did. This limits the theoretical maximum
	36	rate of passive monitoring to one bit per message, far less than the
	37	information content of conversations.
	38
a498e01e EK	39	Recipients of private messages are not included in the line that filters
a498e01e EK	40	match on, so staff cannot use spam filters to see who is talking to whom.
99ffeb40 EK	41
	42	- _Could this be used to shadowban users?_ No. If a filter blocks a message,
	43	its sender is either disconnected from the network or sent an error message.
	44
a498e01e EK	45	Currently, the filter system is configured not to use the nick, username, or
	46	hostname for filter matching, so it can't discriminate against particular
	47	users at all.
99ffeb40 EK	48
	49
	50	The exact information filters "see" is as follows:
	51
	52	- The type of message (PRIVMSG/NOTICE)
	53	- The target of the message, if that target is a channel. For private
	54	messages, filters can see that they are PMs but not who their target is.
	55	- Whether or not the sender is identified (but not their account name)
	56	- The full contents of the message
	57
	58	The code can be configured to filter on the `nick!user@host` of the sender. We
	59	haven't enabled this, and have no current plans to, but this is subject to
	60	change should the nature of the spam demand it.
	61
	62	Filtering is always performed on the server originating a message, and inside
	63	the ircd process. This system will never cause a message to be distributed
	64	more widely than before.
	65
	66	Staff can, as always, answer your questions about this change, and we welcome
	67	constructive feedback. Private messages to staff are not subject to filtering.