= !DeepNet DNSBL Tools = This is a set of scripts for maintaining your own DNS-based IP and URI blacklists for use either in a cumulative score-based filter system like [http://spamassassin.apache.org SpamAssassin] or (if you trust anyone who has access to the system) hard-blocking in your MTA. The API and UI for both the IP list and the URI list should be considered functional but incomplete. Some changes must be done directly in the database, or a custom script written. These are NOT designed for a major publicly-available list similar to Spamhaus' [http://www.spamhaus.org/zen/ zen], [http://www.spamcop.net/bl.shtml SpamCop], or Barracuda's [http://www.barracudacentral.org/rbl BRBL]. If nothing else, the extra bitmask handling slows things down enough that it would take an unreasonable amount of time to maintain a multi-million-record dataset. As of performance tweaks in v0.2, a local dataset containing ~19000 IPs and ~12000 netblocks, resulting in ~20K internally-published entries, takes about 20 seconds to export to an rbldnsd-formatted file on otherwise lightly-loaded modern hardware. Past versions of the code on the same dataset may run as long as 10 minutes. == IP blacklist == For IP listing, the RIR allocation (from ARIN, APNIC, LACNIC, AfriNIC, or RIPE) and two levels of internal delegation are supported. This allows tracking of how many IPs in a given block have been reported, and automatically listing the entire block if that goes beyond a certain threshold. Additional levels of delegation could be supported with minimal changes. The thresholds are defined in a database table. No UI is currently built to modify these. A fairly reasonable set of defaults (in production use since ~2010) is provided in the initial SQL tabledef. Multiple lists are supported via autodetection of the web URL that will load different database connection information. This allows you to maintain one list for general scoring, and a second with different autolisting thresholds for hard blocking. Data can be exported for various DNS systems. [http://www.corpit.ru/mjt/rbldnsd.html rbldnsd] is recommended as it supports CIDR netblocks rather than forcing divisions along classful octet boundaries (leading to having to add 128 lines for a /17, for instance). Manually tagging a block or netblock owner/operator for listing is possible but no UI has been implemented. An IP can have a bit in a bitmask set for any one of the following: * IP count exceeds automatic threshold (threshold is set for each CIDR block size) * IP count in RIR allocation * IP count in first delegation * IP count in second delegation * Netblock has been manually tagged to be listed (this is useful to tag blocks marked in WHOIS as dynamic IP space - these IPs should not be sending mail directly to your MX) * RIR allocation tagged * First delegation tagged * Second delegation tagged * Netblock owner/operator tagged (useful to catch new IPs assigned to netblock owners who generate lots of spam) * RIR allocation tagged * First delegation tagged * Second delegation tagged [https://secure.deepnet.cx/demos/dnsbl/sa-dnsbl-example.cf Example !SpamAssassin configuration fragment] == URI blacklist == URI listing is much simpler; one or more domains can be added at a time, each at a different listing level or type. Currently supported designations are "black", "grey", and "URL shortener". The supplied !SpamAssassin configuration fragment uses these designations. [https://secure.deepnet.cx/demos/dnsbl/sa-uribl-example.cf Example !SpamAssassin configuration fragment] == Releases == No stable release has been designated. Current code is in production use. You can check out the current code from SVN at https://secure.deepnet.cx/svn/dnsbl/trunk, or download a development snapshot current as of writing from https://secure.deepnet.cx/releases/dnsbl/. The IP blacklist has a [https://secure.deepnet.cx/demos/dnsbl demo]. You can see the entire dataset [https://secure.deepnet.cx/demos/dnsbl/dnsbl/browse.cgi here]. This results in an rbldnsd data file like [https://secure.deepnet.cx/demos/dnsbl/demoexport this] (may be out of date with respect to the listing on browse.cgi). == Configuration == The web UI takes the hostname and root path to the installation, converts non-alphanumerics to underscores, and appends `.conf` - for instance, the demo site above looks for `secure_deepnet_cx_demos_dnsbl.conf` in /etc/dnsbl. This file contains the database connection information. The URI blacklist looks for uribl.conf in /etc/uridb along the same pattern. For export, the export script requires an argument to determine which configuration to pick up. I use symlinks with convenient short names linked to the longer filenames (`dnsbl` -> `secure_deepnet_cx_demos_dnsbl.conf`). Thresholds for automatically listing a block are defined in the `autolist` table. The CIDR mask length is used on the `masklen` column to look up the IP count threshold from the `ipcount` column. Manual tagging of a block or block owner is done by setting the `listme` column in the `blocks` or `orgs` tables respectively to `y`. The `comments` column gives you a place to put notes about the block - I've used this to note an "advisory" block added when I see a grouping of IPs that keep showing up in missed spam but which doesn't show in WHOIS, and to tie together apparently unrelated block owners. It's also useful to remind you why a block might be listed in spite of being part of a squeaky-clean network otherwise (typically for blocks labelled "dynamic" in WHOIS, but not listed on eg Spamhaus' [http://www.spamhaus.org/pbl PBL]). All further configuration relies on the key-value "misc" table in the database. Supported keys are: * blzone (default "`company.dnsbl`")[[br]] DNS zone to publish the data under. A non-public TLD like `.dnsbl` is recommended to insure you don't leak the data beyond your own usage. * bladmin (default "`systems.company.com`")[[br]] Admin contact for the zone's SOA record * iplisted (default "`$ relayed a reported spam`")[[br]] String to use when a TXT lookup is done on an IP that is explicitly listed. `$` is replaced by rbldnsd with the actual IP on the fly. * blocklisted (default "`Netblock listed on one or more criteria`")[[br]] String to use when a TXT lookup is done on an IP in a block that's listed, but the IP itself isn't. The latter two support some customization on a per-entry basis by replacing the literal string `:ENTITY:` with the IP or block being written.