README.md 10 KB

ASN_IPFire_Script

IPFire network object creator for IPv4 addresses based on ASN information.

The script collects ASN numbers, registered by / assigned to a company and then creates a corresponding list of IPv4 networks. This list of IPv4 networks is then automatically included into IPFire firewall groups (networks and network/host groups). These groups can be used in the IPFire firewall settings to simply block whole company networks.

For detailed description please read the article on Kuketz-Blog: ASN-Skript: Datensammler haben ausgeschnüffelt – IPFire Teil3 or see the Wiki page.

Originally this script was invented and started by Mike Kuketz. He also wrote similar scripts to use same IPv4 networks lists in other output formats to be used directly with tools like iptables and Android AFWall+. These separate tools have been integrated into one single script with further optimizations and additional features.
By default asn_ipfire.sh creates entries for IPFire, but custom specific output options for other applications are possible:

  • IPFire Groups (default)
  • custom specific formats (for iptables, AFWall+,...)
  • pure asn or network list

The script is intended to be run on an IPFire installation, but it is also running on other Linux distributions as well as on Android terminals (root needed).

Outlook to version 0.8+

Compatibility break! From version 0.8 on, the options "--iptable" and "--afwall" will be removed as well as the corresponding default output file names. Results will then be written to the output file "asn_result.lst" by default. Though custom specific files will be still possible via configuration file. These changes are irrelevant for the ipfire mode.

Update to version 0.7.9

This version added options for a custom specific output format and output file. Also all script header variables can now be changed in an optional configuration file "asn_script.conf" (default name).

Mainly intended for the IPFire mode, there is now an option -i or --interactive, which allows to stop the script before overwriting existing files.

Option "--network_raw" has been removed. An equivalent output result can be achieved with the option "--network", combined with the new option "--raw". Latter option can also be combined with the other modes.

Update to version 0.7.8

This version fixes a bug that is relevant in IPFire mode. It can lead to a grow up of the customgroups and customnetworks files and to misadjusted firewall rules. It only occurs, if the company names contains spaces (~) or wildcards (*).
Therefore it is recommended to run asn_ipfire.sh --remove ALL once, to clean up these files. Then run asn_script as usually. And finally re-assure that your IPFire firewall rules are still correctly assigned:
IPFire GUI --> Firewall Rules --> Edit all rules with company names --> re-assign the "Network/Host Groups" --> Update --> Apply changes

Files are only written at the end of the script now. This allows a cancellation (CTRL-C) any time before the last company has been fully consolidated.

Update to version 0.7.4+

IPfire firewall rules are case sensitive. This leads to wrong assignment if the script is run with different spellings of company names. To prevent the risk of non-blocking groups due to accidentally misspelling, company names are written uppercase now. This requires once a manual re-assignment inside IPFire firewall rules. Run the script with all company names you want to block and do following steps in IPFire:
IPFire GUI --> Firewall Rules --> Edit all rules with company names --> re-assign the "Network/Host Groups" --> Update --> Apply changes

Update to version 0.7+

If you update from previous versions be aware that the format of COMPANY names has been changed.
Only comma separation is supported now. Company names must no longer be separated by spaces.
This may affect you, if you automatically start the script, e.g. via crontab or an launcher script. Also if you use a company file, which contains space saparated names, you need to adapt it.

The default source for ASN entries has been switched from ultratools to cidr-report . That is because results from cidr-report seemed to be more complete. Because of this change, the script will take a bit longer, depening on the download transfer rate and the number of companies to be processed. See the wiki page how to change the sources.

Company names:

  • Names needed to be separated by commas (,) (e.g. CampanyA,CompanyB,CompanyC,...)
  • If spaces are used for better readability, the string must be put into double quotes ("). (e.g. "CompanyA , CompanyB, CompanyC,...")
  • Any spaces are skipped. (e.g. "Comp AnyA" is searching for "CompAnyA")
  • To find names with spaces in between, use the tilde (~) sign as substitution for space (e.g. "Comp~AnyA")
  • Any matches which contain the given name as substring are found.
  • If the result shall be limited strictly to exact matches, the tilde (~) signs can be used as limiter (e.g. ~CompanyA~).
  • Asterisk (*) sign is allowed as wildcard, to find names with any characters in between of two name parts. (e.g. "Comp*AnyA")


    Usage

    Output of asn_ipfire.sh --help :

Usage: asn_ipfire.sh [OPTION] [COMPANYs | -f FILE] [PARAMETER]
Add or remove networks to IPFire firewall Groups: Networks & Host Groups

IPFire-Mode options:
  -a, --add         Add new company networks
  -r, --remove      Remove company networks from customnetworks & customgroups

IPFire-Mode parameter:
  -l, --list        List of companies already added by this script
      --renumber    Renumber lines of customnetworks & customgroups
      --sort        Re-sort lines of customnetworks & customgroups
      --backup      Backup customnetworks & customgroups before change
      --rmbackup    Remove backup files of customnetworks & customgroups
      --restore     Restore customnetworks & customgroups from backup

Non-IPFire-Mode options:
  --network         Create FILE 'network_list.txt' with networks
  --asn             Create FILE 'asn_list.txt' with ASNs only
  --custom          Create FILE 'asn_result.lst' with custom rules
  --iptable         Create FILE 'iptable_rules.txt' with iptables rules
  --afwall          Create FILE 'afwall_rules.txt' with afwall rules

Common parameter:
  -f, --file FILE      Get company list from FILE
  -i, --interactive    Ask user before a file will be overwritten
  -k, --keep           Keep temporary source files after finish
  -o, --outfile FILE   Custom result file FILE (Non-IPFire-Mode)
      --raw            Networks not consolidated
  -v, --verbose        Verbose mode
  -V, --version        Show this script version and exit
  -h, --help           Show this help and exit

COMPANY to be one or more company names, put into double quotes ("...")
  Multi company names must be comma separated
  Substitute spaces with tilde (~)
  Restrict to exact matches with tilde (~) before and after the name
  Company names are handled case insensitive.
  example: asn_ipfire.sh --add "CompanyA,Company~NameB,~CompanyC~" 

FILE to be a name of a file, containing one or more company names.
  Company names to be separated by comma or line feed.
  examples: asn_ipfire.sh -a -f company.list 
            asn_ipfire.sh --network -f company.list 

Option --remove only affects entries made by asn_ipfire.sh itself.
  These entries are recognized by the 'Remark'-column in IPFire.
  To remove all entries done by this script, use 'ALL' as COMPANY 
  examples: asn_ipfire.sh -r "CompanyA, CompanyB" 
            asn_ipfire.sh -r ALL 

ASN IPFire Script Wiki

Find more useful information in the Wiki.


License

This script is under GNU GPL v3

Copyright 2017-2019 Mike Kuketz, maloe

To support open source software and fair use, kindly refer to the origin authors and source if you copy and modify this script (i.e. keep the origin header).


Change log

v0.7.9 (2019-12-21)

  • added optional configuration file
  • added custom output format
  • added custom output file
  • added option "--raw"
  • removed option "--network_raw"
  • added interactive switch

v0.7.8 (2019-05-06)

  • fixed a bug in ipfire mode, with company names containing spaces or wildcards
  • changed: files are written only at the end of the script
  • speed up option "--asn"
  • added warning in ipfire mode, in case of single IPs
  • added md5 and sha1 hashes
  • code optimization

v0.7.7 beta (2019-03-18)

  • added compatibility with FreeBSD, MacOS (thanks to MyLogin)
  • added a termux version for Android as separate script (thanks to MyLogin)

v0.7.6 (2018-09-24)

  • fixed adjacent consolidation function
  • code optimization

v0.7.5 (2018-09-23)

  • bug: adjacent consolidation function deactivated
  • code optimizations (thanks to Dimitar)

v0.7.4 (2018-02-11)

  • auto upper-case group names in ipfire
  • fixed issue with some wget versions
  • added variable to use curl over wget
  • added option --sort
  • code optimization
  • added fair use hints

v0.7.3 (2017-11-25)

  • fixed not working option --file

v0.7.2 (2017-11-19)

  • improved query

v0.7.1 (2017-11-19)

  • bugfix

v0.7.0 (2017-11-19)

  • changed default ASN source to cidr-report
  • added whois query to alternative sources
  • optimized company name handling
  • simplified source selection
  • speed up static sources
  • auto select wget or curl
  • disabled networks ending with prefix "/0"
  • added simple backup/restore function
  • changed option -v (verbose) and -V (version)
  • added options keep, restore, backup, rmbackup

v0.6.3 (2017-08-30) (beta only)

  • added support for companies with space in the name
  • forced https only for wget

v0.6.2 (2017-08-25)

  • exchanged curl with wget
  • added info about result file in non-ipfire mode
  • fixed alternative sources functions

v0.6.1 (2017-08-22)

  • bugfix on some systems

v0.6.0 (2017-08-21)

  • fixed missing networks on 32 bit systems
  • fixed network_raw mode
  • added stats function via verbose parameter
  • eliminated bc dependency
  • added function for network-source ipinfo.io
  • code optimization (ip filter from source, cosmetics)
  • switched mathematics to work on 32 bit

v0.5.2 (2017-06-19)

  • first public beta
  • added consolidation of dublicate and adjacents networks
  • integrated ipfire, afwall and iptables output into one script
  • added different features