IPFire network object creator for IPv4 addresses based on ASN information

maloe 7947961c23 update to v0.7.13		2 years ago
LICENSE	54907004b9 Initial commit	7 years ago
README.md	4fe6a844e6 update to v0.7.13	2 years ago
asn_ipfire.sh	7947961c23 update to v0.7.13	2 years ago
asn_ipfire_v0.7.7_beta2_termux.sh	d7fe431298 update to v0.7.7beta2	6 years ago
asn_script.conf	3ab728f368 update to v0.7.13	2 years ago

ASN_IPFire_Script

IPFire network object creator for IPv4 addresses based on ASN information.

The script collects ASN numbers, registered by / assigned to a company and then creates a corresponding list of IPv4 networks. This list of IPv4 networks is then automatically included into IPFire firewall groups (networks and network/host groups). These groups can be used in the IPFire firewall settings to simply block whole company networks.

For detailed description please read the article on Kuketz-Blog: ASN-Skript: Datensammler haben ausgeschnüffelt – IPFire Teil3 or see the Wiki page.

Originally this script was invented and started by Mike Kuketz. He also wrote similar scripts to use same IPv4 networks lists in other output formats to be used directly with tools like iptables and Android AFWall+. These separate tools have been integrated into one single script with further optimizations and additional features.
By default asn_ipfire.sh creates entries for IPFire, but custom specific output options for other applications are possible:

IPFire Groups (default)
custom specific formats (for iptables, AFWall+,...)
pure asn or network list

The script is intended to be run on an IPFire installation, but it is also running on other Linux distributions as well as on Android terminals (root needed).

Outlook to version 0.8+

Compatibility break! From version 0.8 on, the options "--iptable" and "--afwall" will be removed as well as the corresponding default output file names. Results will then be written to the output file "asn_result.lst" by default. Though custom specific files will be still possible via configuration file. These changes are irrelevant for the ipfire mode.

Update to version 0.7.13

cidr-report is back and set as ASN default source again. To get bgplookingglass source work properly again, small redirect adjustments were done.

Update to version 0.7.11

The default ASN source cidr-report seems to be down. Therefore the default source has been switched to bgplookingglass. This has to be considered because it's only available with http protocol! Alternatively the newly added bgpview.io can be selected, which has other drawbacks like no white space support in the company names and it's rarely tested so far.

Update to version 0.7.10

To reduce the risk of cleared network entries, the new option "-c" or "--check" can be used to perform a previous availability check of all activated sources. In case that any source cannot be reached, the script will stop without any changes to the ipfire. This option might be set by default in later versions.
Note: Even with this option activated, wrongly blank network entries can still happen, e.g. if a source is not reachable after the pre-check or if the source contain no data, etc.

Update to version 0.7.9

This version added options for a custom specific output format and output file. Also all script header variables can now be changed in an optional configuration file "asn_script.conf" (default name).

Mainly intended for the IPFire mode, there is now an option "-i" or "--interactive", which allows to stop the script before overwriting existing files.

Option "--network_raw" has been removed. An equivalent output result can be achieved with the option "--network", combined with the new option "--raw". Latter option can also be combined with the other modes.

Update to version 0.7.8

This version fixes a bug that is relevant in IPFire mode. It can lead to a grow up of the customgroups and customnetworks files and to misadjusted firewall rules. It only occurs, if the company names contains spaces (~) or wildcards (*).
Therefore it is recommended to run asn_ipfire.sh --remove ALL once, to clean up these files. Then run asn_script as usually. And finally re-assure that your IPFire firewall rules are still correctly assigned:
IPFire GUI --> Firewall Rules --> Edit all rules with company names --> re-assign the "Network/Host Groups" --> Update --> Apply changes

Files are only written at the end of the script now. This allows a cancellation (CTRL-C) any time before the last company has been fully consolidated.

Update to version 0.7.4+

IPfire firewall rules are case sensitive. This leads to wrong assignment if the script is run with different spellings of company names. To prevent the risk of non-blocking groups due to accidentally misspelling, company names are written uppercase now. This requires once a manual re-assignment inside IPFire firewall rules. Run the script with all company names you want to block and do following steps in IPFire:
IPFire GUI --> Firewall Rules --> Edit all rules with company names --> re-assign the "Network/Host Groups" --> Update --> Apply changes

Update to version 0.7+

If you update from previous versions be aware that the format of COMPANY names has been changed.
Only comma separation is supported now. Company names must no longer be separated by spaces.
This may affect you, if you automatically start the script, e.g. via crontab or an launcher script. Also if you use a company file, which contains space saparated names, you need to adapt it.

The default source for ASN entries has been switched from ultratools to cidr-report . That is because results from cidr-report seemed to be more complete. Because of this change, the script will take a bit longer, depening on the download transfer rate and the number of companies to be processed. See the wiki page how to change the sources.

Company names:

Names needed to be separated by commas (,) (e.g. CampanyA,CompanyB,CompanyC,...)
If spaces are used for better readability, the string must be put into double quotes ("). (e.g. "CompanyA , CompanyB, CompanyC,...")
Any spaces are skipped. (e.g. "Comp AnyA" is searching for "CompAnyA")
To find names with spaces in between, use the tilde (~) sign as substitution for space (e.g. "Comp~AnyA")
Any matches which contain the given name as substring are found.
If the result shall be limited strictly to exact matches, the tilde (~) signs can be used as limiter (e.g. ~CompanyA~).
Asterisk (*) sign is allowed as wildcard, to find names with any characters in between of two name parts. (e.g. "Comp*AnyA")

Usage

Output of asn_ipfire.sh --help :

Usage: asn_ipfire.sh [OPTION] [COMPANYs | -f FILE] [PARAMETER]
Add or remove networks to IPFire firewall Groups: Networks & Host Groups

IPFire-Mode options:
  -a, --add         Add new company networks
  -r, --remove      Remove company networks from customnetworks & customgroups

IPFire-Mode parameter:
  -l, --list        List of companies already added by this script
      --renumber    Renumber lines of customnetworks & customgroups
      --sort        Re-sort lines of customnetworks & customgroups
      --backup      Backup customnetworks & customgroups before change
      --rmbackup    Remove backup files of customnetworks & customgroups
      --restore     Restore customnetworks & customgroups from backup

Non-IPFire-Mode options:
  --network         Create FILE 'network_list.txt' with networks
  --asn             Create FILE 'asn_list.txt' with ASNs only
  --custom          Create FILE 'asn_result.lst' with custom rules

Common parameter:
  -c, --check          Check source for availability
  -f, --file FILE      Get company list from FILE
  -i, --interactive    Ask user before a file will be overwritten
  -k, --keep           Keep temporary source files after finish
  -o, --outfile FILE   Custom result file FILE (Non-IPFire-Mode)
      --raw            Networks not consolidated
  -v, --verbose        Verbose mode
  -V, --version        Show this script version and exit
  -h, --help           Show this help and exit

COMPANY to be one or more company names, put into double quotes ("...")
  Multi company names must be comma separated
  Substitute spaces with tilde (~)
  Restrict to exact matches with tilde (~) before and after the name
  Company names are handled case insensitive.
  example: asn_ipfire.sh --add "CompanyA,Company~NameB,~CompanyC~" 

FILE to be a name of a file, containing one or more company names.
  Company names to be separated by comma or line feed.
  examples: asn_ipfire.sh -a -f company.list 
            asn_ipfire.sh --network -f company.list 

Option --remove only affects entries made by asn_ipfire.sh itself.
  These entries are recognized by the 'Remark'-column in IPFire.
  To remove all entries done by this script, use 'ALL' as COMPANY 
  examples: asn_ipfire.sh -r "CompanyA, CompanyB" 
            asn_ipfire.sh -r ALL