gfcptun: A fast and low-latency tunnel using GFCP over UDP. https://github.com/johnsonjh/gfcptun/

Jeffrey H. Johnson c66ede94f5 Merge pull request #61 from johnsonjh/renovate/github.com-johnsonjh-gfsmux-digest 2 years ago
client 9225050fa8 Update dependencies, import paths, etc. 3 years ago
examples d4a1f6fcec multi: Minor fixes and corrections, appease linter 3 years ago
generic 51235f3c5a Reapply last commit to correct repos. 3 years ago
server 9225050fa8 Update dependencies, import paths, etc. 3 years ago
.deepsource.toml d272296875 Update .deepsource.toml 2 years ago
.gfcp_root a4bbd65ce5 Update gitignore 3 years ago
.gfcptun_root 832ee1fc25 Update dependenices for 1.16rel. 3 years ago
.gitignore d4a1f6fcec multi: Minor fixes and corrections, appease linter 3 years ago
.whitesource 200aedf625 Add .whitesource configuration file 3 years ago
LICENSE 3848d07f1e Update LICENSE text; unmarkdownify 2 years ago
README.md 68fa4ee526 Update README.md 2 years ago
SECURITY.md 51235f3c5a Reapply last commit to correct repos. 3 years ago
clean d4a1f6fcec multi: Minor fixes and corrections, appease linter 3 years ago
compile 51d30c83e4 compile: Warn when missing UPX, detect UPX failure 3 years ago
go.mod 4d378e97f4 Update github.com/johnsonjh/gfsmux commit hash to af4ead3 2 years ago
go.sum 4d378e97f4 Update github.com/johnsonjh/gfsmux commit hash to af4ead3 2 years ago
renovate.json a7b09987bc Restyled by prettier-json 3 years ago

README.md

gfcptun

GRC Maintainability

gfcptun: A fast and low-latency connection tunnel using GFCP over UDP.


Basic gfcptun/GFCP recommendations

net.core.rmem_max=26214400       # Tune for BDP (bandwidth delay product)
net.core.rmem_default=26214400
net.core.wmem_max=26214400
net.core.wmem_default=26214400
net.core.netdev_max_backlog=2048 # (Adjust proportional to receive window)
  • Increase buffering for high-speed local links to 16MiB or more, example:
-sockbuf 16777217

Process invocation examples

client -r "IN:4321" -l ":8765" -mode fast3 -nocomp -autoexpire 900 -sockbuf 33554434 -dscp 46
server -t "OUT:8765" -l ":4321" -mode fast3 -nocomp -sockbuf 33554434 -dscp 46
  • Application → Out (8765/TCP) → Internet → In (4321/UDP) → Server (8765/TCP)

    • Other useful parameters: -mode fast3 -ds 10 -ps 3, etc.

Tuning for increased total throughput

  • To tune, increase -rcvwnd on client, and -sndwnd on server, in unison.

    • The minimum window size will dictate the maximum link throughput: ( 'Wnd' * ( 'MTU' / 'RTT' ) )

    • MTU should be set by -mtu parameter and never exceed the MTU of the physical interface. For DC/high-speed local links w/jumbo framing, using an MTU of 9000-9702 is highly recommended.


Tuning for reduced overall latency

  • Adjust the retransmission algorithm aggressiveness:

    • fast3 fast2 fast normal default

Avoiding N 1 multiplexing head-of-line blocking behavior

  • Raise -smuxbuf to 16MiB (or more), however, the actual value to use depends on link congestion as well as available contiguous system memory.

  • SMUXv2 can be used to limit per-stream memory usage. Enable with -smuxver 2, and then tune with -streambuf (size in bytes).

    • Example: -smuxver 2 -streambuf 8388608 for an 8MiB buffer (per stream).
  • Start tuning by limiting the stream buffer on the receiving side of the link.

    • Back-pressure should trigger existing congestion control mechanisms, providing practical rate limiting to prevent the exhaustion of upstream capacity and also avoiding downlink starvation (bufferbloat scenario).
  • SMUXv2 configuration is not negotiated, so must be set manually on both sides of the GFCP link.


Memory consumption control

  • GOGC runtime environment variable tuning recommendation:

    • 10-20 for low-memory systems and embedded devices

    • 120-150 (or higher) for dedicated servers

  • Notes regarding (GF)SMUX(v1/v2) tuning:

    • Primary memory allocation is done from a buffer pool (xmit.Buf), in the GFCP layer. When allocating, a fixed-size buffer, determined by the MtuLimit, will be returned. From there, the rx queue, tx queue, and, fec queue will be allocated, and will return the allocation to the buffer pool after use.
  • The buffer pool mechanism maintains a high watermark for in-flight objects from the pool to survive periodic runtime garbage collection.

  • Memory will be returned to the system by the Go runtime when idle. Variables that can be used for tuning this are -sndwnd,-rcvwnd,-ds, and -ps.

    • These parameters affect the high watermark - the larger the value, the higher the total memory consumption can be at any given moment.
  • The -smuxbuf setting and GOMAXPROCS variable can be used to tune the balance between the concurrency limit and overall resource usage.

    • Increasing -smuxbuf will increase the practical concurrency limit, however, the -smuxbuf value is not linerally proprotional to the concurrency handling maximum because Go runtime's garbage collection is, for practical purposes, non-deterministic.

    • Only empirical testing can provide the feedback required for real-world link tuning and optimization.


Link compression

  • Optional compression (using Snappy) is supported.

  • Compression saves bandwidth on redundant, low-entropy data, but will increase overhead (and CPU usage) in all other cases.

    • Compression is enabled by default: use -nocomp to disable.

    • Both ends of the link must use the same compression setting.


GFCP SNSI monitoring

  • Upon receiving a USR1 signal, detailed link information will be displayed.

Low-level GFCP tuning

  • Example: -mode manual -nodelay 1 -interval 20 -resend 2 -nc 1

Availability