forksand-it-manual/source/Firewall-opnsense.tex

%
% Firewall-opnsense.tex
%
% Fork Sand IT Manual
%
% Copyright (C) 2018, Fork Sand, Inc.
% Issued by Oleksandr Papevis
%
% This document is licensed under the Creative Commons Attribution 4.0
% International Public License (CC BY-SA 4.0) by Fork Sand, Inc.
%

\section{Hardware Overview}

\begin{itemize}
    \item OPNsense is based on FreeBSD \\ \url{https://opnsense.org/}
        \\ \url{https://wiki.opnsense.org/index.html}
    \item Iris FW1100 datasheet \\ \url{https://www.supermicro.com/products/system/1U/1018/SYS-1018D-FRN8T.cfm}
\end{itemize}

The Supermicro SuperServer 1018D-FRN8T is a 1U server with front I/O.
That means that both the rear I/O ports as well as the I/O expansion
ports are found along the front side of the rack. In many cases this
is a desirable configuration as it can make cabling very simple.

\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ss-front.png}
    \caption{Supermicro SuperServer 1018D-FRN8T Front}
    \label{fig:supermicroSSfront} 
\end{figure}

The rear of the unit has a redundant 400W power supply. Rated at 80
Plus Platinum the power supplies are efficient as well. The remainder
of the rear is simply a bezel for fans.

\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ss-rear.png}
    \caption{Supermicro SuperServer 1018D-FRN8T Rear}
    \label{fig:supermicroSSrear}
\end{figure}

The onboard I/O is plentiful. There are two USB 3.0 ports along with
a VGA port for \gls{kvm} carts. Above the USB ports there is a RJ-45
Ethernet port for out-of-band management that can be directly
connected to a dedicated management network. 
%-------------------
Furthermore there are
six 1GbE ports connected to two Intel i210-at controllers and an
Intel i350-am4 controller. The two SFP+ ports are controlled by the
Xeon D’s Intel X552 NIC. For \glspl{firewall} and other appliances, this is
a very strong configuration.

\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/iris-fw1100-front.png}
    \caption{Supermicro SuperServer 1018D-FRN8T interfaces}
    \label{fig:supermicroSSinterfaces}
\end{figure}

Inside the system we see a redundant set of fans near the PSU bezel
and a very small motherboard inside. One can see our two stacks of
Seagate Enterprise Capacity V3 1TB 7200rpm drives as well. We removed
the PCIe riser and the airflow shroud from this picture to show off
the internals better.

\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ss-noshroud.png}
    \caption{Supermicro SuperServer 1018D-FRN8T Internal no shroud}
    \label{fig:supermicroSSnoshroud}
\end{figure}

\subsection{Remote Management}

%(11:43:34 PM) forksand@jabb.im: I'm doing the install a bit different.
%After doing the opnsense installer and booting up, i *only* set up the
%firewall WAN interface statically. This allows you to admin from the WAN
%interface. If you configure a LAN, it firewalls out the WAN from remote.
%So to get started, I have to just to the WAN, then write a rule that
%allows WAN remote access.
Supermicro’s \gls{ipmi} and \gls{kvm}-over-IP enables deployment flexibility.
One can do remote power up, power down, and reset of the server in
the event that it becomes unresponsive.

\begin{itemize}
    \item fan speeds, chassis intrusion sensors, thermal sensors,
        and etc. can be monitored remotely
    \item remote power control. One can do remote power up, power
        down, and reset of the server in the event that it becomes
        unresponsive.
    \item alerts can be setup to notify the admins of issues.
    \item remotely mount CD images and floppy images to the machine
        over the dedicated management Ethernet controller. This keeps
        maintenance traffic off of the primary Intel NICs.
        At the same time it removes the need for an optical disk to
        be connected to the Supermicro motherboard.
\end{itemize}

Supermicro's BIOS has a feature: the BMC IP address shows
up on the post screen!
If you have a \gls{kvm} cart hooked up to the system, it gives an
indicator of which machine one is connected to during post.

Supermicro does include \gls{kvm}-over-IP functionality with the motherboard.

\begin{itemize}
 \item Default \gls{ipmi} connection is in cleartext http.
 \item SSL certificate for Supermicro \gls{ipmi} is bad (like all of them).
 \item Can't change password on \gls{ipmi}.
 %\item Root password for server and \gls{ipmi} is sent via email.
 %\item There is an attack window between their machine imaging and first login.
 %\item Customer should control timing of first power on.
 %\item System is also possibly vuln during the ISP's initial power up and commissioning period.
 %\item First reboot, the system hung (.png XXX).
 %\item Hard reset, lots of DHCP queries at boot.
 %\item A \texttt{debian} user was on the system, password unknown. Check \texttt{/home}!
 %\item They block NTP to prevent \gls{ddos}, so you have to use their time server
 % \texttt{time.sharktech.net}
\end{itemize}

\subsection{Supermicro Setup over IPMI bios}
{{\grenewcommand{\currentColor}{secondary-brown}}}
{{\grenewcommand{\currentTextColor}{ao-black}}}
\providecommand{\sharkIPConfigItem}[4]{}
\renewcommand{\sharkIPConfigItem}[4]{
    \rowcolor{\currentColor}   \vspace{-1pt}
    \rule[-0.3em]{0pt}{-0.5em}  \vspace{-1pt}
    \small{\textcolor{\currentTextColor}{#1}} & \vspace{-1pt}
    \small{\textcolor{\currentTextColor}{#2}} \\
}
\providecommand{\sharkIPConfigLastItem}[4]{}
\renewcommand{\sharkIPConfigLastItem}[4]{
    \rowcolor{\currentColor}   \vspace{-1pt}
    \rule[-1.0em]{0pt}{1em}  \vspace{-1pt}
    \small{\textcolor{\currentTextColor}{#1}} & \vspace{-1pt}
    \small{\textcolor{\currentTextColor}{#2}} \\
    \tabucline[2pt]{1-2}
}
\providecommand{\SIPCCwidth}{3.5cm}
\renewcommand{\SIPCCwidth}{5cm}

Before \gls{ipmi} Initialization, choose in Boot Agent GE an entry PXE
(Preboot eXecution Environment)

In Aptio Setup Utility set the following Boot Features:

\begin{table}[!htb]
    \caption{sf-fw BIOS configs}% \label{tab:sharkNodeIPConfig}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Boot Feature}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { SMCBiosActionFlag       }{ \char`[0\char`]          }{}{}
        \sharkIPConfigItem    { SumBbsSupportFlag       }{ 48                       }{}{}
        \sharkIPConfigLastItem{ Bridge ports            }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigItem    { SumBbsSupportFlag       }{ \char`[Force BIOS\char`] }{}{}
        \sharkIPConfigItem    { SumBbsSupportFlag       }{ \char`[On\char`]         }{}{}
        \sharkIPConfigItem    { SumBbsSupportFlag       }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigItem    { SumBbsSupportFlag       }{ \char`[Immediate\char`]  }{}{}
        \sharkIPConfigLastItem{ Subnet mask             }{ \char`[Disabled\char`]   }{}{}
    \end{tabu}
\end{table}

\begin{table}[!htb]
    \caption{sf-fw BIOS configs continued}% \label{tab:sharkNodeIPConfig}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Boot Feature}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Power Configuration     }{}{}{}
        \sharkIPConfigItem    { Watch Dog Function      }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigItem    { Power button Function   }{ \char`[4 Seconds Override\char`] }{}{}
        \sharkIPConfigLastItem{ Restore on AC Power Loss}{ \char`[Power On\char`]   }{}{}      
        \multicolumn{2}{|[2pt]c|[2pt]}{
        \rule[-0.7em]{0pt}{2em}  \vspace{-1pt}
        \cellcolor{\currentColor} Set system Date/Time}\\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Onboard LAN1 OPROM      }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigItem    { Onboard LAN2 OPROM      }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigLastItem{ Onboard LAN3 - LAN8 OPROM }{ \char`[Disabled\char`]   }{}{}
        \sharkIPConfigItem    { Legacy Boot Order \char`#1}{ \char`[USB Key:Virtual Disk\char`]   }{}{}
        \sharkIPConfigLastItem{ Legacy Boot Order \char`#2 - \char`#7}{ \char`[Disabled\char`]   }{}{}
        \multicolumn{2}{|[2pt]c|[2pt]}{
        \rule[-0.7em]{0pt}{2em}  \vspace{-1pt}
        \cellcolor{\currentColor} Let default option 5 execute}\\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Adapter }{LSI2116-IT}{}{}
        \sharkIPConfigItem    { PCI Slot }{0B}{}{}
        \sharkIPConfigItem    { PCI Address(Bus/Dev) }{02:00}{}{}
        \sharkIPConfigItem    { MPT Firmware Revision }{20.00.07.00-IT}{}{}
        \sharkIPConfigItem    { SAS Address }{50030480:1E300A01}{}{}
        \sharkIPConfigItem    { NVDATA Version }{14.01.40.00}{}{}
        \sharkIPConfigItem    { Status }{Disabled}{}{}
        \sharkIPConfigItem    { Boot Order}{0}{}{}
        \sharkIPConfigLastItem{ Boot Support}{ \char`[Disabled\char`]   }{}{}
    \end{tabu}
\end{table}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-ipmi-init.png}
    \caption{Supermicro SuperServer 1018D-FRN8T PEI-IPMI Initialization}
    \label{fig:supermicroSSCIpmiInit}
\end{figure}


\subsection*{\textcolor{ao-white}{ Supermicro Setup over IPMI bios1}}
\begin{picture}(0,0)\put(-10000,0){
        \gls{ipmi}
}\end{picture}
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-ipmi-boot1.png}
    \caption{Supermicro SuperServer 1018D-FRN8T Bios prompt for boot-menu}
    \label{fig:supermicroSSCIpmiBoot1}
\end{figure}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-ipmi-boot2.png}
    \caption{Supermicro SuperServer 1018D-FRN8T Bootstrap loader}
    \label{fig:supermicroSSCIpmiBoot2}
\end{figure}
\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-ipmi-opnsense-boot1.png}
    \caption{Supermicro SuperServer OPNsense Boot variant}
    \label{fig:supermicroSSCIpmiOpnsenseBoot1}
\end{figure}

\newpage
\subsection{Configurate with OPNsense Dashboard}
{{\grenewcommand{\currentColor}{primary-blue}}}
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash1.png}
    \caption{Supermicro SuperServer OPNsense Dashboard}
    \label{fig:supermicroSSCIpmiOpnsenseDash1}
\end{figure}
\begin{table}[!htb]
    \caption{sf-fw LSI Corp Config Utility}% \label{tab:sharkNodeIPConfig}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Hostname }{sf-fw1}{}{}
        \sharkIPConfigItem    { Domain }{forksand.com}{}{}
        \sharkIPConfigItem    { Language }{English}{}{}
        \sharkIPConfigItem    { Primary DNS Server }{216.146.35.35}{}{}
        \sharkIPConfigItem    { Secondary DNS Server }{208.67.222.222}{}{}
        \sharkIPConfigLastItem{ Override DNS }{unchecked}{}{}
        \sharkIPConfigLastItem{ Enable Resolver}{checked}{}{}
        \sharkIPConfigLastItem{ Others }{leave unchecked}{}{}
    \end{tabu}
\end{table}

\begin{itemize}
    \item Set server time information
    \item Configure WAN interface,  DHCP, subnet masks /32, Block .. Flags checked, others empty
    \item Configure WAN interface, IP 192.168.1.1 change to 192.168.110.21, subnet mask /24
    \item Set Web GUI Password
    \item Reload to apply changes
    \item Finished initial configuration, click a href "continue to the dashboard"
    \item Configure console appears, refer to table
        \ref{tab:supermicroSSCIpmiOpnsenseDash2} on p. \pageref{tab:supermicroSSCIpmiOpnsenseDash2}
    \item Set root password and reboot
    \item Re-enter Aptio Setup Utility Boot tab
    \item Switch Legacy Boot Order \char`#1 \char` to [Hard Disk: SATADOM-...\char`]
    \item Start the boot
    \item OPNsense: Let default option 5 execute
\end{itemize}
{{\grenewcommand{\currentColor}{secondary-brown}}}
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash2.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Continued}
    \label{fig:supermicroSSCIpmiOpnsenseDash2}
\end{figure}
\begin{table}[!htb]
    \caption{sf-fw LSI Corp Config Utility} \label{tab:supermicroSSCIpmiOpnsenseDash2}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Configure Console }{Accept these Settings}{}{}
        \sharkIPConfigItem    { Select task }{Guided installation}{}{}
        \sharkIPConfigItem    { Select a disk }{ada0: 600.00MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)}{}{}
        \sharkIPConfigItem    { Select install mode }{GPT/UEFI mode}{}{}
        \sharkIPConfigItem    { Swap Partition }{yes}{}{}
        \sharkIPConfigLastItem{ Enable Resolver}{checked}{}{}
    \end{tabu}
\end{table}
{{\grenewcommand{\currentColor}{primary-blue}}}
\subsection{Update OPNsense Firmware using Dashboard}
\begin{itemize}
    \item Enter OPNsense dashboard and make a backup, System -> Configuration -> Backups, save the XML
    \item Execute update firmware, refer to figure
        \ref{fig:supermicroSSCIpmiOpnsenseDash3} on p. \pageref{fig:supermicroSSCIpmiOpnsenseDash3}
\end{itemize}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash3-update.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Update Firmware}
    \label{fig:supermicroSSCIpmiOpnsenseDash3}
\end{figure}
\begin{itemize}
    \item Standby until updating finished, refer to figure
        \ref{fig:supermicroSSCIpmiOpnsenseDash4} on p. \pageref{fig:supermicroSSCIpmiOpnsenseDash4}
    \item Switch to tab Settings, refer to figure
        \ref{fig:supermicroSSCIpmiOpnsenseDash5} on p. \pageref{fig:supermicroSSCIpmiOpnsenseDash5}
\end{itemize}
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash4-update.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Update Firmware Continued}
    \label{fig:supermicroSSCIpmiOpnsenseDash4}
\end{figure}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash5-fw.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Firmware Settings}
    \label{fig:supermicroSSCIpmiOpnsenseDash5}
\end{figure}
\begin{itemize}
    \item Set mirror to LeaseWeb (San Francisco, US)
    \item Set Flavour to LibreSSL
    \item Set Release Type to Production
    \item Click save and return to Updates tab.
\end{itemize}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash6-fw-updates.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Firmware Pending Updates}
    \label{fig:supermicroSSCIpmiOpnsenseDash6}
\end{figure}
\begin{itemize}
    \item Click Update now.
    \item Standby until Update is completed.
    \item Restore configs from XML, refer to figure
        \ref{fig:supermicroSSCIpmiOpnsenseDash8} on p. \pageref{fig:supermicroSSCIpmiOpnsenseDash8}
\end{itemize}
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash7-fw-update.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Firmware Update Processing}
    \label{fig:supermicroSSCIpmiOpnsenseDash7}
\end{figure}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash8-fw-backupandreboot.png}
    \caption{Supermicro SuperServer OPNsense Dashboard restore from XML config backup}
    \label{fig:supermicroSSCIpmiOpnsenseDash8}
\end{figure}
\begin{itemize}
    \item Upload the config and restore
    \item Add a user, refer to figure
        \ref{fig:supermicroSSCIpmiOpnsenseDash9} on p. \pageref{fig:supermicroSSCIpmiOpnsenseDash9}
        using parameters from table
        \ref{tab:supermicroSSCIpmiOpnsenseAddUser} on p. \pageref{tab:supermicroSSCIpmiOpnsenseAddUser}
\end{itemize}
\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash9-user.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Add User}
    \label{fig:supermicroSSCIpmiOpnsenseDash9}
\end{figure}
\begin{table}[!htb]
    \caption{sf-fw OPNsense Dashboard Add User} \label{tab:supermicroSSCIpmiOpnsenseAddUser}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Username }{jebba}{}{}
        \sharkIPConfigItem    { Disabled }{unchecked}{}{}
        \sharkIPConfigItem    { Full name }{Jeff Moe}{}{}
        \sharkIPConfigItem    { E-mail }{moe@forksand.com}{}{}
        \sharkIPConfigItem    { Comment }{}{}{}
        \sharkIPConfigItem    { Expiration date }{}{}{}
        \sharkIPConfigLastItem{ Group Memberships }{Member of admins}{}{}
        \sharkIPConfigItem    { Certificate }{unchecked}{}{}
        \sharkIPConfigLastItem{ OTP seed }{}{}{}
    \end{tabu}
\end{table}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash10-dhcpv4.png}
    \caption{Supermicro SuperServer OPNsense Dashboard DHCPv4}
    \label{fig:supermicroSSCIpmiOpnsenseDash10}
\end{figure}
\begin{itemize}
    \item Disable DHCPv4
\end{itemize}
\begin{table}[!htb]
    \caption{sf-fw OPNsense Dashboard DHCPv4} \label{tab:supermicroSSCIpmiOpnsenseDhcpv4}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Enable }{unchecked}{}{}
        \sharkIPConfigItem    { Deny unknown clients }{unchecked}{}{}
        \sharkIPConfigItem    { Subnet }{192.168.110.0}{}{}
        \sharkIPConfigItem    { Subnet mask }{255.255.255.0}{}{}
        \sharkIPConfigLastItem{ Range }{192.168.110.10 - 192.168.110.245}{}{}
        \sharkIPConfigLastItem{ Others }{leave unchanged}{}{}
    \end{tabu}
\end{table}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash11-plugins.png}
    \includegraphics[keepaspectratio=true,trim=360mm 190mm 10mm 80mm,clip,width=1.0\textwidth,angle=0]
        {sf-fw/ssc-opns-dash11-plugins.png}
    \caption{Supermicro SuperServer OPNsense Dashboard Plugin Installation}
    \label{fig:supermicroSSCIpmiOpnsenseDash11}
\end{figure}
\begin{itemize}
    \item Make sure os-dyndns plugin installed
    \item Install os-acme-client
\end{itemize}
%\begin{table}[!htb]
%    \caption{sf-fw OPNsense Dashboard Plugins} \label{tab:supermicroSSCIpmiOpnsensePlugins}
%    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
%        \tabucline[2pt]{1-2}
%        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
%        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
%        \tabucline[2pt]{1-2}
%        \sharkIPConfigItem    { Enable }{unchecked}{}{}
%        \sharkIPConfigItem    { Deny unknown clients }{unchecked}{}{}
%        \sharkIPConfigItem    { Subnet }{192.168.110.0}{}{}
%        \sharkIPConfigItem    { Subnet mask }{255.255.255.0}{}{}
%        \sharkIPConfigLastItem{ Range }{192.168.110.10 - 192.168.110.245}{}{}
%        \sharkIPConfigLastItem{ Others }{leave unchanged}{}{}
%    \end{tabu}
%\end{table}

\newpage
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
        {sf-fw/ssc-opns-dash12-lea.png}
    \caption{Supermicro SuperServer OPNsense Dashboard add Let's Encrypt account}
    \label{fig:supermicroSSCIpmiOpnsenseDash12}
\end{figure}
\begin{itemize}
    \item Add Let's Encrypt account
    \item Modify global Let's Encrypt settings
    \item Apply Let's Encrypt settings
    \item Refer to Certificates menu
\end{itemize}
\begin{table}[!htb]
    \caption{sf-fw OPNsense Dashboard Let's Encrypt account and settings} \label{tab:supermicroSSCIpmiOpnsenseLea}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Enable }{checked}{}{}
        \sharkIPConfigItem    { Name }{sf-fw1}{}{}
        \sharkIPConfigItem    { Description }{\Gls{sharkfork} \Gls{firewall} 1}{}{}
        \sharkIPConfigLastItem{ E-Mail address }{sharkfork@forksand.com}{}{}
        \sharkIPConfigItem    { Enable Plugin }{checked}{}{}
        \sharkIPConfigItem    { Auto Renewal }{checked}{}{}
        \sharkIPConfigItem    { Let's Encrypt Environment }{Production Environment \char`[Default\char`]}{}{}
        \sharkIPConfigLastItem{ HAProxy Integration }{unchecked}{}{}
    \end{tabu}
\end{table}

\newpage
%\begin{figure}[!htb]
%    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]
%        {sf-fw/ssc-opns-dash13-cert.png}
%    \caption{Supermicro SuperServer OPNsense Dashboard add Certificate}
%    \label{fig:supermicroSSCIpmiOpnsenseDash12}
%\end{figure}
\begin{itemize}
    \item Add Validation Method
    \item Add Certificate
    \item Apply ``Issue/Renew Certificates Now''
\end{itemize}
\begin{table}[!htb]
    \caption{sf-fw OPNsense Dashboard Let's Encrypt validation} \label{tab:supermicroSSCIpmiOpnsenseLeaValid}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Validation Method }{}{}{}
        \sharkIPConfigItem    { Enable }{checked}{}{}
        \sharkIPConfigItem    { Name }{sf-fw1-http}{}{}
        \sharkIPConfigItem    { Description }{\Gls{sharkfork} \Gls{firewall} 1 http validation}{}{}
        \sharkIPConfigLastItem{ Challenge Type }{HTTP-01}{}{}
        \sharkIPConfigLastItem{ HTTP Service }{OPNsense Web Service (automatic port forward)}{}{}
        \sharkIPConfigItem    { IP Auto-Discovery }{checked}{}{}
        \sharkIPConfigItem    { Interface }{WAN}{}{}
        \sharkIPConfigLastItem{ IP Addresses }{}{}{}
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Certificate }{}{}{}
        \sharkIPConfigItem    { Enable }{checked}{}{}
        \sharkIPConfigItem    { Common Name }{sf-fw1.forksand.com}{}{}
        \sharkIPConfigItem    { Description }{\Gls{sharkfork} \Gls{firewall} 1}{}{}
        \sharkIPConfigItem    { Alt Names }{}{}{}
        \sharkIPConfigItem    { LE Account }{sf-fw1}{}{}
        \sharkIPConfigItem    { Validation Method }{sf-fw1-http}{}{}
        \sharkIPConfigItem    { Restart Actions }{}{}{}
        \sharkIPConfigItem    { Auto Renewal }{checked}{}{}
        \sharkIPConfigLastItem{ Renewal Interval }{60}{}{}
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Interfaces -\char`> Lan }{}{}{}
        \sharkIPConfigItem    { Enable }{checked}{}{}
        \sharkIPConfigItem    { Lock }{checked}{}{}
        \sharkIPConfigItem    { Description }{LAN}{}{}
        \sharkIPConfigItem    { IPv4 Configuration Type }{Static IPv4}{}{}
        \sharkIPConfigLastItem{ IPv6 Configuration Type }{none}{}{}
    \end{tabu}
\end{table}
\begin{itemize}
    \item Refer to System -\char`> Gateways -\char`> Single -\char`> WAN\char`_DHCP6
    \item Set Disabled flag to checked
    \item Press Apply changes
    \item Modify LAN and WAN interfaces, disable IPv6 at both
    \item Modify \Gls{firewall} Rules, disable IPv6
    \item Add new rula to \Gls{firewall} Rules WAN
\end{itemize}
\begin{table}[!htb]
    \caption{sf-fw OPNsense Dashboard \Gls{firewall} Rules} \label{tab:supermicroSSCIpmiOpnsenseLeaRules}
    \begin{tabu}{|[2pt]p{\SIPCCwidth}|[2pt]p{\SIPCCwidth*2}|[2pt]}
        \tabucline[2pt]{1-2}
        \multicolumn {1}{|[2pt]l|[2pt]}{\rule[-0.7em]{0pt}{2em} \cellcolor{\currentColor}{Parameter}}&
        \multicolumn {1}{l|[2pt]}{\cellcolor{\currentColor}{Value}} \\
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { Interfaces -\char`> WAN }{}{}{}
        \sharkIPConfigItem    { Enable }{checked}{}{}
        \sharkIPConfigItem    { Lock }{checked}{}{}
        \sharkIPConfigItem    { Description }{WAN}{}{}
        \sharkIPConfigItem    { IPv4 Configuration Type }{DHCP}{}{}
        \sharkIPConfigLastItem{ IPv6 Configuration Type }{none}{}{}
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { \Gls{firewall} -\char`> Settings -\char`> Advanced }{}{}{}
        \sharkIPConfigLastItem{ Allow IPv6 }{unchecked}{}{}
        \tabucline[2pt]{1-2}
        \sharkIPConfigItem    { \Gls{firewall} -\char`> Rules -\char`> WAN }{}{}{}
        \sharkIPConfigItem    { Action }{Pass}{}{}
        \sharkIPConfigItem    { Disabled }{unchecked}{}{}
        \sharkIPConfigItem    { Interface }{WAN}{}{}
        \sharkIPConfigItem    { TCP/IP Version }{IPv4}{}{}
        \sharkIPConfigItem    { Protocol }{TCP}{}{}
        \sharkIPConfigItem    { Source/Invert }{unchecked}{}{}
        \sharkIPConfigItem    { Source }{any}{}{}
        \sharkIPConfigItem    { Destination/Invert }{unchecked}{}{}
        \sharkIPConfigItem    { Destination }{This \Gls{firewall}}{}{}
        \sharkIPConfigItem    { Destination port range }{https to https}{}{}
        \sharkIPConfigItem    { Log }{unchecked}{}{}
        \sharkIPConfigItem    { Category }{}{}{}
        \sharkIPConfigItem    { Discription }{Enable https to \Gls{firewall}}{}{}
        \sharkIPConfigItem    { Source OS }{Any}{}{}
        \sharkIPConfigItem    { No XMLRPC Sync }{unchecked}{}{}
        \sharkIPConfigItem    { Shedule }{none}{}{}
        \sharkIPConfigLastItem{ Gateway }{default}{}{}
    \end{tabu}
\end{table}

\newpage
\section{Alternatives Hardware Overview}
Some resellers:
\begin{itemize}
    \item \url{https://www.deciso.com/}
    \item \url{https://www.pfwhardware.com/}
    \item \url{https://www.osnet.eu/}
\end{itemize}

\begin{itemize}
    \item (8) 1 gig ethernet ports
        Connects to (1) 100M ethernet upstream fiber optic
        Connects to (1) 100M ethernet upstream wifi
        Various LAN
    \item (Hot swap?) Dual Power Supplies
    \item (How swap?) RAID (Linux md), with SSD storage.
    \item 2.5'' drive bays
    \item Total ~8GHz CPU
    \item ~8-16 gigs RAM ? Depends on OS.
    \item Two servers total, for standby/failover
\end{itemize}