Can't you throw iptables on a rackmount linux box? Free and stable...

for the firewall on a rackmount machine: depends on the OS. If Linux, then iptables. If *BSD, then pf. I can help.