ProxmoxVE, virtual OPNsense to protect underlying VMs

Virtual networking setup

After installing ProxmoxVE two bridges need to be created, one for the WAN and another for the LAN. Depending if a slave port is selected on a hardware interface the network will be available outside the machine. Because the WAN is in another subnet than the management port we will select a slave port available and do not inset CIDR, gateway or IPV6.

Go to: Datacenter > pve > System > Network > Create

Create two Linux Bridges, one for LAN and the other for WAN and leave them empty except the comment. Add an unused slave interface if you want to expose them physically. Once all the networks has been created check the configuration to avoid locking yourself out and apply the changes.

OPNsense installation inside Proxmox VM

After downloading, extracting, and re-uploading OPNsense to proxmox the VM can be created. On the OPNsense download page select dvd image type to make sure you can boot the image as an usb installer.

For this VM two cores have been allocated. 16GiB of storage and 4096MiB of RAM. During the creation process only one nick can be selected therefore once the VM has been created the second nick has to be added.

On the image above you can see that the two Linux Bridges created in the previous sections have been added to the VM hardware. They will act ast the WAN and LAN.

Now OPNsense must be installed.
Boot the VM and wait till it asks for login. Login using the user: installer and password: opnsense
Select your keymap and continue selecting GPT/UEFI partition mode.

Now let the installer clon the system into the disk.

Change the default root password before finishing the process. While the vm reboots stop it and remove the CD/DVD drive as it is no longer needed.

Now you can boot again the vm and login using root and the password that you selected the previous step.

Once you have done login select the option 1; Assign interfaces

Select no with n to LAGs and VLANs configuration if you won't use them.

Now you will be presented with the two interfaces and asked to select the WAN. You can go to HARDWARE page of the VM and check the Linux Bridge MAC address. If you don't recall witch one you selected as WAN go to pve > System > Network and check the comment to find witch bridge is the WAN and which is LAN.

After you have confirmed that the networks are correct the DHCP of the ISP router will assign IP to the WAN of the OPNsense. Note that now to set up the LAN and configure the DHCP of the OPNsense a cable will have to be plugged from the LAN hardware port.

OPNsense configuration wizard

OPNsense LAN is by default at the subnet 192.168.1.0/24 therefore you will be provided an IP from that range. After connecting a wire to the LAN port, write https://192.168.1.1/ in the browser.
This first time that OPNsense GUI is accessed it will ask to run the configuration wizard. This procedure will be used to set the initial configuration.

On the general information a hostname must be selected, for the DNS servers I have selected OpenDNS IPs 208.67.222.222, and 208.67.220.220, keep the rest of the fields as they are.

Leave the time-server hostname and timezone as they are filled and move to the next screen.

As there is a DHCP server enabled on the ISP router there is no change on this screen.

For the LAN setup we will change the subnet to 10.0.1.0/8 and assign to the router the ip 10.0.1.1

Now the firewall will set the configuration accordingly and restart the necessary processes.

Finally, a network adapter restart is necessary to get the new subnet ip from the DHCP. To access the router now use https://10.0.1.1 url

On the Proxmox console the LAN now has the correct subnet.

Now, you can start adding VMs and set the OPNsense LAN adapter as their network adapter.

OPNsense third interface for management on Proxmox management subnet

On this section we will add ProxmoxVe management interface to OPNsense firewall. This will enable the monitoring of the firewall without having to plug the computer in a specific ethernet port. Additionally, it will disable the VMs of accessing the firewall in case they get compromised.

This is a quite tricky step because by default OPNsense listens at all interfaces but the anti-lockout rules are applied to LAN.

Disable the DHCP on the LAN interface. Go to Services > ISC DHCPv4 > [LAN] and uncheck the Enable tick.

Add the vmbr0 network to the OPNsense vm.

From the OPNsense GUI go to Interfaces > Assignments and add the newly added interface. Add a comment indicating is the new LAN for VMs interface. I used VMs_LAN

On OPNsense go to Interfaces > [LAN]. Check Prevent interface removal, and on IPV4 Configuration type select DHCP. Save and apply the changes. Now the LAN is waiting to get a DHCP IP. Now on the PVE console you need to reassign the interfaces.

To reassign the interfaces login with the root user and press 1. Skip with n the LAGGs and VLANs configs. Fill the interface name for WAN, LAN, and OTP1. This time you have to switch the Proxmox Linux Bridge for management vmbr0 with OPNsense LAN interface.

As the lan is expecting for an IP from the DHCP the management router will allocate an IP for the LAN and the VMs_LAN interface will become the interface for the VMs. This allows us to control the firewall from the LAN.

To login now you have to reconnect your computer to the management subnet. On the console of Proxmox your new assigned IP for the LAN should be displayed. If it does not get an IP directly restart the services using the option 11.

Now go to Interfaces > [VMs_LAN]. Enable and check Prevent interface removal, and on IPV4 Configuration type select Static IPv4. Scroll down a bit and on IPv4 address fill the field with 10.0.1.1 and 24 for the mask. Save the changes and apply the configuration.

Apply the changes after saving the configuration.

Now we have to enable the DHCP in the VMs_LAN subnet if we don't want to configure each IP statically for each VM.

Go to Services > ISC DHCPv4 > [VMs_LAN] check Enable and fill the range according to your needs.

Reboot the router.

While trying to find ways to isolate OPNsense GUI on an interface I found that using OPX instead of LAN interface proved to be problematic.

The Interface swap feels like is not the appropriate way to do it, but it does the job.

We still need to add several firewall rules to allow the VMs to reach the public domain but protect the firewall GUI from internal VMs access.

Go to Firewall > Rules > VMs_LAN and press the + button

After this rules the VMs will be able to resolve the DNS queries and access the public domain. Ping to the firewall has been added for debugging purposes and NTP queries have been allowed aswell. All the rest connection have been rejected on the last rule by allowing everything not targeting the router.

Now we can start creating the VMs, but they are not yet protected under suricata. Follow the next steps to enable suricata and protect the VMs

Enable and configure suricata

On this section suricata will be configured to protect the underlying VMs. You can enable suricata on Services > Intrusion Detection > Administration

Enable the service and I would recommend not enabling IPS mode after having reviewed some days of logs. If you have no traffic like this webpage you can go enable it but there is a good change you break some service. It will also depend on the quantity of the rules you enable.

On pattern method select Hyperscan, on interfaces select VMs_LAN and leave the logs Weekly.

Now proceed downloading the rulesets by going to the Download tab, select everything and press Enable selected then once enabled press Download & Update Rules.

This does not mean that those rules are being applied. They are only downloaded into the firewall.

To enable the rules we will create a policy. Go to Services > Intrusion Detection > Policy and press the + button.

On Rulesets select the ruleset that you want to enforce. For example, I selected dos, exploit, malware, ftp, sql, pop3, rpc, shellcode, smtp, tftp, telnet but this will vary on your needs.

On New Action, Drop has been selected for now. Unless IPS mode is enabled these rules won't actually drop packets.

On Description add a comment to help you better identify the ruleset triggering the drop

Don't forget to press apply once the ruleset has been created.

Go to Schedule tab and enable the schedule. This will keep the rules updated daily.

Now is just watch the logs from time to time and disable the false positives. Once you feel confortable with the logs you can activate IPS mode and those drops will take effect.