Linux systems unable to reach out to internet behind pfsense while Windows work fine.
I am trying to set up my #homelab for #forensics and I have encountered a very weird issue. I am running an internal pfsense #firewall for my environment. On this firewall there are 3 interfaces: WAN and two Isolated Labs.
On the “WAN” interface, both #linux and windows systems are able to access the internet without any problems.
But on the other two lans - named Infrastructure and Lab - the Linux systems are not able to connect to the internet after booting, while the systems are able to communicate with each other. In order to get them working I need to go the respective interface and make any change there and save the page, practically refreshing the settings on the interface. After this all currently online linux systems on that interface are working as expected and connectivity is restored. As you can imagine this is a huge pain, especially when I am testing things and I need to turn on various systems at different times. If i put the systems on the WAN interface they face no issue at all, so what could be the solution here for fixing connectivity for linux systems on the other interfaces?
edit: it issue is both on pfsense 2.7.2 and 2.8.0
#dfir #proxmox #dfir #homelab #firewall #pfsense #networking #network
@Chewt
The node has others vms as well, some of them are windows vms but they are not discussed as they do not face any problems connecting to the internet no matter what interface they are connected on. The Linux VMs that are on the WAN network which is the one that has the linux bridge that is connected to a port and those systems do not have any connectivity issues.
The other two lans also have a linux bridge that is not bound to any port and the Linux systems need the interface refresh in order to access the internet. Windows systems on those networks are connected to the internet with no issues.
yes the pfsense hardware is as you describe it. (picture attached)
For the last paragraph, I need to have some devices that will be isolated from my WAN (home network) because I want to create a cybersecuirty lab and I do not want anything to “escape” to my home devices.
Plus this teaches me how to manage a network with proper segmentation etc.
ok, if you’re fine with doing a little more troubleshooting, I have a few more things to try.
You said before that the systems are able to communicate with each other on the LAN network, just not connect out via WAN. Does this include the pfsense box? That is to say are you able to
ping 10.10.10.1(assuming that’s the IP of the Infra interface on pfsense) from one of you Ubuntu VM’s when it is on the Infra network? Also what is the output of both commandsip -br aandip route? What I’m am curious about here is maybe the default gateway is not being correctly set when new devices connect to the pfsense. If this ends up being the case, it might be an issue with DHCP (or if you aren’t using DHCP then just the configuration on each Ubuntu VM). I suspect that Windows may be doing some fancy things to figure out on its own some network settings without them being explicitly set.@Chewt
yes they can still ping my firewall while not connected to the internet.
Here are two screenshots from one of the ubuntu vms on the home network and one from the kali vm on the infra network
Well that doesn’t really indicate any problems with the default gateway. My last bit of troubleshooting advice is to traceroute and see if packets get stuck at the router. And check the firewall logs in pfsense to see if you can find anything there.
Hopefully it ends up being a relatively painless problem to fix!
@Chewt found the issue in the end… i had two interfaces share the same IP and was creating a conflict on the linux hosts. I dont know why Windows was not affected by it but all good in the end
Happy to hear it! Address conflicts are always a pain, I recently ran into an issue with my VPN to my home network, where my home network and the network I was on shared the same subnet and mask, so I had to specify some custom routes to make it work. Good network design is a must to try and mitigate these things!