Rebuilding my network - Part 0: Uh Oh Spaghetti-O, or how I broke everything and had to start over.

Parts:

• Part 0, the intro: Part 0 - Uh oh. Spagettio
• Part 1, the planning: Part 1 - Planning out my new network
• Part 2, Dynamic DNS: Part 2 - Informing Namecheap about OPNsens IP
• Part 3, WireGuard VPN Part 3 - Setting up Wireguard on OPNsense

The breaking

Alright, lets preface a bit on what is going to happen in this series.
I've been experimenting a bit with Kubernetes at home. To do this i have been leveraging Harvester from Rancher / Suse. And as all of this is on premise i also needed a load balancer i could use on premise. For this i wanted to user MetalLB with its BGP (aka Layer 3) mode.

While debugging why the BGP announcements did not update anything on the PFsense box i managed to break the settings in a unrepariable fashion. In retrospect i was running pfsense 2.7.0.alpha, which well... never said it was stable. Funnily enough i was running the alpha as Dynamic DNS towards Namecheap was broken..

In the end what happened was that i deleted a BGP allow list config, while it was still referenced. Somehow this (and probably some other things) broken a multitude of files. The WebUI simply did not work anymore, so tried an update via SSH.


Seems like a perfect time to break my home network

This update litteraly broke the rest of the box, it did not even want to boot anymore. Seemed like something really wrong happened during the update, which wiped the OS partition.


Uuh, i actually broke my network :')

Thankfully my Firewall is based on one of the more magical pieces of hardware i have laying around.
A APU2 from PCEngines. https://pcengines.ch/apu2.htm. The reason i opted for this piece of hardware is that it is 1) Strong enough to run 1gbps+++ PFsense 2) Its designed to be fanless.

I bought a APU4C4 from https://TekLager.se

Getting things installed

Getting output from the (black)box

So of course i need a couple of things to talk to the hardware. There is no video out, so i need an alternative. Thankfully it talks Serial (Or RS-232 DB9 if you are fancy )

Thankfully i have a adapter from ST Lab


To use it with Linux i Use Minicom

For debianish distros: sudo apt-get install minicom

To find the device i usually just connect the device and do a sudo dmesg


And connecting to the interface: minicom -D /dev/ttyUSB0

Getting the USB ready

To setup the USB, just download https://opnsense.org/download.

Unpack the file with whatever you have handy.

Now, i usually advice people to just use Balena Etcher to write the usb, but at this point i had not more network, so used what i always have. Good old trusty dd

sudo dd if=OPNsense-23.7-serial-amd64.img of=/dev/sdf status=progress bs=64

Sewing it all togheter

If the minicom connection has worked you should be getting some lines of text about SeaBIOS.

And if the memorystick also worked you will see something like this:


Here i more or less just went trough the motions with all default setup.

WAN on ibg0 and LAN og ibg1, more on what is on the backside of LAN later.

More over in Part 1, Planning out my new network

Parts:

• Part 0, the intro: Part 0 - Uh oh. Spagettio
• Part 1, the planning: Part 1 - Planning out my new network
• Part 2, Dynamic DNS: Part 2 - Informing Namecheap about OPNsens IP
• Part 3, WireGuard VPN Part 3 - Setting up Wireguard on OPNsense