HowTo: pfSense 2.5.2 on Watchguard XTM 5

Hey everyone, just wanted to share my experience with you on how to install pfSense on a Watchguard XTM 5 525, in case anyone has an old box laying around with no active subscription.

The reason I did that, was I needed to separate my 1G fiber connection for 4 different flats we have here in 2 buildings with an option to isolate all the IoT stuff and guest Wifi. Consumer routers will have their limits here, so I was looking into an alternative.

I got two of these boxes for 29€ ea. on eBay in perfect condition, one acts as a cold standby. It´s running on x86 hardware, it´s upgradable and has 6 Intel Gigabit NICs. There are different models of the XTM 5, 505, 515, 525, 545.

The difference in hardware is mainly the CPU and memory, where for example an XTM 515 has a 2.0 GHz Celeron 440 ingle Core and 1 GB Memory and a XTM 525 has a 2.6 GHz Celeron E3400 Dual Core. Because of Socket 775 there are plenty of options when it comes to CPU upgrades.

I threw in a XEON 3050, which clocks a bit lower with 2.13 GHz, has a bit faster FSB and a lower Tcase of 61,4 °C instead of 74,1 °C of the E3400. But you can also use much more powerful Quad Core Core2Duo, Core2Quad and XEON 3000 series and Pentium 4 Extreme series, for the drawback of higher power consumption, more heat dissipation, noise and probably lower lifetime.

If you consider a CPU upgrade visit the Intel website for the specs, to check the power consumption and heat dissipation. A Celeron 440 will draw only 30 Watts of power, while dual or quad core’s can easily draw 60 to 70 watts, which will increase your monthly power bill fast. You can get Socket 775 CPU´s on eBay for 2-3 €, same for memory. I recommend 2x 2GB DDR2 800 modules because of dual channel mode and the very low price. It´s reported that the mainboards in those boxes only support 4GB of memory, but pfSense will run just fine with only 2GB or even 1GB depending on the workload.

  1. Prerequisites

Things you need or should consider:

  • Watchguard XTM 5 Series appliance
  • 2.5” HDD or SSD which can withstand 24/7 reliably
  • A PC or Laptop with a free SATA Port
  • 4GB USB pen drive
  • SATA II / III cable of approx. 30 cm with one side angled 90 degree for better mounting
  • USB – Console Cable (optional)
  • 2x 2GB DDR2 800 Memory (optional, non-ECC)
  • Socket 775 CPU with more or less power (optional)
  • Thermal grease (optional, when you want to replace the CPU, make sure to clean the heatsink and use new thermal grease)
  • Silent 40mm silent fans if you want to make the appliance even more quiet (optional)
    • 3x for the case and CPU
    • 1x for the power supply
    • Normal 40x20mm 3 pin fans e.g. from NOCTUA, but I recommend to try the default setup first, because the BIOS of the XTM 5 does thermal management and lowers the fan speeds quite good

  1. Hardware Upgrade

If you´re planning to upgrade the hardware here are some pictures, you will at least need to open the box to mount the hard drive and remove some parts. You see the PSU on the left and a sort of usual x86 mainboard with all the NICs. Watchguard´s OS is flashed to the 1GB industrial grade CF card. Next to it is a VPN accelerator board from CAVIUM which is not supported by pfSense due to the lack of drivers. In any case you will need to remove the VPN accelerator board and the CF card!

https://preview.redd.it/to441ibtgh481.jpg?width=1505&format=pjpg&auto=webp&s=9956f096608c733ea91556049c1c554c0a607aae

I used the following upgrade to give my appliance a bit more juice, got all of them on eBay for less than 50 €:

  • 2x 2GB DDR2 800 Memory
  • 250GB Samsung SSD
  • Intel XEON 3050 2.12 GHz Dual Core

Remove the heatsink and install the new CPU, also apply thermal grease. I used some Arctic MX-2, apply only a tiny bit in the size of a pea. Make sure the CPU is seated properly and mount the heatsink carefully. Replace the old memory, but do not mount the hard drive yet. Remove the PCI-Express CAVIUM Nitrox VPN accelerator board and the 1GB Compact Flash card, keep it if you want to go back to Fireware OS sometime .

Installation Part 1

Because you have no VGA out you will have to do a little workaround. While you could use a regular USB to Console cable, like one from Cisco you won´t be able to run the installer from a USB pen drive, because this option has been deactivated in BIOS for security reasons and the password is hardcoded, so you can´t just enable it.

For that reason, we´re going to pre-install the OS on the hard drive/SSD with the help of another PC or Laptop, then attach the drive to the Watchguard and do the rest of the configuration tasks on the web interface.

  1. Download pfSense ISO from Download pfSense Community Edition
    1. Choose AMD64
    2. USB Memstick Installer
    3. Serial
    4. Mirror of your choice
  2. Burn the ISO with “Win32 Disk Imager” Win32 Disk Imager to the USB drive
  3. Hook up your SSD to an old Laptop or PC and make sure you are booting in legacy mode and the SSD is the only drive attached, otherwise you will run into trouble when the Watchguard tries to boot it
  4. Attach the USB Drive and boot from it, the installer will start automatically
  5. You can blindly run the installer with the default settings, make sure you select the SSD and hit automatic partitioning
  6. When the installer finishes and asks for a reboot, do so but immediately after the shutdown, power off the PC and to not boot the SSD drive

https://preview.redd.it/tn54pn7rih481.jpg?width=1874&format=pjpg&auto=webp&s=930dbf3b36ff6e0a70505874026366056a82ce29

Installation Part 2

  1. Mount the SSD into your Watchguard, use the SATA power connector from the PSU and the SATA cable and attach them to the drive and the mainboards SATA connector.
    1. There is no genuine mounting bracket which will hold your drive in place, so you either build yourself one from scrap parts or fix it somehow with zip ties. For me, an old 2.5” drive cage from a Lenovo I guess, did the trick. I mounted it with two screws which fit the holes in the cage, the box has four mounting pointsin total which you could use.
  2. Connect an ethernet cable from your PC to the Watchguard on Port #2, where DHCP runs by default and which will be the first optional LAN port
  3. Power on the box and wait until it booted, you will hear some friendly beeps when it´s ready
  4. Open pfSense webinterface through http://192.168.1.1 Default Login is admin / pfsense
  5. You´re now all set and good to go to do all the configurations you need

Installation via Console

You need a RJ45 to USB console cable, e.g. from a Cisco Box and then you can connect to the Terminal with these settings:

  • Speed: 115200
  • Data Bits: 8
  • Stop Bits: 1
  • Parity: None
  • Flow Control: None

You need to burn the ISO to USB, hook it up in of the ports. Attach the console cable and open a Terminal. Boot up the Firebox and you should be able to install pfSense through terminal.

Maybe someone finds this helpful, because there are several guides in the net, but most of them are outdated.

Edit:

Hardware compatibility

Check the Hardware specs on Watchguards “Hardware Guides” site: Hardware Guides | WatchGuard Technologies some run on Freescale CPUs, some others on Intel.

Also some are having a CF card, while newer boxes already use mSata.

Thanks for this write up!!! Very cool this can be done and re-use a firewall like the watch guard. Are there any other models that run x86 with ddr3?

Dang this is awesome…

What’s your throughput with one of these?

Awesome article, enjoyed reading it!

I have been doing something of the same with the CISCO WAVE branded machines.

Specifically the 294, 694 & 7541 and rather then CISCO WaaS installed I have either a baremetal Proxmox or pfSense install, ESXi I have install worries over but have been able to overcome them temporally but I can see reliability issues which mainly revolve around ethernet address assigning after a reset however this only affected the 294 & 694’s while the 7541 even with some install issues did retain or was able to assign such after a reset and without any further scripting after boot.

The reason for me stretching to such limits is because I’ve tried many times to entertain CISCO with the purchase of official CISCO WAVE WaaS software or media which they do have listed online for download but you need a pretty large commercial account. I was also mainly rejected I’m very sure because I also mentioned I purchased the hardware secondhand and after that everything was a total brick wall or lip service sadly, hence the developments as they stand with modifications.

I too have Cavium and at present it’s disabled or inactive however for me this is onboard and no major worry in the long run if not because I have upgraded the CPUs, RAM, Storage and such but I’ve noticed your’n to be an addon card adapter… You could remove that I assume and you’ve a spare x8 PCI-e, within my CISCO WAVE’s I have utilised GFX adapters, quad port NICs.

The CISCO WAVE’s have networking addon cartridges which can be slotted in and out on-demand to suit needs which was another attraction along with the WAAS notion of sorting traffic and being able to have 'VM’s as ‘blades’ but it was SO locked down, I got as far as I could with what I have and also worked around the issue.

I did this back in like 2005/2006 I think with some older generation box. It was a totally normal X86 intel cpu of some kind, the only thing it was missing was a VGA jack, so I just had to hobble together a connector from the jumper on the board to see the output. Worked great.

What kind of throughput to WAN are you getting? I’ve got 1GB internet but I’m only seeing about 300Mb/s for speed tests. I swapped out the CPU for a Core2Duo E7400 and 4GB RAM, booting off of a 180GB SSD, so it would seem to be well capable of pulling more than 300Mb/s. Just the initial install, I do not have anything else running yet, like pfBlockerNG, so nothing to slow it down.

Thanks

CMOS reset (to clear bios settings, ie password) is jumper J19. Using a jumper connector to bridge the connection then powering on the system will remove the password and reset the stored settings in BIOS. Remove the jumper after powering on the unit. (J19 is below the CMOS battery, to the bottom right of the battery towards the front of the unit behind eth3. It is a two pin normally open connector)

VGA output is pin header J9. (On Motherboard V1.2) pinout as follows:
1 - Red //
2 - Gnd //
3 - Green //
4 - Gnd //
5 - Blue //
6 - Gnd //
7 - Hsync //
8 - Gnd //
9 - Vsync //
10 - Gnd //
11 - DDC Data //
12 - DDC CLK //

Pin 1 is identified by white silkscreen triangle, holding the box with the eth ports facing you, 1 is bottom right, 2 is bottom left, 3 is one directly up from pin 1 and so on.

And if anyone really wanted to, pin headed J16 is for connecting PS2 mouse and keyboard.

J8 - USB 2.0 port connector
J13 - Serial port connector
J2 - CF card master or slave mode selection (in theory could wire a two position switch to this pin header and manually boot either from the cf card or SATA (J3/J4) ports. Bios will boot from first boot medium. If cf is set to slave it will boot sata first over cf. Then if you wanted to quickly switch back over to default OS, move the two position switch over, that or just move the jumper)
J11 - AT/ATX mode selection
CONN1 - AT mode power button connector

CPU is LGA775 socket
Ram is 200 pin DDR2 800/667 memory. Max supported is 8GB.

PCIe slot is an PCI-Ex8 Golden Finger. You would need to find the riser card to actually attach a PCIe card. But it supports AGP or RAID controller.

I looked on eBay. The cost for a used one is between $100-200.

Does the front LCD display retain any functionality when running pfSense?

Is the 10/100 port shown on the front of the box really only 10/100 ?

Does that mean of the 6 ports then first port is wan and the rest are lan ?

Seems a great little box to convert :slight_smile:

I have 2 m440s that I saved from trash. Think they will work?

I have done something similar like this but I am unable to get the XTM 5 to boot from the SSD. I’ve been googling that the XTM will only boot to CF card. You need to unlock the bios. The links don’t work anymore(from 2016). Does anyone have info about this?

This is awesome. I just picked up an old XTM 515 from work and am working on setting this up.

Are you able to provide more detail one what needs to be done with the original fireware OS? I hooked up my PFsense hdd to the xtm box but it still booted into the original watchguard system. Am I missing a step where I overwrite the original config?

Thanks!

I mean, if you have the xtm already, maybe it’s a fun project, but years ago I just wanted to build a great firewall from start, so I picked up a dq77kb and a 3470t (aes) 35tdp and 4GB of ram. Threw in a msata for boot and a 2.5in spin platter for logs, rewriting and cache. Plinkusa 1u chassis, i’m in like 300$ total, but it splits wigs.

Yes, even older models like the x750e run pfsense, XTM 5 Series, M400 and M500 Series and I guess M600 Series. They are true x86 boxes. Hopefully will get a hand on one of the newer ones. It´s worth looking for these on eBay.

M200 and M300 Series runs on ARM, as well as XTM 2 and 3 Series.

They make 1G in all directions LAN <> WAN and LAN <> LAN. But there is no webfiltering, proxy or stuff like that currently active. I mainly use them at the moment to create 5 separated networks, inter VLAN routing on specific IPs, DHCP for all networks, DNS and OpenVPN for 4 mobile clients.

Thanks a lot :slight_smile:

Never heard of these boxes before, but they do look quite interesting and so the specs of your mentioned models. I guess hardware wise they are indistructable and very reliable. Very interesting with these addons cards and what you are using the machines for :slight_smile:

The support drops in the moment, when you tell them that you have no active warranty/support/etc, same with HPE for instance when I had a storage where the carepack expired about 2 months before my call, but thats how they make money. pay for support or get a product with an active/extended support.

out of curiosity, how did you end up doing this? Were you able to have it load a custom recovery image including the OS you wanted, or did you do something more direct like writing over the built-in boot device?

I´m getting 1Gbit/s download and 500 mbit/s upload. Both values are limited by my contract. I guess it´s also capable of doing 1Gbit/s upload too. My current active box has just a Celeron 440 @ 2.00 Ghz Single Core and 4GB RAM, bc my XEON box is down for maintenance :smiley:

I ain´t doing any webfiltering or so on. There must be an issue somewhere else in your configuration. How´s the load on the CPU when you run a speedcheck. If you have different networks on different interfaces, how is the speed when moving files between networks?