This build consists of an Intel Xeon E5-2620 v4, Asus X99-E WS/USB3.1 (SSI-CEB) mainboard, Crucial 32GB RDIMM ECC DDR4 RAM, Corsair AX860i PSU, and a Coolermaster Evo 212 cooler.
I plan to eventually migrate these components into a 4U Rackmount chassis in the future, with hotswap bays and an Extended SGPIO 6Gb/s SAS backplane.
Here are some questions I received on my build-log from the forums, so I thought that would be a good starting point.
What's your ultimate goal for the system?
I wanted to stick to X99 as it is a mature platform. Early on I decided ECC was a requirement that I was settling on as well, this lead me to going Xeon.
From this point, I came across the Asus X99 WS boards - they are on the more expensive side - sure, but they are a cross between the 'gamer/overclocker' aesthetic and server grade components. I like the LED-error code read out, hardware start/reset switches, having a thunderbolt header etc. These are 'additions' a cheaper Supermicro board would generally lack.
The I/O at the back on the Asus X99 WS board is great too, has BIOS flash-back/reset switch. The Asus UEFI is fantastic - so that settled my choice there. I chose one of the cheaper E5 Xeons, enough cores to allow running at least 1-2 VMs if needed.
I went with Crucial 32GB RDIMM RAM; crucial make quality RAM so no issues there. Power supply, I went for a Corsair AX860i, the second unit I own. I also run a HX1000i and a HX750i.
In my other Synology units I'm running WD Red Pros that have served me well over the last 2+ years, and I went with WD Reds as they were cheaper.
One point to note, I went with the Asus X99 WS/USB3.1 board vs the Asus X99 WS/10G board, where the latter has dual-10G NICs (both are Intel X550 grade NICs) but you loose the Thunderbolt header. I wanted to give myself the option of having the FreeNAS box in another room, and run a Corning thunderbolt cable the way Linus did.
I can buy Intel X520 cards for 10G ethernet at around $200 per card; additional cost - sure, but hey at least I have the option of Thunderbolt (in the future).
Why did you choose FreeNAS?
I was keen in ZFS and that was the 'priority'; as far as operating systems go, FreeBSD has a track-record of being solid, considered to have a more robust network stack; it's Unix. Had there been a linux-alternative, I may have gone with that. For my professional work I mainly rely on Debian for production environments.
I didn't spend too much time unRAID; it certainly is there as an option. I do plan to try my hand at setting up a EXSi lab at home, and virtualise other aspects of my lab-stuff. I'd do it mostly out of curiosity, as it's typically cheaper to just go with instances in AWS for any real need, rather than paying EXSi licensing costs.
In any case, running VMs wasn't an initial priority - but I did pick the CPU/Mobo ensuring they would support it.
Thunderbolt - well, VNC is nice, running headless is great (99% of my systems are headless over SSH), but if you want to have a physical terminal, say many meters away, Thunderbolt is a real option. Why? Cause PCs/Servers generate heat/noise. I can move them to my server-rack space downstairs and have a near-latency free console over Thunderbolt if I so ever wanted it.
Do I need it? No. But I have the option. True, that also lets me manage UEFI stuff; an alternative would have been either (i) an IPMI chip on the board or (ii) an IP-based KVM solution.
Belly of the Beast: Corsair Air 740 Case
The tricky part was figuring out a way to get all the drives to sit inside my Corsair Air 740 case.
My solution was to design and 3D print brackets that were optimised to be cost-effective, and modular, allowing to stack 3.5" HDDs without any limit on the total stack size.
Given the design, orientation is also configurable to either mount the drives vertically — and using the included base-plate — or in a typical caddy style setup.
All 3D Printing files are hosted on Github for you to use.
TIL Setting up my FreeNAS 11 Xeon Server
You'll find various tips n' tricks that I picked up along the way detailed on my blog, including how I setup UPS monitoring and ensured that ECC is working etc.