Repurposing an outdated Xeon workstation into a quiet, high efficiency, high speed (10Gbps) TrueNAS server

[Honken]

It really is a shame that, as far as I know, Intel has ditched the idea of integrating their Xe cores into any modern Xeon. They'd make a fantastic home server, where Intel's modern iGPUs could handle video transcoding much better than their competition.

For just some sort of a video out, an nVidia 710 makes a lot of sense. I think I have a completely fanless one here in a drawer somewhere.

 

[dualazmak]

Repurposing an outdated Xeon workstation into a quiet, high efficiency, high speed (10Gbps) TrueNAS server
Part-07: Present status of the TrueNAS ZFS-RAIDZ2 SMB share storage and complete backup platform


Hello, dear ASR friends and colleagues,

Present status of the TrueNAS ZFS-RAIDZ2 SMB share storage
As I have repeatedly mentioned in previous posts on this thread, the current TrueNAS ZFS-RAIDZ2 storage configuration still remains in the testing, evaluation, and validation phase.

I have intentionally mixed HDDs of different brands (Western Digital, Hitachi) and capacities (two 6TB drives and three 4TB drives) to verify the robustness and durability of a RAIDZ2 storage pool even composed of these five "non-standard, or odd" combination of drives.

Of course, I well understand that for capacity-wise the RAIDZ2 pool is equivalent to “five 4TB HDD configuration” and hence 2TB areas in two 6TB HDDs are not in use in present RAIDZ2 storage pool.

Consequently, the total capacity of the RAIDZ2 pool is 12TB (or 10.92TiB) and I can use 10.63TiB storage space as SMB share storage which is quite acceptable for my present test and validation purposes.

I have been much impressed that, even in such somewhat "non-standard, or odd" HDD configuration for ZFS-RAIDZ2, so far the SMB storage has been very much stable and robust with no issue at all; I have already transferred (written) total ca. 7.79 TiB data —including small jpg photos to huge VHDx disk image files (ref. Part-01 post)— now occupying 73.3% of the available SMB share storage space of 10.63TiB, as shown in Fig.36 below.

TrueNAS OS is suggesting (or alerting) me that we may better to limit the used space of the SMB storage to be below 80%, so in my case below 8.5TiB, to maintain stable and efficient operation of ZFS-RAIDZ2.

I like to accept and obey such suggestion, and it is easy for me to accomplish it for the time being; I have already saved four (4) generation of the “bimonthly Windows-OS-drive VHDx images” of nine (9) client PCs—means I can go back to eight (8) months into the past—, and I can erase/delete the oldest ones after transferring new ones into the “VHDx folder” in the TrueNAS SMB share storage.

Please also note that the 64GB RAM in the Xeon TrueNAS server, working efficiently as RAIDZ2 cache, is still non-ECC type (I have plan to replace it with ECC RAM), and I have not yet installed M.2 NVMe SSD for further advanced caching player. I will discuss the “needs” (or not) for ECC RAM and M.2 NVMe SSD in my coming Part-08 post.

After intensively continuing further test and evaluation, I will replace all the five (5) HDDs with 8TB (or 10TB) Western Digital Red server HDD, hopefully within a year or two; the expected market trends for such replacement will be also discussed in my coming Part-08 post.


Complete backup platform for ZFS-RAIDZ2 SMB share storage
I well understand that “complete backup platform” for NAS SMB share storage is always one of the critical and indispensable aspects in any kind of NAS server operation.

For this purpose, many people use USB-connected multiple-HDD boxes and/or even second identical NAS server acting as “automatic” backup (or complete replication) platform and emergency replacement.

In my present and future NAS sever operation, however, —where I power-on the NAS server only for ca. 8 hours in a week, and never operate 24-365 non-stop manner— I do not need the second NAS sever nor multiple-HDD USB box; I can utilize, alternatively, a stone-age generation (very old-generation) client PC having high efficiency I/O throughput via 10GbE LAN “dedicating” for the complete backup as well as benchmark tests for TrueNAS SMB share storage.

Please be reminded that, in my present post series, I repeatedly mentioned about ASUS B75M-PLUS M/B client Windows 11 Pro 25H2 PC —intentionally selected (from my junk PC drawer) stone-age generation M/B— for benchmark tests, and it always gives/gave 650 – 790 MB/s sequential I/O results via 10GbE LAN to TrueNAs SMB share storage benchmarked by CrystalDiskMark.

I decided, therefore, at least for the time being before the full 8TBx5 HDD installation in the NAS server (or even after that), to use this client PC “dedicating” not only for benchmark tests but also for the complete backup platform for the TrueNAS SMB share storage.

Furthermore, in this client PC building, I continue my present policy of “to build everything in reasonably affordable manner” in present “Xeon 10GbE TrueNAS server project”; all the parts for such client PC need to be found in my junk boxes and/or drawers, and I do not purchase even affordable chassis, but to build it in open-frame configuration in “again” near-silent shape since I will place it in one corner of my SOHO office upstairs.

Fortunately, in my near-junk HDD/SSD storage desiccator box, I could find “healthy” four (4) 3.5” HDDs; one WD-RED 6TB, one WD-RED 4TB, one Hitachi 4TB, and one WD-RED 2TB, all can be used as complete backup HDDs in the client PC for TrueNAS SMB share storge, even though I will rarely use the 2TB HDD in my backup purpose.

The simple use of 6TB, 4TB, 4TB HDDs in NTFS format will provide total 14TB backup space; not in any RAID nor JBOD configuration, I can easily and semi-automatically (using automatic batch command of file copy tools) perform folder-by-folder “differential” backup of SMB share contents into the three backup HDDs in the client PC.

Once the complete backup has been done, the subsequent periodic “differential backup” will be accomplished within rather short time thanks to the high efficiency I/O throughput — 650 – 790 MB/s sequential I/O — via 10GbE access to the True NAS server SMB storage.

Based on above “strategy and policy”, I assembled the ASUS B75M-PLUS client Windows 11 Pro 25H2 PC in open-frame configuration on affordable kitchen dish cages, capable of near-silent operation, as shown in Fig.37 below for which you surely have LOL! Please note all the parts have been picked/discovered in my junk boxes/drawers except for the kitchen dish cages.

Using the excellent file copy tool “FastCopy ver 5.12.0” and its batch commands, I could easily prepare reliable batch command sequences (e.g. ref. here) for complete backup and subsequent differential backup of the whole TrueNAS SMB share contents —folder-by-folder— into the three HDDs (6TB, 4TB, 4TB all in NTFS format) of this stone-age generation client PC via the 10GbE access to the TrueNAS server; the backup procedure always proceeds very efficiently surpassing my anticipations.

An example of differential (Size/Date) backup command of FastCopy is;
FastCopy.exe /cmd=diff /auto_close "Z:∖folder-A∖" /to="D:∖SMB_Backup∖"
In this example, the drive letter of the network-drive-mounted SMB share storage is Z:, and the drive letter of the client backup HDD is D:
This example command will do differential copy of all the "folder-A" and its subfolder(s) in drive Z: (TrueNAS SMB) to go under the "SMB_Backup" folder in drive D: of the client PC.
Of course, you need to place the batch command(s) .bat file within client PC's FastCopy installed folder, or another folder, where FastCopy.exe exists. You can find the FastCopy's Command Line list here on its official site.

Another example of sync (Size/Date) backup command —delete the files in backup HDD which have been deleted in SMB share storage— is;
FastCopy.exe /cmd=sync /auto_close "Z:∖folder-A∖" /to="D:∖SMB_Backup∖"

In any way, if you would like to use FastCopy and its command lines, I highly recommend you to fully try/test/validate the commands you wrote using tentative test folders in SMB share storage and in your backup platform for your proper understandings of the differential and sync copy behaviors.


Please stay tuned; my coming Part-08 post will focus on “The Way Forward” with my “Xeon 10GbE TrueNAS server project”.

 

[dualazmak]

Repurposing an outdated Xeon workstation into a quiet, high efficiency, high speed (10Gbps) TrueNAS server
Part-08: “The Way Forward” with present “Xeon 10GbE TrueNAS Server Project”

Hello, dear ASR friends and colleagues,

Throughout this post series, I believe I have been able to share the initial hardware and software configuration of the present “Xeon 10GbE TrueNAS Server Project,” which already provides SMB I/O performance surpassing my expectations.

I will be slowly and steadily implementing subsequent several possible features, in the near future, towards the tentative “finalization” of the 10 GbE TrueNAS server; the features to be discussed one‑by‑one in this Part‑08 post are as follows:


64GB (8GB x8) ECC RAMs to replace present non-ECC ones, or not
I repeatedly mentioned and shared that the present DDR2 non-ECC RAMs (8GBx8=64GB) are so far working with no issue at all in the TrueNAS server built on ASUS P9X79 WS M/B, not only for TrueNAS OS operations but also for efficient data caching layer in ZFS RAIDZ2 configuration.

I am well aware that the use of ECC RAM is generally recommended for NAS systems, especially for heavy‑duty 24/7 (i.e., 365-day continuous operation) with frequent multi‑client access or additional server workloads. I understand this is especially true for rather heavy-duty NAS utilization like 24/7 non-stop operation, for frequent simultaneous I/O access from multiple client PCs as well as for many add-on server software implementations such as music server plug-in.

My present (and future) objectives and requirements of the 10GbE TrueNAS server, however, is mainly “data-storage-oriented, and not for multi-purpose “active” server functionalities. This means I will not operate the NAS server in a 24/7 manner, I will power-on only for ca. 8 hours in a week, and simultaneous multiple I/O from client PCs —say, simultaneous “heavy-duty” access from more than two client PCs— would be rarely happening. I assume that my rather “gentle” access to the TrueNAS server is one of the reasons for “no-issue-at-all operation” with present non-ECC RAMs.

Furthermore, I found this interesting web article frankly discussing “the need or not of ECC RAM” in home- or SOHO-use TrueNAS server; I hope your web browser will properly translate the page into your preferred language.

In my specific use‑case—short weekly operation, single‑client heavy transfers, and no 24/7 workloads—the practical benefit of ECC may be smaller than in typical NAS deployments. Still, I continue monitoring the used market for affordable ECC modules fit for P9X79 WS namely unbuffered (UDIMM) ECC 8GB DDR3 240-pin RAM; I need 8 RAM modules of the same lot.

I would highly appreciate hearing your thoughts, experiences and recommendations on “the actual needs” of ECC RAMs in my XEON 10GbE TrueNAS project.


M.2 NVMe 1TB or 2TB Caching SSD
My current RAM‑based ARC cache already performs surprisingly well, so the NVMe cache remains an optional upgrade rather than an urgent requirement.

Even though I have already installed PCIe “adapter” for M.2 NVMe SSD in the Xeon TrueNAS server for data caching in ZFS RAIDZ2 operation (see my Part-03 post), “the actual need for M.2 NVMe caching SSD”, or not, seems quite similar to what I mentioned above on ECC RAMs.

As I mentioned in my Part-03 post, most of the 64GB non-ECC RAM works quite efficiently as data cache for ZFS RAIDZ2 operation, and hence I am still not fully in confidence for the immediate need for caching M.2 NVMe SSD.

By the way, the largest single data file I would routinely transfer/write into the SMB share storage of the 10GbE NAS server is ca. 600GB VHDx (thin-provisioning VHDx derived from MBR-booting 2TB SSD of Windows 11 OS plus various datafiles in a notebook PC).

I assume that installing a 1TB or 2TB M.2 NVMe SSD as a cache device—large enough to fully absorb my routine 600GB VHDx transfers—would allow the 10GbE link to operate at saturation (around 800 MB/s in my setup). This would “liberate” the client PC much earlier, while TrueNAS continues writing from the NVMe cache to the RAIDZ2 HDDs in the background.

Are my above understanding and expectation correct?

As for selection and durability of M.2 NVMe SSDs for NAS cache (as of September 2026), I found this informative web article. It is interesting for me that the reporter uses/refers CrystalDiskMark for benchmarking NVMe SSDs.

In any way, again, I would highly appreciate hearing your thoughts, experiences and recommendations on “the actual needs” of M.2NVMe caching SSD in my XEON 10GbE TrueNAS project, even though I will carefully follow the pre-owned market trends, especially price-wise, for previous generation 1TB or 2TB M.2 NVMe SSD to be used in the TrueNAS server.


8TBx7 or 10TBx7 Server HDDs (five for ZFS-RAIDZ2 pool, two for reserve)
This is “the must way forward” in my Xeon 10GbE TrueNAS Project; I would like to replace all the present test/evaluation RAIDZ2 HDDs —now 6+6+4+4+4TB odd HDDs in the ZFS RAIDZ2 pool— by new 8TB (or 10TB) WD-RED server HDD.

BUT! what a “crazy” market situation right now for HDDs, SSDs and RAMs; the prices still remain extraordinarily “crazy” high level, and the very limited availabilities look to be continuing for the time being, several months or years ahead, as we are having much discussions on the thread hosted by @TonyJZX entitled “is anyone affected by the rampocalypse? gpupocalypse? nandpocalypse?” where I briefly participated with my post #23.

On the other hand, many of the web articles on this regard tell me that the enterprise sectors, mainly AI related industries, are simultaneously planning and actually moving towards replacing their HDD-based data storage platform with advanced enterprise-oriented high-capacity NVMe SSDs, mainly because of much higher capacity-densities per physical space and save-the-earth requirements.

This means that, once such enterprise high-capacity SSDs will start to be provided in flooding manner, the pre-owned still healthy HDDs may be released to the used market in numerous quantities to which we as end consumers can access at affordable prices; I hope such timing would come to a reality within the coming two years. I think we may expect the same trend for the used-while-healthy previous generation M.2 NVMe SSDs.

In any way, “now” in early September 2026, is definitely not the timing for purchasing brand new server HDDs, especially in my present Xeon TrueNAS Project of “repurposing and affordable budget” policies. I will be waiting for the possible steep price-down at least until the coming Black-Friday period in November, or until further later; healthy and guaranteed preowned enterprise 8TB or 10TB HDDs will be welcome to my present Xeon 10GbE TrueNAS Project.


Complete backup platform for ZFS-RAIDZ2 SMB share storage
As I intensively discussed and shared in my previous Part-07 post, for the time being, I decided to continue using the old-but-efficient Windows 11 Pro PC for complete back up and benchmark platform for the TrueNAS SMB share storage, even after the installation of 8TBx5 (or 10TBx5) HDDs.

Please note that after I would have 8TBx5 HDDs, present 6+6+4+4+4TB HDDs will be kicked out from the TrueNAS server, and they will be able to serve backup HDDs together with present 6+4+4TB HDDs on the backup client PC.

Since the ZFS RAIDZ2 SMB capacity with five (5) 8TB HDDs will be ca. 22TiB, the possible backup platform consists of “existing” 6+6+6+4+4+4TB HDDs, i.e. six-(6)-HDD-array of total 27TiB in simple NTFS format, will be more than enough in my Xeon 10GbE True NAS Project.

For such HDD array of six (6) HDDs, it may be better to replace the present 240W PSU in the client PC by new 500W PSU enabling more-than-enough electricity supply for startup simultaneous spinning-up and stable robust utilization of all the simultaneous HDDs backup operations, though.

I have no worry about the shortage of SATA3G port numbers on the M/B ASUS B75M-PLUS (it has 5 SATA3G ports) since I could already find 4-port PCIe x1 SATA3G expansion card in my junk drawer which can be installed onto PCIe x1 slot of the M/B ASUS B75M-PLUS.


UPS (Uninterruptible Power Supply) for the Xeon 10GbE TrueNAS Server
This would be another “must” item, if I (we) will operate our NAS server in 24/7 continuous mode.

In my present Xeon 10GbE TrueNAS Project, however, the possible purchase of UPS is at the bottom of the priority list since I will power-on the server only ca. 8 hours in a week, and I will definitely do not power it on during predicted thunderstorm weather conditions.

Furthermore, I already have, and I will always have, the latest complete backup of the whole SMB share storage in the dedicated client backup PC.

Even though the priority is not so high, I have already identified —through discussion with Co-Pilot— a UPS unit that perfectly fits my Xeon 10GbE True NAS server; I mean a domestically manufactured unit in Japan, with enough use-case reports, well validated compatibility/affinity with TrueNAS OS, having nice warranties and free first battery replacement.

For your reference, during the past two‑month test/evaluation period, I experienced one unexpected power loss caused by a very distant thunder strike. Fortunately, the TrueNAS server restarted normally with no issue reports, which again impressed me with the robustness of ZFS RAIDZ2.

By the way, for ordinary “shutdown” of the TrueNAS server, I always confirm that any of the access LEDs for five HDDs is not blinking at all; I can easily check it since the five (5) HDDs are loaded in the front-access HDD cage with I/O access indicator LEDs (ref. Fig.01 in Part-01 post), and the TrueNAS server sits under my office table. Consequently, after such non-HDD-access confirmation, I place my “order” of shutdown to the TrueNAS OS.


Prior to closing this Part-08 post, for your convenience of hyperlink jumps, let me look back the titles of my post series so far shared:
Repurposing an outdated Xeon workstation into a quiet, high efficiency, high speed (10Gbps) TrueNAS server
Part-01: Introduction, background, requirements, test and validation platform, initial SMB access evaluation

Part-02: Selection, flashing, and implementation of HBA Card in the TrueNAS NAS server

Part-03: An “intermezzo” of the post series: PCIe adapter for M.2 NVMe SSD and cooling fans on its heat sink

Part-04: Selection and implementation of 10 GbE NIC (Network Interface Card) in the TrueNAS server

Part-05: Selection and implementation of 10 GbE NICs (Network Interface Cards) in client Windows PCs, and silent cool-running 10GbE switching hubs

Part-06: An “intermezzo” of the post series: one-slot thickness simple VGA card in the True NAS server, and summary of optimized PCIe slots utilization

Part-07: Present status of the TrueNAS ZFS-RAIDZ2 SMB share storage and complete backup platform

Part-08: “The Way Forward” with present “Xeon 10GbE TrueNAS Server Project” (this post)


This Part-08 post serves as "Finale" of Act-1 on this thread.

Please stay tuned, however, since after a few months of "Intermission", I will resume the Act-2 of this thread with my "Preludio" post Part-09 when I would make a major next step forward in present "Xeon 10GbE TrueNAS Project".

Until then, I mean even during the coming "Intermission" period for my post series, of course your comments and our discussions will be highly welcome!
_
_

 

[voodooless]

As I mentioned in my Part-03 post, most of the 64GB non-ECC RAM works quite efficiently as data cache for ZFS RAIDZ2 operation, and hence I am still not fully in confidence for the immediate need for caching M.2 NVMe SSD.

By the way, the largest single data file I would routinely transfer/write into the SMB share storage of the 10GbE NAS server is ca. 600GB VHDx (thin-provisioning VHDx derived from MBR-booting 2TB SSD of Windows 11 OS plus various datafiles in a notebook PC).

I assume that installing a 1TB or 2TB M.2 NVMe SSD as a cache device—large enough to fully absorb my routine 600GB VHDx transfers—would allow the 10GbE link to operate at saturation (around 800 MB/s in my setup). This would “liberate” the client PC much earlier, while TrueNAS continues writing from the NVMe cache to the RAIDZ2 HDDs in the background.

Are my above understanding and expectation correct?

No, they are not! As I already mentioned earlier, such a ZIL/SLOG cache isn’t effective for a NAS workload because writes are not synchronous. So it will effectively not be used at all. And configuring SAMBA for synchronous writing will severely degrade performance regardless of SSD usage. You can do metadata storage on SSD, which will make overall feeling faster, especially while browsing, but that’s about it. If you want accelerated writes, then use MergerFS. It worlds great, but it’s only effective for new files. Once the files are on the HDDs, any rewrites takes place there.

As for ECC, why bother? You run the thing for 8 hours per day. Hundreds of millions of laptops, phones and PCs run just fine without ECC memory. ZFS will save you from most data corruption issues that might occur from broken memory. You’ll be fine.

Another thing to consider: a daily boot also means that the HDDs start and stop every day. And this is actually a big factor in reliability. Running those disk 24/7 will be much more reliable and mechanically stable.

As for the UPS, it seems you would need a tiny one, just enough to shut down the server in case of power loss. I also use a small UPS which I only need for a few 100ms while my house would switch to battery backup. Works perfectly fine.

 

[phofman]

I run ZFS on a few servers, as well as on my linux workstation (incl. root, booting from the great ZFSBootMenu). While ZFS guards against faulty RAM, it will put the pool to read-only when regular scrubbing detects too many issues. That's safe for data, but inconvenient for work, so good memory is of major importance. I would use ECC if possible, e.g. second-hand registered DDR4 if the MB allows it (not cheap these days either, but less expensive than other alternatives). On my non-ECC workstation I added a pair of DDR4 DIMMs, despite he same params as the existing pair (though not the very same model), the scrub threw errors. In fact Memtest did too, after a few hours of run.

As of offline backups - I can only recommend the combo sanoid/syncoid. Having an automatically maintained history of zero-overhead local snapshots is extremely handy - one appreciates that fully when the moment of deleted file/work strikes. Syncing filesystems to a pool of offline drives is simple (some non-server-hw machines use hot-swap cages for backup SATA drives, installed in 5 1/4 slots of the cases, my workstation is OK just with one 6TB drive in USB3 cradle. I have a simple script which takes care of the mounting, running syncoid, unmounting, trivial to use. The larger machines use two sets of offline drives (rotated weekly), automatically selected by their partition label name. Syncoid supports that directly (it's just adding the set ID to its snapshots names). I guess the other backup tools offer that too.

Yes ZFS is a great helper.

 

[voodooless]

I would use ECC if possible, e.g. second-hand registered DDR4 if the MB allows it

He has DDR2 now, so no, DDR4 won’t work. But second hand ECC memory is cheap, probably below $€£ 100 for 64GB in the configuration needed.

Motherboard spec says it DDR3.. well there goes the price advantage then…

 

[phofman]

To be honest, I do not read those extremely long posts in detail.

About a year ago I dumped like 128GB of registered DDR2 DIMMs (full bag of them) from DL580s gen5, ebay was flooded with them (those 4GB PC2-5300Fs). I would not have thought they could be useful But of course they require a specific hw, useless for a standard motherboard. I would not want to pay the electricity they consume, when one can get a dirt-cheap fanless DDR4 thin client with comparable computing power (likely more, RAM-wise).

 

[voodooless]

I would not want to pay the electricity they consume, when one can get a dirt-cheap fanless DDR4 thin client with comparable computing power (likely more, RAM-wise).

That’s the whole problem with such a system: it’s sounds like a lot, this 12 core Xeon with 8 ram slots. But in reality, an entry level Ryzen CPU (or even a mobile variant) with two sticks of DDR4 or 5 will crush this thing on performance by a wide margin. It’s just 13 year old technology, made in a process node 10x larger than the latest gen CPUs. Something has to give. But on the upside, these kind of systems are cheap to get, and they do still offer respectable performance for such workloads. @dualazmak is contemplating a 500W PSU, meanwhile, I’m running my NAS from a simple power brick That’s quite a contrast.

 

[phofman]

I understand buying a new HW is expensive. But used enterprise-level workstations/thin clients are just as cheap as old server hardware, yet incomparably less consuming. My favourite one for a server size (i.e. with 3.5 HDDs) is workstation FS Celsius M740 - dirt cheap refurbished, 8 slots for cheap (used to be ) ECC RDIMM4, abundance of PCI-e slots for NVMe M2 adapters and additional cards if needed, some low-power Xeon, very nicely designed with only two very large fans (front-back tunnel like proper servers, no CPU fan, plus PSU fan), almost silent. As a corporate workstation it's designed for 24/7. At idle it takes < 50W (incl 2 HDDs), some 120W at full CPU load which never happens. Mine in the office has uptime of 180days, but only due to power cuts It sits below desk and only the admin who every week inserts two tray-less hdds for offline backup knows about it There are two 6TB HDDs inside for large-volume ZFS-mirror pools (slow files - fileserver, mail server), 2NVMes for fast-storage ZFS-mirror pools (fast databases, it also works as an in-office slave for our online production databases + testing server), and two 3.5" hot-swap trays for offline backup (2x 6TB zfs stripe) of all the ZFS pools. IMO it's an overkill for a home server.

 

[dualazmak]

Hello, @voodooless and @phofman;

Thank you very much for your kind attention to this project thread, and for your invaluable comments, suggestions, and information.

First and foremost, please understand that I am strictly limiting this project to the scope defined by the thread title — namely, repurposing the outdated P9X79 WS workstation into a 10GbE TrueNAS server, mostly for data‑storage purposes, in a reasonably affordable way.

Furthermore, I power on the TrueNAS server only about eight (8) hours per week (not per day). Its electricity consumption under normal workload is around 110 W, so I care little about the electricity bill attributable to this server.


ECC RAM, or not
After carefully re‑reading post #35 kindly provided by @voodooless, and considering the comments from both of you, my decision at this point is that I can simply “forget about implementing ECC RAM” for this TrueNAS project.

I essentially agree with the stance of @voodooless and the contents of the referenced web article.


M.2 NVMe SSD for ZFS caching?
I thank you again, @voodooless, for your clarification in this regard and your suggestion of possible "way forward" to MergerFS. I quickly reviewed a few nice web articles introducing and recommending MergerFS plus SnapRAID, and I found the combination looks quite attractive.

However, for my present 10GbE TrueNAS project and its objectives, I feel MergerFS‑plus‑SnapRAID is “too much” and outside the scope of my current test‑evaluation phase. Also, I prefer to keep my “Linux‑shell and NAS‑OS command‑line world” as small as possible. Writing batch commands for Windows is perfectly fine for me, though.

Just like your system, I already have efficient 64 GB RAM, a 10GbE NIC, and a 10GbE LAN, enabling average 670–800 MB/s sequential I/O (CrystalDiskMark, default 4TiB 4GiB parameters) from client PCs to the NAS SMB storage.

Therefore, my present conclusion is that I will not use an M.2 PCIe NVMe SSD (4 TB or 8 TB) as a cache layer for the ZFS RAIDZ2 HDD pool. Instead, I will use it as a simple “additional” storage pool providing high‑speed SMB access, while the HDD RAIDZ2 pool remains the main SMB storage.

Client PCs can access the main SMB storage mounted as drive Z:, and the high‑speed SSD SMB storage mounted as drive X:.

For huge data transfers (e.g., larger than ~50 GB), the client PC will write to drive X:, achieving near‑saturation throughput over 10GbE and finishing its workload quickly. The transferred files can then be moved into the main SMB storage within the NAS using batch commands or similar tools.

My dedicated Windows client PC — used for backup, benchmarking, and remote control — can also perform these internal data moves via batch scripts.

All weekly data transfers, internal moves, and differential backups can be easily completed within the eight (8) hours of weekly NAS operation, together with the “gateway” client PC.


Complete offline backup platform
Thank you, @phofman, for sharing your backup practices.
In my present project, I will continue using a dedicated Windows 11 PC with several NTFS‑formatted HDDs for complete offline backup of the NAS SMB storages via 10GbE LAN, as described in Part-07 and Part-08.


UPS for NAS server
The candidate UPS unit I have identified — OMRON BW55T — perfectly fits both your comments and mine in Part‑08.
__

 

[voodooless]

When using a separate fast SSD share, you could just create a poor man’s MergerFS by having a simple rsync script copy the files from SSD to the HDDs once every day for safe keeping.

At 8 hours per week, I would not worry too much about on-off cycles of the disks. They will last for quite a long time like that.

 

[voodooless]

the candidate UPS unit I have already identified is OMRON BW55T which perfectly fit for your comment and mine shared in Part-08 post.

Note that this is a standby UPS, it needs to switch to battery power, and this takes 10ms worst case. ATX spec says a PSU should handle upto 17ms to interruption. So in theory you should be fine. I would make sure to use a good quality PSU, though, just to make sure that it meets spec.

Alternatively a different UPS type would take away the switch time. Maybe opt for BA75T?

 

[dualazmak]

Hello, ASR friends and colleagues,

Today, I happened to come across with this nice web article entitled "How to Build a NAS With TrueNAS Scale: 10 Steps [2026]" written by Elias Virtanen, dated August 15, 2026, within TECH INSIDER site.
https://tech-insider.org/how-to-build-a-nas-with-truenas-scale-2026/

I was both greatly impressed and much surprised to find that the contents of this web article dated August 15 align so closely with "my" present "Xeon 10 GbE TrueNAS Server Project" I am currently working on — started in April 2026 (summary ref. my post #1) — and sharing on this thread.

Going beyond the recommendations and suggestions in the web article, my project on this thread implements;
(1) Xeon E5-2697 v2 CPU 12-core 24-thread,
(2) 64GB RAM,
(3) a 2-port 10GbE server NIC,
(4) a PCIe adapter for M.2 NVMe SSD with push-pull dual near silent fans (4TB or 8TB M.2 NVMe SSD not yet installed, though),
(5) optimal PCIe utilization topology,
(6) very quiet (near silent) several cooling fans with RPM controls —the TrueNAS server is less than 25 dB(A) at 1m—,
(7) silent 10GbE switches,
(8) silent 10GbE NICs in client PCs,
(9) an independent silent 10GbE LAN using Cat6A cables for fast data transfer to/from the TrueNAS server as well as among the client PCs,
(10) complete backup platform for TrueNAS SMB storage in a gate-way client PC having several HDDs.

 

[dualazmak]

Complete scrub time, intentional resilvering time for one 4TB HDD replacement

Hello, dear ASR friends and colleagues,

During present “intermission” period of this thread, I have been intensively testing/evaluating the stability and robustness of ZFS RAIDZ2 pool in the TrueNAS server — Xeon E5-2697 v2 12-core 24-thread, Mem 64GB — intentionally consists of mixed 5 HDDs of different brand, different capacity, different rpm;
(1) WD60EFRX 5700rpm,
(2) WD60EFRX 5700rpm,
(3) WD40EFRX 5400rpm,
(4) WD40EFRX 5400rpm,
(5) Hitachi HDS724040ALE640 7200 rpm.

Time needed for complete “scrub” of the whole RAIDZ2 storage pool
I know well that the total time for complete “scrub” (full maintenance) of a RAIDZ2 storage pool depends on the RAIDZ2 configuration, i.e., size of each HDD, total pool capacity, total data size already wrote, as well as numbers of small to huge data files, etc.

At present, my RAIDZ2 pool consists of intentional mixture of 5 HDDs as described above (6+6+4+4+4TB), total pool size is 10.63TiB, into which I have already transferred total 8.22 TiB data files of small sizes (a few kB) to huge ones (up to ca.520 GB VHDx OS SSD image) occupying 77.4 % of the SMB storage.

I decided to perform the complete “scrub” of the RAIDZ2 pool monthly, and just for your (our) possible reference, it takes about 7.5 hours with no other I/O access to the NAS.


Time needed for rebuilding/resilvering one HDD replacement after cold-swapping
Even though the above complete “scrub” completes successfully with no issue at all — means all the 5 HDDs are still completely healthy now —, I decided to test/evaluate one HDD replacement (cold-swap) and “resilvering” within the present RAIDZ2 pool; in the TrueNAS community, the operation known as "rebuilding" in standard RAID systems is referred to as "resilvering”.

After shutting down the TrueNAS server, I intentionally removed one of the 4TB HDD WD WD40EFRX, and inserted old but healthy/empty 4TB HDD Seagate ST4000DM000.

Consequently, upon rebooting the TrueNAS server, the “Storage” panel alerted that one of the five HDDs was missing/unavailable while all the data have been being completely secured. Then, I went into the HDD replacing procedure by directing the newly inserted ST4000DM000 as the new member of the RADZ2 pool by ordering “resilvering”.

While no other I/O access was going, under my present RAIDZ2 configuration and status, the “resilvering” process took about 13.6 hours with no issue at all.

After the “resilvering”, the “healthy” ZFS RAIDZ2 pool configuration is;

(1) WD60EFRX 5700rpm,
(2) WD60EFRX 5700rpm,
(3) ST4000DM00 5900rpm,
(4) WD40EFRX 5400rpm,
(5) Hitachi HDS724040ALE640 7200 rpm,

enabling 10.63TiB SMB share storage.


It is well known that we can repeat such “resilvering” process furthermore with larger capacity HDDs, for example five 8TB HDDs, and when the final fifth “resilvering” will be completed, the RAIDZ2 pool capacity will suddenly expand to 24TB enabling SMB share storage of ca. 21TiB.

From the points of time and electricity consumptions as well as I/O workload and data safety, however, it would not be feasible/recommended to apply such repeated "resilvering" for the purpose of enlarging the RAIDZ2 capacity since, according to my above result, such five times "resilvering" will take so long as ca.13.6 x 5 = 68 hours with very busy I/O workload in all the HDDs.

Consequently, after I will purchase 5 (five) of new 8TB server HDD, it should be better to just build completely new RAIDZ2 pool at once enabling empty 21TiB SMB storage, and to write/transfer all the total 8.22TiB data from the complete backup platform — a client gateway PC having several NTFS HDDs of average 100MB/s sequential read speed, ref. #42 Part-08 post— via 10GbE LAN; the total write-back process can be finished within 30 hours using simple batch commands.
_

 

[rcstevensonaz]

Furthermore, by ultimately configuring a RAIDZ2 setup using five 8TB WD Red HDDs

I'm not sure if this has been mentioned, but be sure you get the right type of WD RED hard drives.

I believe WD has recently split their RED line into sub-lines, one of which is not as well suited for ZFS applications.

 

[dualazmak]

I'm not sure if this has been mentioned, but be sure you get the right type of WD RED hard drives.

I believe WD has recently split their RED line into sub-lines, one of which is not as well suited for ZFS applications.

Thank you for your point.
I too have heard that story; I should be careful enough in selecting the right ones when I will purchase 5 or 8 of new WD RED server CMR HDDs.

The Toshiba N300 Pro or N300 NAS Systems series CMR 8TB 7200rpm HDDs —3-year domestic warranty in Japan— will be also in my scope of selection for ZFS RAIDZ2 pool.

 

[Ruediger]

About the rig picture above and the stack of 4 HDDs with fans I suggest to put more space in between the HDDs (1.5cm) and employ one 14cm fan to pull air through from one side to the other. It should give a very even temperature without hotspots for all 4 drives.

What also works is aluminum enclosures (dissipate heat very good) and one large fan. But for some old drives who cares about protection. Just don't let any PCB touch metal by accident.

 

[dualazmak]

About the rig picture above and the stack of 4 HDDs with fans I suggest to put more space in between the HDDs (1.5cm) and employ one 14cm fan to pull air through from one side to the other. It should give a very even temperature without hotspots for all 4 drives.

What also works is aluminum enclosures (dissipate heat very good) and one large fan. But for some old drives who cares about protection. Just don't let any PCB touch metal by accident.

Thank you for your kind attention on this TrueNAS project thread.
I think you refer to Fig.37 in my post #42, right?

Spoiler: Fig.37 in post #42

I appreciate and understand well about your point.

As I wrote in #42, in each of the three (3) spaces between the stacked four (4) HDDs as well as in bottom of the lowest 4th HDD, I have placed 20x20x5(thickness)mm gel spacers at the four (4) corners of the HDD enabling effective 5mm space of air-through given by the two quiet fans under rpm controls —one is 100x100x20mm front blow, the other is 120x120x10mm side blow—.

I always monitor the temperature levels of the HDDs (and SSDs) by Libre Hardware Monitor's desktop gadget, and I occasionally also use HWiNFO 64 (now ver.8.44-5935). The figures shown below were screen shots taken at 12 hours after regular operation of the backup PC in room temperature of ca. 25 degree C.

_

 

[Ruediger]

For just some sort of a video out, an nVidia 710 makes a lot of sense. I think I have a completely fanless one here in a drawer somewhere.

There are quite some passive cooled GT710 cards out there. But often not very slim, although low profile.

For servers, usually headless systems, VM servers etc. this particular model is a choice because it's only PCIe x1 and very slim.

ASUS GeForce® GT 730 | Graphics Card | ASUS Global

The ASUS GeForce® GT 730 Graphics Card features 4x HDMI ports and passive cooling in a single slot, enabling quiet multi-monitor productivity.

www.asus.com

But if size is not that critical, I would go for the passive cooled MSI GT1030. Newer, more recent driver, better performance.

 

[dualazmak]

There are quite some passive cooled GT710 cards out there. But often not very slim, although low profile.

For servers, usually headless systems, VM servers etc. this particular model is a choice because it's only PCIe x1 and very slim.

ASUS GeForce® GT 730 | Graphics Card | ASUS Global

The ASUS GeForce® GT 730 Graphics Card features 4x HDMI ports and passive cooling in a single slot, enabling quiet multi-monitor productivity.

www.asus.com

But if size is not that critical, I would go for the passive cooled MSI GT1030. Newer, more recent driver, better performance.

I fully agree with you.
In this project, I am very happy that the repurposing ASUS P9X79 WS M/B has so many six (6) PCIe_x16 slots!

Spoiler: Fig.35 in my post #40

Spoiler: Fig.02 in my post #1