Starting TrueNAS Core/TrueNAS Scale for New Users

[TanKianW]

Foreword:
Decided to start this thread after receiving PMs on DIY NAS recommendations and TrueNAS Core settings. This thread will serve as an extension to my thread sharing on the use/setting up of pfsense in a home environment. In this thread, I will be sharing some practical functions of the TrueNAS Core NAS operating system. This will not be an overly complicated thread which touch on too many advanced functions but cover the basic/important setup and function of using TrueNAS Core to get the new users started.

*Take note that the TrueNAS Core System will be relying on the backbone network infrastructure of pfsense. You can learn more from the thread here:
https://forums.hardwarezone.com.sg/...nic-4/starting-pfsense-new-users-6390714.html

*Since I have started work back in office, I will try to share and update the thread as and when I have time to do so. Therefore the updates could be a tad slower.

Reasons/Objectives for starting this thread:

• Provide hardware recommendations for DIY NAS builders using TrueNAS Core.
• Provide some practical use of a home DIY NAS system
• Provide simple steps to set up your IaaS (Infrastructure as a Service) cloud at home
• Promoting ZFS and opensource. And why should you choose ZFS over other NAS file systems offered by OTS NAS offering from Asustor, QNAP, Synology, etc.
• Provide an alternative to OTS NAS (Eg. Synology, QNAP, Asustor, etc.)
• Promote and encourage forumers to build their own NAS and the fun of setting it up on your own. But at the same time put their trust on their DIY NAS and understand the benefits of TrueNAS. For example: Your can change and replace any hardware that failed on your NAS system few years down the road, without replacing the whole NAS. Easily upgrade your NASes with a 10G NIC, a CPU with more cores and mainboard from different vendors to your liking.
• Tries to reduce the learning curve of using TrueNAS core, to make it more accessible to the average users.

*What is ZFS? And what sets it apart from other file system? What is "impeccable" data integrity!


IMPORTANT: PLEASE WATCH!
*Do read up and plan before deciding your RAIDZ, vdev, pool setup:




Servethehome explains it simply too:
https://www.servethehome.com/an-introduction-to-zfs-a-place-to-start/

The typical dashboard:

Hardware Recommendation:
1)CPU: Either AMD or Intel is fine. Recommend at least 2 core and a low powered CPU. Eg. Intel T or U CPUs etc. You will have to get a CPU that supports ECC memory if you going the ECC route (Eg. Xeon, AMD Zen CPUs, EPYC, etc) However, do get a CPU with 4 or more cores, if you want to run VMs.

2)Memory: I will recommend at least 16GB. For normal home use, non-ECC is ok. But for mission critical and production, I recommend ECC. Most motherboards are picky with memory, especially ECC memory. It is preferred that memory choices are based on the motherboard Memory QVL list. Some good memory brands which I normally build my system on:
Non-ECC: Kingston Value, Micron
ECC: TimeTEC, Samsung, Micron, SK Hynix

3)Motherboard: Get a motherboard with as many SATA as possible. I prefer to go with server mainboards for NAS build rigs. Some server motherboard supports OCCUlink cables for multiple SATA drives support. Some brands I prefer: Supermicro, Tyan, Asrock Rack, Asus. Motherboard features I will look out for:​

• Preferably with 1-2 Intel NICs.​
• At least 4x SATA port. The more the merrier​
• M2 slot for SATA expansion (with adaptor) or a NVMe drive as boot drive​
• Preferably with an internal USB Type A female connector on board (Eg. Asrock rack C236 motherboard)​
• Optional: ECC memory support for mission critical​
• Optional: With onboard display chip, so you can choose a low powered CPU without GPU, if not you will waste the PCIE slot on a GPU. Reason being you might want to add in a 10G NIC on the PCIE slot​
• Preferably mini-ITX size due to space constraint on a NAS chassis. If you going for a bigger chassis, a full size ATX will provide sufficient future expansion.​

4) HDD: Make sure you get a CMR drive that is build for NASes/storage. Do avoid SMR and Western D RED drives, they are not CMR, except for "Plus" and "Pro" series. Try to go with Helium filled drives for lesser hit and better performance at higher capacity (>8TB).
HDD I recommend (no brand support, for your reference only): Seagate Iron Wolf, Iron Wolf Pro, WD HGST drives, Seagate EXO, WD Red Plus/Pro

5)NIC: Most Chelsio, 10Gtek and Intel OEM NICs will work. But I recommend Chelsio since their driver support for FreeBSD has always been good. In fact, most IXsystem's TrueNAS Core pre-build using Chelsio. I will go with a dual 10G SFP+ NIC which you can do LACP lagg.

6)For reference only-10G/Multi-G Network Switches (Home use): Can consider Mikrotik CRS312, CRS305, CSS610, Netgear GS110EMX, Netgear MS510TX.

7)Chassis:You can get OEM cases of TrueNAS system or UNAS cases from taobao. Silverstone also provide several NAS compatible case designs.

8)Power Supply:Go for a Gold Certified power supply if possible. Since you might be running the NAS 24/7, especially if for hosting servers.

9) UPS:
i) I will recommend all NAS owners to connect their TrueNAS system to a UPS even though ZFS is a very resilient file system. Try to get the "multi-socket" UPS so you do not need to use an adaptor.
ii) As I always says, you may not cause the power to trip but your spouse may just trip the whole house power with an oven switch!
iii) You do not need the UPS to run for very long, but long enough for it to initiate a proper shut down.
iv) To work with your TrueNAS to initiate a proper shut down, it must have a communication port which either comes with a USB or a serial port. Most APC BACK-UPS with USB port will have proper driver support on TrueNAS Core.​

Lower Capacity (may be suitable for 1-2 NAS units):
APC Back-UPS BX700U-MS

Higher Capacity for (may be suitable for 1-2 NAS units and a pc):
APC Back-UPS BX1400U-MS

Recommended setup of TrueNAS core system:

1. *Host the TrueNAS server to be access externally through (https) using HAProxy and Let's Encrypt certificate function of pfsense. OR a VPN server on pfsense such as OpenVPN.Read the thread here to learn more: https://forums.hardwarezone.com.sg/130207504-post3.html
2. LACP lagg the NAS to your managed switch which also acts as a failover
3. Connect your TrueNAS system to a UPS. I will be sharing how to configure the APC UPS over USB which you could easily purchase locally. Set the main NAS to master and the other to Slave, so that both will shut down on their own when there is a power lost
4. Schedule HDD scrub every week
5. Schedule Short and Long SMART test every week
6. Set Resilver priority
7. Set up SSH between 2 TrueNAS system to backup each other using Rsync or Replication tasks
8. Set the BIOS on the TrueNAS system to auto power up when power resume. This is important for your system to resume back online after an earlier shut down due to reboot or power lost.
9. Set up email notification to allow your TrueNAS system to send you alerts when it runs in power lost, reboot or critical errors.
10. Always set up 2x boot drive (USB or SSD) so you can boot up from any of the drives when one fails.
11. Schedule tasks for regular snapshots on your ZFS pool
12. Set up a VPN server at home over pfsense. You can choose to use OpenVPN. Feel free to check out here: https://forums.hardwarezone.com.sg/130207493-post2.html

Some of the practical use of TrueNAS core system:

1. Host your personal IaaS cloud such as nextcloud to back up/sync data on your work PCs, mobile devices, etc.
2. Torrenting on the TrueNAS system using qbittorent and manage it on a web interface which you could access from anywhere using VPN on pfsense
3. Sync data across >2 NAS system using Rsync
4. Backup the snapshots between two system using Replication tasks
5. For easy syncing between your NAS, mobile devices and PCs using SyncThing
6. Host your Plex server on TrueNAS system
7. Host the Unifi Controller on TrueNAS system
8. Run VMs on your TrueNAS system
9. Network drive SMB share in your home local network
10. Time Machine like SMB shadow copy, restore data using TrueNAS snapshots
11. Further backed your system up by syncing it to cloud
12. *For enterprise use: Manage several TrueNAS system using TrueNAS scale.

(For Reference Only) An example of a recent TrueNAS Core System build:

• Intel Xeon E3-1245v5
• Asrock Rack C236 WSI (mini-itx)
• 8x 8TB WD RED Plus
• 2x 16GB TimeTEC ECC Memory
• OEM TrueNAS Chassis
• 1x Intel SSD 240G as boot drive via USB3.0
• Gold Certified OEM Flex-ATX PSU
• Chelsio T520-CR with 2x 10G SFP+ NIC
• 2x Noctua 6cm and 1x Noctua 12cm internal PWM fans
• Running TrueNAS Core 12.0 U2
• Noctua NH-L9i as CPU cooler

Some of the TrueNAS Core systems I build for home use/homelab through the years:
8-disks (2 pools, each with 1 vdev in RaidZ)

8-disks (1 pool, 1 vdev, RaidZ2)

4-disks (1 pool, 1 vdev, RaidZ)

 

[TanKianW]

**Some advantages for setting up your own DIY NAS:**

• Recycle and repurpose your old hardware
• ZFS is a business class storage system with bullet proof reliability
• Can use it to host your personal cloud system such as Owncloud or Nextcloud
• RAIDZ is better than conventional RAID. You do not need any RAID hardware card to run, just let FreeNAS OS manage your zpools. During any hardware failure, just replace the parts like a normal computer.
• OS is FOC
• Scalable performance based on the hardware put together. (Eg. 10G NIC, SSDs, L2ARC, etc)
• Has all the function compared to OTS NAS, and more. (Only if you willing to learn)
• You get to choose the hardware specs and tune the NAS to your liking
• Mature NAS OS system with regular updates and patches
• Nice interface and dashboard
• Much easier to use than you think (Can still be subjective)
• The excitement and accomplishment when building your own NAS!

*Initial Setup of TrueNAS Core (FreeNAS)*
Download TrueNAS CORE: https://truenas.com/download

How to install TrueNAS Core


Set up Windows SMB Shares

*Frequently Used Command on TrueNAS Core system for trouble-shooting and access to attributes*
You can easily type in the commands under the "Shell" section of TrueNAS Core system. To keep it simple for new users, I listed down the commonly used command as below:

ZFS status:

zpool status -v #shows zpool status infos

zpool iostat 1 #shows IOps and R/W bandwidth every second

zfs list -t snapshot #lists all the snapshots, add | grep "" to filter

arc_summary.py #shows ARC stats

arcstat.py #shows ARC stats

zdb -C your_pool_name | grep ashift #shows the ashift value

SMART functions:

smartctl -h #shows smart control help, lots of commands examples

smartctl -a /dev/daX #shows all smart attributes

smartctl -A /dev/daX #same as -a but without vendor infos, last test infos, ...

smartctl -l selftest /dev/daX #shows result of last smart test

smartctl -l scttemp /dev/daX #shows the temperatures history

smartctl -t long /dev/daX #executes a long test (replace "long" by "short" for a short test)

HDD status:

glabel status  #shows gptids and devices names

gstat -f daX -I 1000ms #shows IOps and R/W bandwidth every second

iostat daX #shows IOps and R/W bandwidth

HBA:

dmesg | grep mps #used to check FW and driver versions on HBA cards using the mps driver

UPS status:

upsc ups #shows all ups attributes (change "ups" with your ups name)

Misc:

kldload ipmi.ko #loads ipmitool

ipmitool sensor #shows all ipmi sensors values

freenas-debug -h more /var/tmp/fndebug #shows all debug infos

camcontrol devlist #shows all devices

usbconfig #shows all usb devices

systat -vm #shows system infos (cpu load, ram usage, drives I/O, ...)

dmidecode l more #shows all hardware bios infos

*Creating Snapshots, Rysnc and Replication on TrueNAS System*
What are Snapshots and how to create them:


What is TrueNAS Replication:

What is Rsync:


*NOTE: If you are backing up snapshots, you should use replications. If you just need to transfer data from one NAS to another as backup, you should use Rsync. Do take note that Rsync is slower and should be use as a schedule backup between 2 TrueNAS system.

Personal Use Case (For reference ONLY):
1) Backup data on a computer/mobile over nextcloud/SyncThing.
2) Create dataset to store the data sync from nextcloud/SyncThing server but changing the mount point
3) Create snapshots on the data set
4) Sync the dataset with another TrueNAS using Rsync
5) Sync the selected (Frequency) snapshots onto another TrueNAS system
6) Backup the sync data on another TrueNAS system onto the cloud, Eg. Amazon S3.

*Create Cloud Sync Tasks to Cloud Providers (Eg. Amazon S3, BackBlaze B2, Microsoft OneDrive)*
This is very straight forward (Take note that you will need to set up your S3 with a registered AWS account prior setting up on your TrueNAS):

1) Create the cloud credential under System:

2) Create cloud sync task by keying the credential and select whether you want to PUSH or PULL the data to the cloud, and COPY, SYNC or MOVE between the local and hosted cloud.

*Creating Cloud Sync Tasks to Backup to/from Backblaze B2 on TrueNAS Core
For TrueNAS Core users that want to backup from their OneDrive or to their OneDrive, can follow the simple and straight forward steps below:


Instructions, Click Here:
https://forums.hardwarezone.com.sg/...e-for-new-users.6480129/page-5#post-133758423


*Creating Cloud Sync Tasks to Backup to/from OneDrive on TrueNAS Core
For TrueNAS Core users that want to backup from their OneDrive or to their OneDrive, can follow the simple and straight forward steps below:

Instructions, Click Here:
https://forums.hardwarezone.com.sg/...e-for-new-users.6480129/page-6#post-135529154

 

[TanKianW]

*Deploying Nextcloud Server on TrueNAS Core (Version 21.0.1): Updated as of 27 May 2021)*

Setting up Nextcloud on TrueNAS Core:

• Do take note that you will need to add the IP address of your NAS in the trusted domain using the shellcmd on the "jail" during the initial log in. You can add your trusted domain name here if you are setting it up for external access later. Do save after (ee) edit.
• Easy access to the shell of your "jail" install, go to jails -> shell:

Basic freeBSD cmd:

• cd => change up directory
• ls => list
• cd .. => change back directory
• ee => edit

Adding trusted domain:

cd /usr/local/www/nextcloud/config/

ee config.php

'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => '192.168.1.XX',
    2 => '192.168.1.XX',
    3 => 'nextcloud2021.homecloud.com'
  ),

Setting up Nextcloud on TrueNAS Core working in tandem with pfsense:


Using HAproxy to access your Nextcloud server externally, check out the thread here on how to set up HAproxy and Let's Encrypt Certs on your pfsense appliance:
https://forums.hardwarezone.com.sg/130207504-post3.html

Direct to the youtube videos on how to "https" your nextcloud server using Let's Encrypt Certs and HAproxy over pfsense:


IMPORTANT
*Do take note that there will be risks to access your cloud server externally. You must know what you are doing, before trying this!
*The convenience of accessing your cloud server externally is that you do not need to always connect to a VPN server and could sync data across your mobile or portable computer devices with the cloud when there is an internet connection. Which makes it work like a personal Google Drive, OneDrive, Dropbox, etc.

*Do set up the 2FA if you want to access the cloud server externally for extra security. There is a plugin package in Nextcloud, just install it to use the TOTP mobile app to generate the pass key.

*A safer way is to use a VPN such as OpenVPN. You can check out the thread here:
https://forums.hardwarezone.com.sg/130207493-post2.html


*Manually Download & Update Nextcloud using shell command instead of WebGUI updater
Due to the broad install base of nextcloud, the way it should be configured could be quite different between the Operating Systems (Debian, Ubuntu, FreeBSD, etc). There is therefore, not a one size fits all solution when installing your nextcloud on the jails of TrueNAS core. Sometimes, nothing beats running an update through the command, which I will be covering below. By running the update manually using command, you could also update to those "in-between" update that is not released by TrueNAS and at the same time avoid the "latest" buggy update.

Details here:
https://forums.hardwarezone.com.sg/...e-for-new-users.6480129/page-6#post-135536974


*Debugging errors/warnings/alerts on your Nextcloud installation*

Debugging solutions to common error/alert/warning messages on Nextcloud hosted on the Jail of TrueNAS Core. Not all will be listed here, I only list the common ones. Your situation and install environment might be different from mine. Feel free to find the solution on TrueNAS core forums. Most of the time, errors could be easily debugged with a few CLI on the shell:

Fix Alert: to resolve http => https issues (add in the below into the config.php)

cd /usr/local/www/nextcloud/config/

ee config.php

'overwriteprotocol' => 'https',
'overwrite.cli.url' => 'https://nextcloud2021.homecloud.com',

Fix Alert: Resolve php user issues: default user on truenas is "www", group is "www"

chown -R "www:www" /usr/local/www/nextcloud/apps-pkg

Fix Alert: Adding primary keys due to missing keys errors:

occ db:add-missing-primary-keys

Fix Alert: php does not seem to be setup properly to query system environment variables.

cd /usr/local/etc/php-fpm.d

ee nextcloud.conf

add in bottom:
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

Fix Alert: where your installation has no default phone region set. This is required to validate phone numbers in the profile settings without a country code.

cd /usr/local/www/nextcloud/config

ee config.php

add in bottom:
'default_phone_region' => 'SG',

Fix Alert: where memory cache was not set

cd /usr/local/www/nextcloud/config

ee config.php

add in bottom:
'memcache.local' => '\OC\Memcache\APCu',

Manually Updating FreeBSD jails command:

iocage fetch

after fetch, key the command:

iocage upgrade -r 12.X-RELEASE Nextcloud

*Nextcloud trouble-shoot (Using Cron on background jobs):
TrueNAS users experiencing problem using "Cron" under the setting of nextcloud's background jobs setting: Under Basic settings => Background jobs. The problem will look like this where the background jobs failed to execute :

This is a common problem where quite a number of users on the nextcloud help forum could not append the necessary cmd to start the background jobs under the crontab. The documentation could be found here:
https://docs.nextcloud.com/server/l...ion_server/background_jobs_configuration.html

One of the key reasons is that they are unable to key in the command. This is due to the default use of "vi editor" instead of editor like "nano" when you try to append the crontab cmd. "vi editor" is actually commonly use in linux/unix.

Follow the steps here to use the default vi editor to append the crontab cmd to setup "Cron" for back ground jobs on nextcloud:

Open up the shell from the root of the nextcloud jail, type cmd:

crontab -u www -e         (*instead of www-data)

Next you will be inside the vi editor cmd environment with "~" on the left hand side.
<=

~
~
~
~
~
~
~
~
~
~

type "i" or "a" to enter into cmd mode to edit/append where your cursor to key will appear at the top.

Followed by append the following:

*/5 * * * * /usr/local/bin/php -f /usr/local/www.nextcloud/cron.php    (*slightly diff from the documentation due to run env)

After keying, hit "Esc" to exit from the cmd mode, where the cursor will appear at the bottom.

Type ":wq" and hit "Enter" to exit and save.

ALT: You can exit and save by typing "Shift + zz"

After exit from vi editor, type cmd

crontab -u www -l                (*to verify that the crontab command has been appended.)

You should see output:

*/5 * * * * /usr/local/bin/php -f /usr/local/www.nextcloud/cron.php

type exit and you are DONE!

When you enter into your nextcloud basic setting, this will be the end result you see:

*Setting up ClamAV (Virus) Scanning server for Nextcloud on TrueNAS Core
In order to set up ClamAV scanning on your nextcloud server, you could install the ClamAV scanning server in one of the TrueNAS community plugins, then point the nextcloud server to the internal ClamAV scanning server.
TAKE NOTE: Running the ClamAV server will take up some resources.

Step 1: Install ClamAV plugin on TrueNAS core:

Step 2: Set up the ClamAV using cmd:

Open up the shell cmd on the root of ClamAV jail

Key in:

ee /usr/local/etc/clamd.conf

uncomment the "#" from the lines of:

TCPSocket 3310
TCPAddr 192.168.XX.XX                 (*IP Address of your ClamAV server)

Press Esc, choose "a" to leave and save

restart the ClamAV server using cmd:

/usr/local/etc/rc.d/clamav-clamd restart

After restart, type:

sockstat -l -4

You should be able to see the IP address and configured port of the ClamAV server as output

exit and we are DONE!

Step 3: Complete the set-up by pointing the nextcloud server to the ClamAV daemon host server", click "save" and it is done! Test it out by uploading a virus test file (Do not use an actual file with virus!):

*Preventing Lock Ups when using 2FA on Nextcloud*
1) Generate Backup code (Under Security) and save it in a safe location before setting up the 2FA using TOTP App.

2) Instead of keying in the 2FA, key in the Backup code after the initial log in page.

 

[TanKianW]

*TrueNAS Core ZFS Encryption - Setup & Encryption Keys Backup/Restore Process


Points to note:

• I recommend for home use, ZFS encryption is "Good to have", not a "Must have". You will not want to loose you data when your encryption keys are lost.
• For production, it should be a "Must Have", but do make sure data are constantly backed up (duplicated) and encryption keys backed up.
• Do mirror your drives that is containing the encryption keys and also save in a secure location
• Choosing encryption using paraphrase will requires you to "unlock" the pool every time you reboot. Use saved encryption keys method will unlock itself upon every reboot.
• If you are using the older version (FreeNAS before 12.0), you will be using the legacy encryption offered. You will not be able to use the new encryption features. The only way is to rebuild the new pool somewhere else using the latest version of TrueNAS 12.0 then copy/migrate the data to the new pool.
• There will be a slight hit in performance from pool encryption.

*Connecting APC UPS to TrueNAS Core system through USB & Configuring UPS settings*
Master and Slave UPS setting under "Services" section:
*In this way, when power is lost for a set period of time, the slave will initiate a proper shut down followed by the master. So you do not need 2 UPS for 2 different system. However, you need to set static IPs for both NAS system.
For Eg: Master NAS is 192.168.1.2, Slave is 192.168.1.3

On Master NAS: Auxiliary Parameters to key in: "LISTEN 192.168.1.2"

On Slave NAS: Remote Host is the Master NAS IP at 192.168.1.2, Port or Hostname is the Slave NAS IP at 192.168.1.3

Set up Gmail notification:
*During power lost or shut down, the NAS system will send you email notification or to report any errors/warnings/alerts/emergencies.

*Torrenting using qbittorent on TrueNAS*
*This will be very straight forward. Just need to install the "qbittorent" community plugins, set a static IP address and port (8080) for the web GUI to access the controls. It works like any torrenting applications.
*Take note that you will need to port forward the torrenting port. And I will recommend to turn on encryption.

The qbittorrent plugins:

The qbittorrent Web config:

*If you are using pfsense, below are instructions for port forwarding:


Changing the mount point of a plugin:
*(Recommended) You might want to set the download folder away from the JAIL (iocage) folder by mapping the mount point onto another created folder. Eg. qbittorrent_files. You can change the mount point by selecting the mount point on the plugins and edit the mount points.
*Take note that you can only change the mount point when the plugins are "STOP"

HomeNAS = name of your NAS
qbittorent_jails = name of your JAILS
qbittorent_files = Dataset created to house and organise your downloads map from the JAILS download folder.
*Take note that plugins in TrueNAS was installed in "JAILS" away from the main file folders.

Destination from:
/mnt/HomeNAS/iocage/jails/qbittorent_jails/root/Downloads

Source mount to:
/mnt/HomeNAS/qbittorrent_files

**Setting Up VLANs on TrueNAS Core interfaces**

Some of you might have several network interfaces (like me) on your TrueNAS system and thinking how could you make use of them as they are laying around disabled. You can either set it up for LAN aggregation (LACP) for redundancy/bandwidth, or you can set up VLANs for the different jails/plugins/containers it ties to. From a security point of view, your external facing servers (Eg. Nextcloud & Home Assistant) should be on a dedicated VLAN itself that is seperated from your main home network. This is to prevent compromising your whole network during an attack on your external facing servers. Therefore, it is good practice to segregate them unto different VLANs. Which leads to the reason of covering it here and answering to some of the user queries.

To set up VLAN on TrueNAS, you are required to set up the VLAN (Eg. VLAN70) and the Bridge for the VLAN without an IP address assigned.

Video tutorial watch here:


*NOTE: Even though Tom is setting up on his custom installed "jails". It will actually work for "plugins" too.

On the Switch side: Do remember to set up VLANs (Eg. Nextcloud Server) at your switch side, (*correction*) tag the VLAN you assign to the interface and set to default PVID (Eg. On RouterOS).

 

[TanKianW]

**INFORMATIVE READ: Why ZFS??**

The ZFS Copy-On-Write (C.O.W) File System
For those who seek to understand more about the ZFS (Zetta File System), its strength in data integrity and consistency. A simple break down explanation (Data, metadata, journal) from Tom of Lawrence System.

​

Worthy Read. The article Tom took reference from:
https://arstechnica.com/information...01-understanding-zfs-storage-and-performance/

*Updated on Apr 2021*

*Motherboards for TrueNAS Core system (For reference ONLY)*
I guess the selection of motherboards is the most daunting task for some, when setting up a DIY TrueNAS system. Therefore decided to list down some of the (more available) motherboards which I have use for setting up TrueNAS system:

ECC Memory Alert: Do take note that there is a difference between "unbuffered" and "registered" ECC memory. Make sure your motherboard support the type of ECC memory you are purchasing for your TrueNAS system.

The guys from servethehome website explains it clearly here:
https://www.servethehome.com/unbuffered-registered-ecc-memory-difference-ecc-udimms-rdimms/#:~:text=Unbuffered ECC versus Registered ECC Memory&text=The basic difference is that,being sent to the modules.

Embedded system:
1) Preferred Choice: Supermicro A2SDi-8C-HLN4F (excellent ECC memory compatibility and stability)
https://www.supermicro.com/en/products/motherboard/A2SDi-8C-HLN4F
2) Less expensive choice: Asrock Rack C3758D4I-4L (picky with ECC memory but cheaper)
https://www.asrockrack.com/general/productdetail.asp?Model=C3758D4I-4L#Specifications

Non-embedded System:
1) AMD CPU with 10G: Asrock Rack X570D4I-2T (Hot Intel X550 NIC chips which requires adequate cooling, AMD Zen CPU compatible)
https://www.asrockrack.com/general/productdetail.asp?Model=X570D4I-2T#Specifications
2) Intel Xeon CPU with 10G: Asrock Rack E3C246D4I-2T (Hot Intel X550 NIC chips which requires adequate cooling case)
https://www.asrockrack.com/general/productdetail.asp?Model=E3C246D4I-2T#Specifications
3) Intel Xeon CPU with 8x SATA: Asrock Rack C236 WSI (Not so particular with speed and conscious with cost)
https://www.asrockrack.com/general/productdetail.asp?Model=C236 WSI#Specifications

*Will add on to this list*

*COMMONLY ASKED QUESTIONS*
Some of the most commonly asked questions I received on the use of TrueNAS Core is on (1) how do you extend the ZFS pools and (2) how to replace the disks when it fails. Below are the practical tutorials answering to the questions. Do watch.





*(OPTIONAL) Tuning your TrueNAS Core System
For those who are interested in improving the performance of their TrueNAS Core system, feel free to try out these settings under System => Tunables. These has been shared by Patrick Kennedy from servethehome in the past. Feel free to check it out.

For copying and pasting:

• kern.ipc.maxsockbuf = 16777216
• net.inet.ip.intr_queue_maxlen = 2048
• net.inet.tcp.recvbuf_inc = 524288
• net.inet.tcp.recvbuf_max = 16777216
• net.inet.tcp.recvspace = 4194304
• net.inet.tcp.sendbuf_inc = 32768
• net.inet.tcp.sendbuf_max = 16777216
• net.inet.tcp.sendspace = 2097152
• net.route.netisr_maxqlen = 2048

**TrueNAS: Backup Immutability and Hardening**

Video Tutorial: Good backup practices on TrueNAS

There are several ways of backing up your data automatically on TrueNAS, which helps you to remove the “chores” of backing up your precious data. You can even set TrueNAS to inform you of unsuccessful backup task due to any mis-configuration or remote access settings screw ups. You can choose between replications task, cloud sync task, rsync, periodic snapshots or a combination of them. Below are examples of how you could backup your cloud sync (eg. Nextcloud on mobile) dataset and protect it against malware using TrueNAS Core/Scale.​

Good practices: Following the 3-2-1 rule of backing up data:

Backup Methods:
Periodic Snapshots (tasks) on the dataset to protect against Malware:

Snapshots Replication (tasks) to another (local) NAS system:

Auto copy data using rsync (tasks) to another (local) NAS:

Offsite backup (tasks) to Backblaze B2:

 

[TanKianW]

*Hosting Home Assistant Locally on TrueNAS core*
Just recently, there is much buzz on the use of self hosted "Home Assistant" due to the garage home automation video on YouTube from LTT. Since the use of Docker, VMs might not be for the know how of average Joes, I decided to share on some small home automation projects which I have been using on TrueNAS core and Home Assistant. Yes, instead of using VM or Dockers, you can install "Home Assistant" using the community plugin of TrueNAS core.

*Take note that I am not linking Google Voice Assistant with Home Assistant. That will require additional configuration.

Working setup on TrueNAS core using "Home Assistant" community plugins:

Some of the more advance integration might requires you to edit the yaml configuration file which you could find in the community documentation and forums. If not, most of the listed integration will work just fine.

Support on iOS. Recommend to setup 2FA using TOTP app if you want to access the server externally:

*NOTE: If you wish to access your Home Assistant server setup externally, instead of the docker/cloudflare DDNS route shown on LTT forum/youtube video, you could just simply use your pfsense appliance to act as a proxy server to "https" the server.

*You can read Post #3 here to learn more on the setup using Let's Encrypt certs and HAproxy:
https://forums.hardwarezone.com.sg/threads/starting-pfsense-for-new-users.6390714/


*Hosting Unifi Controller on TrueNAS Core*
For TrueNAS Core users with Unifi APs and appliances, you could also choose to locally host the Unifi Controller (personally prefer LTS version), instead of using a dedicated pc, cloud host or cloud key.

The installation is very straight forward. Just look under Plugins -> click "browse a collection" -> select "Community" plugins, followed by download and install the "Unifi Controller" or "Unifi Controller LTS" plugins.
An IP using port 8445 will automatically be created to access the Unifi Controller WebGUI:

The Unifi Controller WebGUI hosted on TrueNAS Core:

*Hosting SyncThing Server on TrueNAS Core*




NOTE:

• If you only need simple file sync between devices, SyncThing is a snappy and great solution.
• SyncThing could be a substitute to Nextcloud for simple file sync
• Comparing plugin updates on TrueNAS, it is better supported than Nextcloud
• There is not yet official support for iOS platform
• Personally I use SyncThing to sync files from my worklab to my homelab. It is pretty much hassle free.

*Hosting PLEX Server on TrueNAS Core

Since I constantly received requests on how to set up the Plex server on TrueNAS Core and PLEX being a popular media server set up to manage the libraries on the NAS/storage server, I will therefore provide a quick step by step guide to start some new users going. You can also use DLNA through your self hosted PLEX server on TrueNAS. Currently, I have no issue streaming media content with Dobly Vision and ATMOS over my Nvidia Shield Pro.

Find out more here:​

https://forums.hardwarezone.com.sg/...e-for-new-users.6480129/page-6#post-135833974

AND here:
https://forums.hardwarezone.com.sg/...e-for-new-users.6480129/page-6#post-135941387

 

[croc1977]

thank you...will be creating my own nas soon...now waiting for my mb to arrive next week will set up liao..any issue will promt in this thread. thanks

 

[croc1977]

question

currently i am using asustor nas so they recommded me use ironwolf so it has another layer of scanning when incoperate with ironwolf drives.

does truenas have a prefer brand of nas drive to be use so that it can take advantage of the hard disk extra feature?

or any brand nas class is fine ? even toshiba nas drive which is cheaper?

 

[TanKianW]

does truenas have a prefer brand of nas drive to be use so that it can take advantage of the hard disk extra feature?

or any brand nas class is fine ? even toshiba nas drive which is cheaper?

1) Not of I know of on specific NAS drive for extra disk features. So long your HDD is a CMR drive will do. Avoid SMR.

2) I will prefer HDD build for NASes. I have never use a Toshiba Drive for my NAS build before, therefore cannot comment on whether it is fine or reliable. Do check if they are CMR. For my personal home build, I have only used Hitachi HGST enterprise drives, WD Red Plus, WD Red Pro, Ironwolf, Ironwolf Pro and EXOs. Hitachi HGST being the longest lasting for my home use case, >8-9years and still running 24/7. Not sure if their acquisition by WD will affect their reliability.

For new builds, I will recommend going Seagate Ironwolf Pro.

 

[TanKianW]

Update on Post #1

Recommended setup of TrueNAS core system:
1) *Host the TrueNAS server to be access externally through (https) using HAProxy and Let's Encrypt certificate function of pfsense. OR a VPN server on pfsense such as OpenVPN.
Read the thread here to learn more: https://forums.hardwarezone.com.sg/130207504-post3.html
2) LACP lagg the NAS to your managed switch which also acts as a failover
3) Connect your TrueNAS system to a UPS. I will be sharing how to configure the APC UPS over USB which you could easily purchase locally. Set the main NAS to master and the other to Slave, so that both will shut down on their own when there is a power lost
4) Schedule HDD scrub every week
5) Schedule Short and Long SMART test every week
6) Set Resilver priority
7) Set up SSH between 2 TrueNAS system to backup each other using Rsync or Replication tasks
8) Set the BIOS on the TrueNAS system to auto power up when power resume. This is important for your system to resume back online after an earlier shut down due to reboot or power lost.
9) Set up email notification to allow your TrueNAS system to send you alerts when it runs in power lost, reboot or critical errors.
10) Always set up 2x boot drive (USB or SSD) so you can boot up from any of the drives when one fails.
11) Schedule tasks for regular snapshots on your ZFS pool
12) Set up a VPN server at home over pfsense. You can choose to use OpenVPN. Feel free to check out here: https://forums.hardwarezone.com.sg/130207493-post2.html

Some of the practical use of TrueNAS core system:
1) Host your personal IaaS cloud such as nextcloud to back up/sync data on your work PCs, mobile devices, etc.
2) Torrenting on the TrueNAS system using qbittorent and manage it on a web interface which you could access from anywhere using VPN on pfsense
3) Sync data across >2 NAS system using Rsync
4) Backup the snapshots between two system using Replication tasks
5) For easy syncing between your NAS, mobile devices and PCs using SyncThing
6) Host your Plex server on TrueNAS system
7) Host the Unifi Controller on TrueNAS system
8) Run VMs on your TrueNAS system such as XCPNG
9) Network drive SMB share in your home local network
10) Time Machine like SMB shadow copy, restore data using TrueNAS snapshots
11) Further backed your system up by syncing it to cloud
12) *For enterprise use: Manage several TrueNAS system using TrueNAS scale.

 

[TanKianW]

*Updated Post #3*

*Deploying Nextcloud Server on TrueNAS Core

*Also updated the commonly used command list on TrueNAS Core on Post #2

 

[croc1977]

Update on Post #1

Recommended setup of TrueNAS core system:
1) *Host the TrueNAS server to be access externally through (https) using HAProxy and Let's Encrypt certificate function of pfsense. OR a VPN server on pfsense such as Wireguard and OpenVPN.
Read the thread here to learn more: https://forums.hardwarezone.com.sg/130207504-post3.html
2) LACP lagg the NAS to your managed switch which also acts as a failover
3) Connect your TrueNAS system to a UPS. I will be sharing how to configure the APC UPS over USB which you could easily purchase locally. Set the main NAS to master and the other to Slave, so that both will shut down on their own when there is a power lost
4) Schedule HDD scrub every week
5) Schedule Short and Long SMART test every week
6) Set Resilver priority
7) Set up SSH between 2 TrueNAS system to backup each other using Rsync or Replication tasks
8) Set the BIOS on the TrueNAS system to auto power up when power resume. This is important for your system to resume back online after an earlier shut down due to reboot or power lost.
9) Set up email notification to allow your TrueNAS system to send you alerts when it runs in power lost, reboot or critical errors.
10) Always set up 2x boot drive (USB or SSD) so you can boot up from any of the drives when one fails.
11) Schedule tasks for regular snapshots on your ZFS pool
12) Set up a VPN server at home over pfsense. You can choose to use OpenVPN or Wireguard. Feel free to check out here: https://forums.hardwarezone.com.sg/130207493-post2.html

Some of the practical use of TrueNAS core system:
1) Host your personal IaaS cloud such as nextcloud to back up/sync data on your work PCs, mobile devices, etc.
2) Torrenting on the TrueNAS system using qbittorent and manage it on a web interface which you could access from anywhere using VPN on pfsense
3) Sync data across >2 NAS system using Rsync
4) Backup the snapshots between two system using Replication tasks
5) For easy syncing between your NAS, mobile devices and PCs using SyncThing
6) Host your Plex server on TrueNAS system
7) Host the Unifi Controller on TrueNAS system
8) Run VMs on your TrueNAS system such as XCPNG
9) Network drive SMB share in your home local network
10) Time Machine like SMB shadow copy, restore data using TrueNAS snapshots
11) Further backed your system up by syncing it to cloud
12) *For enterprise use: Manage several TrueNAS system using TrueNAS scale.

10) Always set up 2x boot drive (USB or SSD) so you can boot up from any of the drives when one fails.

how to do that?

 

[TanKianW]

how to do that?

Read up here:
https://www.truenas.com/docs/hub/tasks/advanced/mirror-boot-pool/

Looks like this:

 

[croc1977]

Read up here:
https://www.truenas.com/docs/hub/tasks/advanced/mirror-boot-pool/

Looks like this:

so when changes made on one...the one will be mirror rite?

 

[TanKianW]

so when changes made on one...the one will be mirror rite?

Yap. Preferably both with same capacity.

Possible Scenario to put it to use:
One of my current system is running on SSD as main boot, USB drive as back-up boot. I experienced an instance where my BIOS failed to boot from the SSD due to a cable fault (OR could be a corruption to boot disk) after an update reboot. During that situation, I can choose to boot into the USB drive to ensure that all my data is fine before start trouble-shooting from there.

 

[TanKianW]

Update on Post #4

*Connecting APC UPS to TrueNAS Core system through USB & Configuring UPS settings*

Master and Slave UPS setting under "Services" section:
*In this way, when power is lost for a set period of time, the slave will initiate a proper shut down followed by the master. So you do not need 2 UPS for 2 different system. However, you need to set static IPs for both NAS system.
For Eg: Master NAS is 192.168.1.2, Slave is 192.168.1.3

On Master NAS: Auxiliary Parameters to key in: "LISTEN 192.168.1.2"

On Slave NAS: Remote Host is the Master NAS IP at 192.168.1.2, Port or Hostname is the Slave NAS IP at 192.168.1.3

Set up Gmail notification:
*During power lost or shut down, the NAS system will send you email notification or to report any errors/warnings/alerts/emergencies.

 

[evildoctor]

Thanks TanKianW for the detailed instructions on how to setup TrueNas for hardwarezone community. Much appreciated.

Regarding the selection of Hard Disks, you are using RAID-Z1 right? Cause i only know the basic of Raid 0,1,2,5 etc. Tried googling about Raid-Z, didn't quite understand it. Correct me if i'm wrong, Raid-Z1 means 1 disk can be faulty while Raid-Z2 means 2 disk can be faulty right? And whats the minimum disk requirement for Raid-Z1?

Also, the speed of the hard disks 7200rpm vs 5900rpm, would it make much difference to the speed of TrueNas?

 

[TanKianW]

Thanks TanKianW for the detailed instructions on how to setup TrueNas for hardwarezone community. Much appreciated.

Regarding the selection of Hard Disks, you are using RAID-Z1 right? Cause i only know the basic of Raid 0,1,2,5 etc. Tried googling about Raid-Z, didn't quite understand it. Correct me if i'm wrong, Raid-Z1 means 1 disk can be faulty while Raid-Z2 means 2 disk can be faulty right? And whats the minimum disk requirement for Raid-Z1?

Also, the speed of the hard disks 7200rpm vs 5900rpm, would it make much difference to the speed of TrueNas?

Yes. Want to know more can watch this (Added this on Post #1):

*Do read up and plan before deciding your RAIDZ, vdev, pool setup

What is ZFS? And what sets it apart from other file system? What is "impeccable" data integrity!

Servethehome explains it in simply too:
https://www.servethehome.com/an-introduction-to-zfs-a-place-to-start/

I will think their speed on TrueNAS is not significantly different, at least for home usage. To speed things up, I will recommend to max up the memory and upgrade that 1G NIC. Preferably 32GB or more, and a 10G NIC.

There are some who debates/recommends a L2ARC drive on TrueNAS to speed things up, but as most ZFS veteran put it simply...you are better off maxing up the memory of your system.

 

[croc1977]

just made my own home nas ....

i have created raid z so afew question . but have not done cable managment yet . maybe tonight countine. explore more

1) How to create individual folder under 192.168.x.x/xxxxx/
2) how to enable dlna on the folder?

 

[TanKianW]

just made my own home nas ....

i have created raid z so afew question . but have not done cable managment yet . maybe tonight countine. explore more

1) How to create individual folder under 192.168.x.x/xxxxx/
2) how to enable dlna on the folder?

1) You need to select the folder (mount point) when creating the SMB share and when you map the network drive (on Windows), it automatically points to the folder in TrueNAS. Do check out the video on creating SMB share.
2) I don’t use DLNA. From what I know you can enable DLNA on the “Emby” plugin if it is still available on the “community” plugin.