storage consolidation or separation?

jk_eugenetay

Member
Joined
Jan 22, 2006
Messages
246
Reaction score
0
Hello to all

I would like to widen my knowledge and understand from the ground (opp reading reviews) what is the pro & cons, trade-offs, benefits, etc. of

Having a standalone NAS e.g. QNAP/Synology vs. attaching the HDD to PC as storage..

etc. in terms of remote access, features, functions, etc.
Any view is welcomed!
 

zennith

Member
Joined
Nov 22, 2012
Messages
105
Reaction score
0
Hello to all

I would like to widen my knowledge and understand from the ground (opp reading reviews) what is the pro & cons, trade-offs, benefits, etc. of

Having a standalone NAS e.g. QNAP/Synology vs. attaching the HDD to PC as storage..

etc. in terms of remote access, features, functions, etc.
Any view is welcomed!

Different type of storage serve their own purpose..
For my case, NAS to consolidate all..
Why so...
NAS with RAID and having larger volume than a NTFS or FAT32 drives.
no file limitation in the sense of file size and capability of sharing between multiple platform...

:D to go in details, it take some times.. well, pro and cons depends on user usage.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,550
Reaction score
1,302
Hello to all

I would like to widen my knowledge and understand from the ground (opp reading reviews) what is the pro & cons, trade-offs, benefits, etc. of

Having a standalone NAS e.g. QNAP/Synology vs. attaching the HDD to PC as storage..

etc. in terms of remote access, features, functions, etc.
Any view is welcomed!

From a pure storage standpoint, consolidated storage such as NAS or SAN are to encourage aggregated usage across various parties and centralize management. This are Enterprising notion of how resources are managed. For consumer, the benefits while still exists are not the priority of most consumers.

When one consolidate numerous storage requirement into a single system, obviously it ressembles "Putting All Eggs in One Basket". In Enterprises, this Basket is normally a very robust basket. This basket comes in Active/Passive setup, also known as Master-Slave configuration. Between these 2 nodes (Master and Slave), realtime or sometimes near-realtime replication happens. Which means one normally have highly available configuration to cope with the master node going down. In each node, storage disks redundancies also exists, as machines are normally at much lower failure rate than disks. Disks are stored in something known as Array Groups. Each Array Groups may comprise of various RAID configurations. It can be RAID 1, RAID 5, with or without spare. You can even find consolidated storage solution able to cope with disaster of the whole data centre when they replicated their data offsite into another geographically different location.

Now back to consumer storage consolidation. It would not have that kind of robustness as the Enterprise notion, yet it does cover some to a certain extent.

1) Redundancies - Your NAS would allow for multiple disks per RAID configuration. This would cover redundancies to prevent single or multiple disks failure from compromising the data integrity. RAID 1 and RAID 5 would only cover 1 failure disk, RAID 6 covers 2, and RAID 10(1+0) covers 1 or 2 depending on situation.

2) Backup - Proconsumer NAS I have come across normally have backup capability to backup into another NAS, another system, externally mounted disks, or even into cloud providers with storage capability such as AWS. This feature will cover backup plus versioning control if your backup feature supports that. Another advantage is this centralize system can function as the backup target for all your other systems. You can backup the whole system to the NAS and in the event any of your systems failed, you can restore the system data back(dependent on backup software implementation)

3) Monitoring - Proconsumer NAS supports data scrubbing and S.M.A.R.T monitoring on the disks to ensure they are in normal condition. These tests can be perform periodically to detect possible hardware failure occurring. So you should change the disks when they are going to fail.

4) Low wattage - Due to the fact that these proconsumer NAS are mostly for storage purpose instead of a general purpose system, they do not have requirement for high power usage and also they can be online 24x7 all year long. Coupled with a dedicated UPS, you can be more sure that it will not suffer from sudden power outrage or electrical spike. Pairing with UPS will also allow for graceful shutdown for prolong power outrage.

5) Sharing - Having a consolidated storage will allow for purpose hosts sharing on a single piece of data. This is good to ensure the version of this data is consistent across all participating hosts.

I guess the cons is as follows

1) Pricey - Compared to most simplest backup using attached devices to your system, this solution is more expensive. One thing is most proconsumer NAS comes with applications in it for various functionalities like bittorrent, private cloud storage, authentication system etc. Even if you are not using it, you are paying for it. But I think people that failed to see the beauty of having an automated system is they thought they are machines themselves that are so regimental, and also they failed to see the importance of their data. They also failed to see the volatile nature of machines and materials.

2) Learning curve - More knowledge required to handle such system. It is more complicated as it entails configuration of the system to get to the point of stable usage. There is learning curve and would normally required certain level of technological savviness.

3) Network requirement - Definitely require a planned network layout within the family for best usability across all devices may it be physical or wireless network. May require good Internet connectivity for backup to cloud or service applications

4) Speed - Network devices are normally of slower speed compared to DAS of same technology. The network latency of network cable is higher and involved more network nodes like routers, switches and quality and length of the cable. DAS have much higher speed connectivity like USB3.0, SATA3, Thunderbolt 2 etc. Unless you can afford Fibre Connectivity or Infiniband and other such network solution, you wouldn't be getting anything that can easily surpass DAS connectivity solutions at the same price level.
 

zennith

Member
Joined
Nov 22, 2012
Messages
105
Reaction score
0
From a pure storage standpoint, consolidated storage such as NAS or SAN are to encourage aggregated usage across various parties and centralize management. This are Enterprising notion of how resources are managed. For consumer, the benefits while still exists are not the priority of most consumers.

When one consolidate numerous storage requirement into a single system, obviously it ressembles "Putting All Eggs in One Basket". In Enterprises, this Basket is normally a very robust basket. This basket comes in Active/Passive setup, also known as Master-Slave configuration. Between these 2 nodes (Master and Slave), realtime or sometimes near-realtime replication happens. Which means one normally have highly available configuration to cope with the master node going down. In each node, storage disks redundancies also exists, as machines are normally at much lower failure rate than disks. Disks are stored in something known as Array Groups. Each Array Groups may comprise of various RAID configurations. It can be RAID 1, RAID 5, with or without spare. You can even find consolidated storage solution able to cope with disaster of the whole data centre when they replicated their data offsite into another geographically different location.

Now back to consumer storage consolidation. It would not have that kind of robustness as the Enterprise notion, yet it does cover some to a certain extent.

1) Redundancies - Your NAS would allow for multiple disks per RAID configuration. This would cover redundancies to prevent single or multiple disks failure from compromising the data integrity. RAID 1 and RAID 5 would only cover 1 failure disk, RAID 6 covers 2, and RAID 10(1+0) covers 1 or 2 depending on situation.

2) Backup - Proconsumer NAS I have come across normally have backup capability to backup into another NAS, another system, externally mounted disks, or even into cloud providers with storage capability such as AWS. This feature will cover backup plus versioning control if your backup feature supports that. Another advantage is this centralize system can function as the backup target for all your other systems. You can backup the whole system to the NAS and in the event any of your systems failed, you can restore the system data back(dependent on backup software implementation)

3) Monitoring - Proconsumer NAS supports data scrubbing and S.M.A.R.T monitoring on the disks to ensure they are in normal condition. These tests can be perform periodically to detect possible hardware failure occurring. So you should change the disks when they are going to fail.

4) Low wattage - Due to the fact that these proconsumer NAS are mostly for storage purpose instead of a general purpose system, they do not have requirement for high power usage and also they can be online 24x7 all year long. Coupled with a dedicated UPS, you can be more sure that it will not suffer from sudden power outrage or electrical spike. Pairing with UPS will also allow for graceful shutdown for prolong power outrage.

5) Sharing - Having a consolidated storage will allow for purpose hosts sharing on a single piece of data. This is good to ensure the version of this data is consistent across all participating hosts.

I guess the cons is as follows

1) Pricey - Compared to most simplest backup using attached devices to your system, this solution is more expensive. One thing is most proconsumer NAS comes with applications in it for various functionalities like bittorrent, private cloud storage, authentication system etc. Even if you are not using it, you are paying for it. But I think people that failed to see the beauty of having an automated system is they thought they are machines themselves that are so regimental, and also they failed to see the importance of their data. They also failed to see the volatile nature of machines and materials.

2) Learning curve - More knowledge required to handle such system. It is more complicated as it entails configuration of the system to get to the point of stable usage. There is learning curve and would normally required certain level of technological savviness.

3) Network requirement - Definitely require a planned network layout within the family for best usability across all devices may it be physical or wireless network. May require good Internet connectivity for backup to cloud or service applications

4) Speed - Network devices are normally of slower speed compared to DAS of same technology. The network latency of network cable is higher and involved more network nodes like routers, switches and quality and length of the cable. DAS have much higher speed connectivity like USB3.0, SATA3, Thunderbolt 2 etc. Unless you can afford Fibre Connectivity or Infiniband and other such network solution, you wouldn't be getting anything that can easily surpass DAS connectivity solutions at the same price level.

Totally Agree ~ :D
 

jk_eugenetay

Member
Joined
Jan 22, 2006
Messages
246
Reaction score
0
From a pure storage standpoint, consolidated storage such as NAS or SAN are to encourage aggregated usage across various parties and centralize management. This are Enterprising notion of how resources are managed. For consumer, the benefits while still exists are not the priority of most consumers.

When one consolidate numerous storage requirement into a single system, obviously it ressembles "Putting All Eggs in One Basket". In Enterprises, this Basket is normally a very robust basket. This basket comes in Active/Passive setup, also known as Master-Slave configuration. Between these 2 nodes (Master and Slave), realtime or sometimes near-realtime replication happens. Which means one normally have highly available configuration to cope with the master node going down. In each node, storage disks redundancies also exists, as machines are normally at much lower failure rate than disks. Disks are stored in something known as Array Groups. Each Array Groups may comprise of various RAID configurations. It can be RAID 1, RAID 5, with or without spare. You can even find consolidated storage solution able to cope with disaster of the whole data centre when they replicated their data offsite into another geographically different location.

Now back to consumer storage consolidation. It would not have that kind of robustness as the Enterprise notion, yet it does cover some to a certain extent.

1) Redundancies - Your NAS would allow for multiple disks per RAID configuration. This would cover redundancies to prevent single or multiple disks failure from compromising the data integrity. RAID 1 and RAID 5 would only cover 1 failure disk, RAID 6 covers 2, and RAID 10(1+0) covers 1 or 2 depending on situation.

2) Backup - Proconsumer NAS I have come across normally have backup capability to backup into another NAS, another system, externally mounted disks, or even into cloud providers with storage capability such as AWS. This feature will cover backup plus versioning control if your backup feature supports that. Another advantage is this centralize system can function as the backup target for all your other systems. You can backup the whole system to the NAS and in the event any of your systems failed, you can restore the system data back(dependent on backup software implementation)

3) Monitoring - Proconsumer NAS supports data scrubbing and S.M.A.R.T monitoring on the disks to ensure they are in normal condition. These tests can be perform periodically to detect possible hardware failure occurring. So you should change the disks when they are going to fail.

4) Low wattage - Due to the fact that these proconsumer NAS are mostly for storage purpose instead of a general purpose system, they do not have requirement for high power usage and also they can be online 24x7 all year long. Coupled with a dedicated UPS, you can be more sure that it will not suffer from sudden power outrage or electrical spike. Pairing with UPS will also allow for graceful shutdown for prolong power outrage.

5) Sharing - Having a consolidated storage will allow for purpose hosts sharing on a single piece of data. This is good to ensure the version of this data is consistent across all participating hosts.

I guess the cons is as follows

1) Pricey - Compared to most simplest backup using attached devices to your system, this solution is more expensive. One thing is most proconsumer NAS comes with applications in it for various functionalities like bittorrent, private cloud storage, authentication system etc. Even if you are not using it, you are paying for it. But I think people that failed to see the beauty of having an automated system is they thought they are machines themselves that are so regimental, and also they failed to see the importance of their data. They also failed to see the volatile nature of machines and materials.

2) Learning curve - More knowledge required to handle such system. It is more complicated as it entails configuration of the system to get to the point of stable usage. There is learning curve and would normally required certain level of technological savviness.

3) Network requirement - Definitely require a planned network layout within the family for best usability across all devices may it be physical or wireless network. May require good Internet connectivity for backup to cloud or service applications

4) Speed - Network devices are normally of slower speed compared to DAS of same technology. The network latency of network cable is higher and involved more network nodes like routers, switches and quality and length of the cable. DAS have much higher speed connectivity like USB3.0, SATA3, Thunderbolt 2 etc. Unless you can afford Fibre Connectivity or Infiniband and other such network solution, you wouldn't be getting anything that can easily surpass DAS connectivity solutions at the same price level.

hi david,

Thanks for your feedback! it was really a good insight shared about enterprise storage considerations. I do understand where you are coming from, but here are some of my thoughts (about my use for the storage):

1. For hosting my VMs
Should I buy a NAS with iSCSI functionality and store on it, or attaching the disks as a DAS? There is only 1 user which is myself for the VMs.

2. Redundancies: Is the Storage Spaces in Windows 8 Pro a good enough alternative to the NAS's RAID capabilities? I read about it, i presume its a software level RAID; do consumer NAS boxes e.g. QNAP use hardware RAID? I understand only rack based servers have a dedicated Reading up more on the Storage Spaces, it does have comparable mirroring capabilities too.
Want to hear from the floor about this comparison.

3. Centralize management & aggregated usage: locating all disks on my PC is it better? since I'm the only one who uses the storage, for VMs, photos, etc. Only the network media player access the HP Mediasmart EX490 server now. its quite old. :(

4. Remote access: I guess there are tools/methods that can be used that is similar to what those NAS boxes offer? Any recommendations?

Good weekend! Hope to hear from you guys soon!
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,550
Reaction score
1,302
1. For hosting my VMs
Should I buy a NAS with iSCSI functionality and store on it, or attaching the disks as a DAS? There is only 1 user which is myself for the VMs.

If you only foresee yourself as the sole user of these virtual machines, I highly recommend you use internal storage to the vm host. No matter how good your iSCSI is, it will be a lot more expensive to get something that have as low latency as internal interfaces.

Your VMs will incur substantial random I/O on your disks, you will need pretty powerful NAS and exceptionally good network interfaces to match up the performance of internal storage solutions.

However having external NAS solution on top of your DAS is a good solution for backup and archiving. For VM purpose, I will choose internal solution over iSCSI most of the time unless you are building HA virtual centre where shared storage is necessary such as VMWare vCenter solution

2. Redundancies: Is the Storage Spaces in Windows 8 Pro a good enough alternative to the NAS's RAID capabilities? I read about it, i presume its a software level RAID; do consumer NAS boxes e.g. QNAP use hardware RAID? I understand only rack based servers have a dedicated Reading up more on the Storage Spaces, it does have comparable mirroring capabilities too.
Want to hear from the floor about this comparison.

Rack or Pedestal form factor is not deterministic factor to H/W or S/W RAID. Synology do have rack NAS solutions too. QNAP and SYNOLOGY are not exactly your top tier enterprising RAID solution like NETAPP or EMC. I have not come across H/W RAID from these 2 providers. They may have, but certainly not in the same price range.

I do not have acquaintance with Windows 8 Storage Spaces, but it seems like an elaborated LVM solution well integrated with Windows disk management and also include RAID and Windows Server clustering solution. It is very much a S/W RAID solution. With Linux, you can also achieve very similar setup using LVM + DRBD + MD, but definitely I would go for Windows Space if you want something well integrated among these technologies since even as a Linux veteran like me, setting up all 3 storage technologies can be pretty involving to get things right and smooth going.

3. Centralize management & aggregated usage: locating all disks on my PC is it better? since I'm the only one who uses the storage, for VMs, photos, etc. Only the network media player access the HP Mediasmart EX490 server now. its quite old. :(

Better if you know what you are doing and how you are using it. I don't enjoy having my laptops or desktop on 24by7, yet I would love to have my NAS running 24by7. Also when I offload my data into the NAS, I can be extremely sure they are "SAFE" In my desktop, it is not necessarily because first of all, my desktop have H/W RAID and hence it is more dependent on the availability of the H/W RAID and also the availability of my desktop.

At the same time, my NAS also does backup and scrubbing automatically which is an approach to have early warning. I doubt you will be having your desktop or laptop online over 24hours just to do the scrubbing ?

I think somehow the way I look upon these proconsumer NAS solution is, they are not sufficient to give the kind of power I would love to have for multiple VM hosts. MY demand for VM performance is pretty high and normally only suffice via FC and a lot of spindles for the necessary IOPs. 1Gbps network will not cut it, 10Gbps network is not cheap either. Hence NAS to me doesn't need to be performant, rather they need to be as SAFE as technologically possible.

4. Remote access: I guess there are tools/methods that can be used that is similar to what those NAS boxes offer? Any recommendations?

Synology offers VPN servers for you to remotely secure connected into your premises. It can be either of OpenVPN, IPSec/L2TP(not cisco IPSEC) and PPTP. Using that you can have remote access to your data in a secure manner via VPN.

Synology also do allow external access for AFP, NFS, CIFS/SAMBA and WebDAV, so it's up to your decision how you want to access your data remotely.
 

jk_eugenetay

Member
Joined
Jan 22, 2006
Messages
246
Reaction score
0
hey david

Thanks so much for your advise!

I have did a fair share of homework about Storage Spaces, and found out its still quite in a buggy (and disappointing) state.

I have bought a Synology DS713+, after considering about it, there are many possible uses & features, plus most importantly, like u said, putting the data in safe, safe place.
VM wise, after sorting out my data, I can free my the 1TB WD black in my PC.

Btw, you mind share what NAS are you using? any tips about functions, features or practice you think its good? thanks!
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,550
Reaction score
1,302
hey david

Thanks so much for your advise!

I have did a fair share of homework about Storage Spaces, and found out its still quite in a buggy (and disappointing) state.

I have bought a Synology DS713+, after considering about it, there are many possible uses & features, plus most importantly, like u said, putting the data in safe, safe place.
VM wise, after sorting out my data, I can free my the 1TB WD black in my PC.

Btw, you mind share what NAS are you using? any tips about functions, features or practice you think its good? thanks!

I am currently using Synology DS1511+. It is a 5-bay NAS. I configured RAID6 for 4 disks array and 1 disk as backup. I also uses AWS S3 for backup of certain highly valuable data. All backup from main array to the backup disk is automated daily, so is the AWS S3 backup.

Every day (except sunday), SMART quick test will run. Every sunday SMART extended test will run. At different times, every month a full scrubbing of the RAID 6 array will run. My NAS runs 24by7, so all these are done automated without my intervention.

My NAS is also backed by a APC UPS CS350 I think. It objective is to provide power surge protection, suddenly power outrage. It is not meant to let the NAS stay online forever. It main purpose is to gracefully shutdown the NAS should the outrage prolong.

This is necessary to prevent RAID5 write-hole issue and data corruption due to sudden power failure.

My NAS also function as RADIUS server for my Wireless router for Enterprise WPA2 authentication. It also function as a bit torrent client and also function as a DNS caching server.
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ Forums. Forum members and moderators are responsible for their own posts. Please refer to our Community Guidelines and Standards and Terms and Conditions for more information.
Top