Tape advice

tifosi

Senior Member
Joined
Dec 19, 2000
Messages
1,608
Reaction score
0
Hi all, I am considering buying a LTO 6 tape drive like the IBM TS2260.

But I realise Taknet at Sim Lim Square has closed down. No idea where it has moved to.

Anyone know where I can get the drive from other shops in Sim Lim?

Thanks a lot!
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
Hi all, I am considering buying a LTO 6 tape drive like the IBM TS2260.

But I realise Taknet at Sim Lim Square has closed down. No idea where it has moved to.

Anyone know where I can get the drive from other shops in Sim Lim?

Thanks a lot!

Just a suggestion for your consideration. Have you consider backup to AWS S3 and Glacier ? You can easily deploy a AWS VTL Gateway ?
 

tifosi

Senior Member
Joined
Dec 19, 2000
Messages
1,608
Reaction score
0
Yo david i did consider S3 but the recurring cost just to keep the data isn't worth it. I also intend to archive approx 8 terabytes. will be too slow to use an cloud solution

The latest LTO 6 can be written to at 150 MB/s.

Thanks so much for chiming in
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
Yo david i did consider S3 but the recurring cost just to keep the data isn't worth it. I also intend to archive approx 8 terabytes. will be too slow to use an cloud solution

The latest LTO 6 can be written to at 150 MB/s.

Thanks so much for chiming in

Indeed if your network bandwidth may be a concern. Just curiously asking, how many LTO-6 tapes will you be getting to store your 8TB, 2 or 3 ? and how much changes are you expecting in percentage and your retention period is for how long and how often do you backup ? I would assume daily with incremental ?

The price for the LTO-6 drive seems hefty at roughly USD3K, just wondering if a decent NAS with green drives would make more sense for you ? Just a suggestion. Even if you are in a UPS and extra drives for failures may still be lower than the price of the tape drive :)
 

tifosi

Senior Member
Joined
Dec 19, 2000
Messages
1,608
Reaction score
0
For now I will get roughly 10 LTO-6 tapes and rotate them. Compression will be used so I can get away using 2 tapes on each backup session which will be done every 2-3 days.

I highly doubt I will require more than 20 terabytes worth of backups at the middle of 2015.

I did consider NAS - but failures are silent and the heat given out by NASes makes it a deal breaker. I am using this for home use.

I have a server with roughly 12 terabytes worth of harddrives and it does give out quite a bit of heat so I was thinking a backup strategy using tapes could be worth considering since they are much more reliable, have lower power consumption (compared to filling the server with more harddrives) and offer better value (in terms of consumable cost)

You seem to know a fair bit. Have you used LTO btw? Any gotchas that made you wary of such things? Thanks!
 

cscs3

Arch-Supremacy Member
Joined
Jun 4, 2000
Messages
21,724
Reaction score
133
Believe LTO 6 can hold up to around 6TB per tape. Tape still around, good thing is you can make multiple backup at media cost.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
Believe LTO 6 can hold up to around 6TB per tape. Tape still around, good thing is you can make multiple backup at media cost.

Probably too ideal a scenario for TS. Unless TS is trying to backup 8TB worth of system data or text file or binaries, the likelihood to achieve 2.5X compression rate is low. I suspect his 8TB worth of data are mostly videos or photos. These are highly incompressible, in fact compressing them may yield larger files instead. I would say if TS ca achieve 1.2X compression would be roughly there.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
For now I will get roughly 10 LTO-6 tapes and rotate them. Compression will be used so I can get away using 2 tapes on each backup session which will be done every 2-3 days.

I highly doubt I will require more than 20 terabytes worth of backups at the middle of 2015.

I did consider NAS - but failures are silent and the heat given out by NASes makes it a deal breaker. I am using this for home use.

I have a server with roughly 12 terabytes worth of harddrives and it does give out quite a bit of heat so I was thinking a backup strategy using tapes could be worth considering since they are much more reliable, have lower power consumption (compared to filling the server with more harddrives) and offer better value (in terms of consumable cost)

You seem to know a fair bit. Have you used LTO btw? Any gotchas that made you wary of such things? Thanks!

My experience with Tape related backup is while I'm working with in a SI, designing system with include the need of a tape library, 20+ LTO-5 tapes for backing up 10+ servers using Symantec Backup S/W

With 10 tapes, you will work like a mechanic trying to switch the tapes during backup if you are using a single drive tape drive. For these much of tapes involved, I would have used a tape library. Unfortunately tape library are rack mounted and probably wouldn't be useful for your case unless you have a rack at home ?

Tapes are reliable when you store them properly like in dry cabinets or cool dry storerooms. However the bad part is what people don't normally think about, how do you know a piece of tape stored in your store room is still usable ? Bit flips ? fungal tape ? Like I say you need to take care of it nicely for it to work properly for you.

I don't know if you have tried restoring a system with a tape, it's a painful process that potentially take hours. Have you used tapes before, you need a catalogue that tell exactly which tape is required when you restore. That is provided by the backup software. You can use the Amanda if you like, but it's not going to be a easy process. So if you have worked with backup/restore operations before, perhaps you can give it a try.

I am not sure which class of NAS have you tried before ? I have a 5 bay Synology DS1511+ NAS running 24by7 for the last 2 years using RAID 6(upgraded halfway from RAID5) on an assorted set of WD RED and WD GREEN disks. With 1 single disk for backup.

Your concern on bit rot can be easily addressed using periodic disk scrubbing. It will create opportunity for the SMART in each disk to detect failing sectors and also for the RAID controller to ensure RAID consistency of the array. This is done automatically and any early detection of failing disks will be notified to you via email.

My NAS is running almost noiseless even during night time, and the power consumption of 5 disks is only 70+W(if I remember correctly).

For your case, I will recommend you get a Synology DS2413+ or equivalent for other branding. That will set you aside for SGD2K only, not a single LTO-6 tape drive that will cost you USD3K. Get 5x 3TB WD RED and form a RAID 6 array, it will give you near 9TB worth of diskspace. Get another 4x 3TB RED and form a RAID 5 across for you to perform backup from your main array to this backup array. That would be a total of 1.6K. Add together it only cost you SGD3.6K. Go get a APC UPS NAS that will cost you roughly <SGD200. You will still have 3 slots for you to play with. If you really have slots issues, get a DX1211+ expansion unit to scale up another 12 slots via Infiniband interface. That should give you sufficient headroom for 20TB or more.

The whole setup have redundancy and daily backup and data scrubbing all happening automatically without your intervention. When you store data in it, you know it is safe.

Any hard disk fail, just change, you have 2 set of redundancies and 2 layer of protection. If you are keen, just use Amazon S3 for your highly critical data. That's how I do it. Hence I have up to 3 layers of data protection.

What if the unit is down. Simplest solution is get another unit or upgrade to another NAS available. Synology is using Linux RAID (I don't suggest going for SHR), so it is just as compatible on any Linux Box. If for some reason you can't get a replacement unit and you are urgent for the data on the hard disk, just go get a linux box with sufficient SATA ports, plug them all in and assemble them manually using the Linux MDADM tool.

That's the beauty of using Synology. Can't say the same for other brands. Think about it and see if what I have suggested works for you.
 

tifosi

Senior Member
Joined
Dec 19, 2000
Messages
1,608
Reaction score
0
wow david, thanks so much for your very comprehensive reply. i really appreciate you so generously sharing your experiences!

yes you are right, compression ratios are normally 1.1x unless there is a lot of text. i admit i was stretching a little bit. and i do not intend to use compression after all.

as for tape, yes good point. playing musical chairs with tapes is no fun at all. no i do not have a rack at home - and was gunning for a external tape drive. i have a dry cabinet at home, so storage of tapes should not be an issue.

thank you for your recommendation of synology. any opinions of thecus btw? i will definitely consider my options. thanks!
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
wow david, thanks so much for your very comprehensive reply. i really appreciate you so generously sharing your experiences!

yes you are right, compression ratios are normally 1.1x unless there is a lot of text. i admit i was stretching a little bit. and i do not intend to use compression after all.

as for tape, yes good point. playing musical chairs with tapes is no fun at all. no i do not have a rack at home - and was gunning for a external tape drive. i have a dry cabinet at home, so storage of tapes should not be an issue.

thank you for your recommendation of synology. any opinions of thecus btw? i will definitely consider my options. thanks!

I would say for the price you have invested, the amount of safety, redundancy and flexibility are very limited. That's why I suggest you consider using harddisk.

Couple of years back I have considered tape too, that was before I got to know tape backup and restore procedure up close. When I start to look at the tape drive back then, I find it hard to justify outside of enterprise usage.

Even when I look at small enterprise usage on project basis, still doesn't justify the cost invested in all the hardware, software and human resources. I would rather the company invested in proper network uplink and use the cloud storage instead. No doubt tapes allow for off-site storage, but today, it's easily achievable using harddisk and off-site cloud storage too.

For your usage, you will benefit a lot more from a near fully automated system versus mundane job playing with the physical tapes.

It might be clearer to you, if you list down what exactly you need to do with tapes and how often you are doing it. I would rather you are paid doing it since it's a chore even from a hobby perspective.

My first hand experience is with Synology due to the cost and flexibility and reputation. Thecus is on the high side, same for QNAP. I think I have did a deeper study on QNAP. Features wise, they are similar, and internal flexibility without the unit is similar too since they are riding on Linux implementation with MD RAID. Thecus seems do it differently, so I'm not too sure.

Synology to my knowledge seems to have a wider community given its price point and highly hackable nature.
 
Last edited:

quekky

Arch-Supremacy Member
Joined
Nov 12, 2000
Messages
18,440
Reaction score
0
What if the unit is down. Simplest solution is get another unit or upgrade to another NAS available. Synology is using Linux RAID (I don't suggest going for SHR), so it is just as compatible on any Linux Box. If for some reason you can't get a replacement unit and you are urgent for the data on the hard disk, just go get a linux box with sufficient SATA ports, plug them all in and assemble them manually using the Linux MDADM tool.

SHR is using linux md too, so you can just plug into any recent linux kernel and can still mount it. Just that Syno created multiple partitions and md them together to make the most out of your space. I tried before previously in DSM4 and it does mount properly in fedora

I also prefer platters to tracks
 
Last edited:

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
SHR is using linux md too, so you can just plug into any recent linux kernel and can still mount it. Just that Syno created multiple partitions and md them together to make the most out of your space. I tried before previously in DSM4 and it does mount properly in fedora

I also prefer platters to tracks

Yes, Synology SHR is implemented using both MD and LVM. It's great to know they can be mounted easily on Linux. My initial reason to keep with normal raid setup mainly is mainly the followings
1) Uncomplicated setup. Using LVM does convolute the setup with flexibility and slight penalty.
2) Due to the interdependencies between the logical volumes, performance is normally lost when both volumes are access together. Mileage varies depending on use cases.
3) I have not tested it out in scenario where during a degraded migration if it still works properly. For normal Linux S/W RAID, it is possible to reassemble an array even with one of the disk missing. For some really disaster case, if you have the original layout, you can assemble things back together and bring the array online. I'm not so such if introducing LVM into the picture makes it still possible.

Of course, if things are tested out, and one is capable of managing it, feel free to go ahead with SHR.
 

abcd0_12340

Senior Member
Joined
Apr 11, 2009
Messages
1,344
Reaction score
0
Just came by this thread to know more about tapes.
I must agree that this is very informative and I've learnt a lot from here.

David, thanks for sharing!
 

Mankul

Arch-Supremacy Member
Joined
Jan 1, 2000
Messages
18,509
Reaction score
0
Hi all,

Im an home ammatuer videoman recording faith related lectures at a rate of 10hrs/week and Ive been using miniDV format cos of its rather convinient archive. Problem is that the proconsumer videocam industry has moved to flash based videocams and the thought of archival process has put me on a dilemma, hence my quick research has pointed me to LTO6 tapes.

The thought of HDD failures and unfamiliarity with NAS(havent got one) has been giving me headaches! Heard also horror stories of how an entire NAS can fail if the system rebuilds itself wrongly cos of failure scenarios. And because of the ever increasing videos I record, NAS will have difficult in storage expansion.

I quick look at canon videocams sugesst that roughly 10hrs worth of video will take 30GB. These videos are usually lectures so high quality is not needed, just something decent. LTO6 tape capacity of say 3TB(rounded number for easy calculation) capacity with slight compression will give about 3TB Ă· 30GB 100 weeks of video(at 10hrs/week) ... so each LTO6 tape an store about 1.5yrs worth of videos.. thats pretty decent. And with each generation of LTO tapes doubling in capacity, the number of tape media will be less when I migrate up my archive. I plan to migrate my archives after 2 generations, ie LTO6 to LTO8.

What do you guys think?

My only startup cost is the cost of LTO6 drive, more conviniently an external one. Need to research on how to set up SAS host interface.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
Hi all,

Im an home ammatuer videoman recording faith related lectures at a rate of 10hrs/week and Ive been using miniDV format cos of its rather convinient archive. Problem is that the proconsumer videocam industry has moved to flash based videocams and the thought of archival process has put me on a dilemma, hence my quick research has pointed me to LTO6 tapes.

The thought of HDD failures and unfamiliarity with NAS(havent got one) has been giving me headaches! Heard also horror stories of how an entire NAS can fail if the system rebuilds itself wrongly cos of failure scenarios. And because of the ever increasing videos I record, NAS will have difficult in storage expansion.

I quick look at canon videocams sugesst that roughly 10hrs worth of video will take 30GB. These videos are usually lectures so high quality is not needed, just something decent. LTO6 tape capacity of say 3TB(rounded number for easy calculation) capacity with slight compression will give about 3TB Ă· 30GB 100 weeks of video(at 10hrs/week) ... so each LTO6 tape an store about 1.5yrs worth of videos.. thats pretty decent. And with each generation of LTO tapes doubling in capacity, the number of tape media will be less when I migrate up my archive. I plan to migrate my archives after 2 generations, ie LTO6 to LTO8.

What do you guys think?

My only startup cost is the cost of LTO6 drive, more conviniently an external one. Need to research on how to set up SAS host interface.

You might as well store them up into the AWS cloud S3 and for archival purpose, setup the lifecycle routine to push them into the much lower cost glacier.

Each 30GB in S3 will only cost you USD0.75/mth. Glacier is not available in SG region, but the nearest and fast enough is at Sydney or Japan at roughly USD0.30-0.35/mth

On long run, definitely more attractive than your tape backup solution. S3 and Glacier offers 99.999999999% durability.

You can even set your own retention period, after sometime like a 12 months that the object auto remove.

The only pre-requisite is good upload speed. Which you can easily get 1Gbps fibre nowadays for consumer broadband at really low cost.
 

Mankul

Arch-Supremacy Member
Joined
Jan 1, 2000
Messages
18,509
Reaction score
0
You might as well store them up into the AWS cloud S3 and for archival purpose, setup the lifecycle routine to push them into the much lower cost glacier.

Each 30GB in S3 will only cost you USD0.75/mth. Glacier is not available in SG region, but the nearest and fast enough is at Sydney or Japan at roughly USD0.30-0.35/mth

On long run, definitely more attractive than your tape backup solution. S3 and Glacier offers 99.999999999% durability.

You can even set your own retention period, after sometime like a 12 months that the object auto remove.

The only pre-requisite is good upload speed. Which you can easily get 1Gbps fibre nowadays for consumer broadband at really low cost.


I'm anticipating an archive size which increase 6TB every 3 years. In ten years time, it probably breach 20TB and the monthly cost will be crazy.
 

Mankul

Arch-Supremacy Member
Joined
Jan 1, 2000
Messages
18,509
Reaction score
0
http://ask.slashdot.org/story/14/03/12/1253218/how-do-you-backup-20tb-of-data

Read the above link. Ignore the first few joking replies.

Its seems that they are all pointing to LTO6 as the solution.

The amount you pay to store 20TB of data on Glacier for a year, can buy you an LTO6 drive. So LTO6 seems to be the most cost-effective way.

Why not hard disks? LTO6 tapes kept in a dry box is least likely to crash compared to an NAS HDD.

What do you guys think?
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
I'm anticipating an archive size which increase 6TB every 3 years. In ten years time, it probably breach 20TB and the monthly cost will be crazy.

For those tapes that are more than 1 year, are you looking at retrieving them, or just pure archival ? If it is pure archival, you can shift them into Glacier.

That would cost you roughly USD250/month to maintain.

Another way is if they are of certain age like more than 5 years, you can encode them into MP4 and store them away.

Of course, this is a recommendation. After all, you are getting a service of 11 '9' durability, not just a NAS or tape which is unlikely to reach that durability.

Just for your consideration
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,300
http://ask.slashdot.org/story/14/03/12/1253218/how-do-you-backup-20tb-of-data

Read the above link. Ignore the first few joking replies.

Its seems that they are all pointing to LTO6 as the solution.

The amount you pay to store 20TB of data on Glacier for a year, can buy you an LTO6 drive. So LTO6 seems to be the most cost-effective way.

Why not hard disks? LTO6 tapes kept in a dry box is least likely to crash compared to an NAS HDD.

What do you guys think?

If you are looking for really long term storage and you have the necessary facility to store your tapes, you can certainly opt for it. Just that, you do know it is not fun to retrieve from tapes and you have little way to detect something stored in the tape is still uncorrupted after 10 years.

You can most certainly built large RAID storage and still have a backup RAID to prevent lost of data.
 

Mankul

Arch-Supremacy Member
Joined
Jan 1, 2000
Messages
18,509
Reaction score
0
If you are looking for really long term storage and you have the necessary facility to store your tapes, you can certainly opt for it. Just that, you do know it is not fun to retrieve from tapes and you have little way to detect something stored in the tape is still uncorrupted after 10 years.

You can most certainly built large RAID storage and still have a backup RAID to prevent lost of data.

Cost of two raids would be equivalent to 1 LTO6 drive.
Just a home user wanting to back up sentimental videos.

Paying monthly recurring charges at usd200+ just doesnt seem appealing.
Im afraid of RAID rebuilts and catastrophic failures.

Just to give you an idea why LTO6 solution is appealing to me... I can reduce this amount of 3 yrs worth of miniDV tapes to just TWO LTO6 tapes! haha see pic below.
FB_IMG_1430293264672_zps19gvj47u.jpg


The videos are just for archive after I uploaded them to my website. I dont think I will need to access them at all. Maybe O just have a set of tapes for redundancy.

I will likely just build an NAS using synology 415+ as first layer of backup.

Since you deal with tapes before, what kind of tape failures can happen? total tape failure or just a section of 'bad sectors'?
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ Forums. Forum members and moderators are responsible for their own posts. Please refer to our Community Guidelines and Standards and Terms and Conditions for more information.
Top