Need advice to build my own NAS

[Lamberto]

Hi everyone,

So the short and sweet is that I want to build a NAS for my video editing and I want some advice. I have a max budget of about R25k and would like it to be at least 20TB.

I have built quite a few pcs but never touched a NAS before so any help would be appreciated.

 

[Oops!!!]

Hi everyone,

So the short and sweet is that I want to build a NAS for my video editing and I want some advice. I have a max budget of about R25k and would like it to be at least 20TB.

I have built quite a few pcs but never touched a NAS before so any help would be appreciated.

Are you going to be using the NAS for production data or just for storage?

What type of size files do you work with?

Do you require 20TB of storage with redundancy?

What switch will the NAS be connected to?

 

[Scarlet Scarab]

Checkout freenas. Try to get redundancy, a drive almost always fails. Also try to get red nas drives new, they usually come with a decently long warranty. You can cheap out on the system a bit but not the drives otherwise it will cost you more overtime. I've seen some nas drives that have run for a good 6 years or more 24h straight.

 

[solz]

Building a Nas is easy, as specially if you are just given the storage requirement. So get you 2 x 10tb drive and you sorted. If you want redundancy do raid 5 with 3 x 10tb. Keep in mind when one fails it will take very long to rebuild.

So give us bit more info, what you want to use them for. If you constantly working from this location/storage. Cause might be better to go raid 10. Will you use 1gb network or 10gb.

Drives are not like ssd. You can get them second hand if you have redundancy. This will save you some bucks as new drives can eat that 25k quickly.

 

[Billionairebum]

Hey

So I built out storage systems specifically for Post Production, usually on the larger end but also a few smaller ones. For your case I would consider the TOTAL amount of USABLE storage you require, consider the codecs, file types, amounts cams or sources in your timeline. This will give you an idea of IOPs and Throughout you need to budget for. From there you can start considering your options.

Filesystems: stornext and gpfs are great from the enterprise side, for this I would look at XFS or ZFS, probably ZFS.

10G base-t runs pretty hot and expensive, if you cannot use a proxy based workflow and require 90+mbps then consider mikrotiks small 10gig fibre offerings or better yet go direct PC to server. Look at FS.com

TrueNAS may suit you best if you have never touched a NAS or ZFS. OMV is another offering. unRAID is not recommended for this use case.

I run a small 160tb(raw) system for Linux ISOs and then an SSD tier for any more performant tasks like dBs, queries, k8s etc.

 

[Billionairebum]

@zakk is also a great source of knowledge.

 

[souljazk]

I'd also consider your network security. Last thing you want is some ransomware or a guest with a compromised device to connect to your network and have access to the same LAN that the NAS is on.

I know I'll get some flack for this but if this is just for basic needs of 1 person, perhaps a Linux or Win 10/11 PC with secured shares on it. I'd also make contingency for offsite backup to cover you from theft, flood, Eskom or fire.

This does not have to be cloud storage, which can become super costly, but a secured spare PC + disks + network segmentation & something like Wireguard / talescale / zerotier or Cloudflare can accomplish this task securely.

 

[Lamberto]

Are you going to be using the NAS for production data or just for storage?

What type of size files do you work with?

Do you require 20TB of storage with redundancy?

What switch will the NAS be connected to?

So this will be used for production data.
The files are about 50GB at max.
Yes, I plan on running RAID 5
I think I have changed my mind to creating a DAS system. My setup is just a macbook connected to a thunderbolt dock and then want to connect this. I do video and photo editing and want to move away from portable SSDs to something more reliable.

 

[Lamberto]

Checkout freenas. Try to get redundancy, a drive almost always fails. Also try to get red nas drives new, they usually come with a decently long warranty. You can cheap out on the system a bit but not the drives otherwise it will cost you more overtime. I've seen some nas drives that have run for a good 6 years or more 24h straight.

Thanks!

I've just read heard the same thing from the one youtube video I watched. If this will be used for data being used rather than just for storage should I rather use different hard drives with an SSD cache?

 

[Lamberto]

Hey

So I built out storage systems specifically for Post Production, usually on the larger end but also a few smaller ones. For your case I would consider the TOTAL amount of USABLE storage you require, consider the codecs, file types, amounts cams or sources in your timeline. This will give you an idea of IOPs and Throughout you need to budget for. From there you can start considering your options.

Filesystems: stornext and gpfs are great from the enterprise side, for this I would look at XFS or ZFS, probably ZFS.

10G base-t runs pretty hot and expensive, if you cannot use a proxy based workflow and require 90+mbps then consider mikrotiks small 10gig fibre offerings or better yet go direct PC to server. Look at FS.com

TrueNAS may suit you best if you have never touched a NAS or ZFS. OMV is another offering. unRAID is not recommended for this use case.

I run a small 160tb(raw) system for Linux ISOs and then an SSD tier for any more performant tasks like dBs, queries, k8s etc.

Thanks for all the advice, I don't think I fully comprehend everything at the moment but might message you later on if I have more questions.

 

[Lamberto]

I'd also consider your network security. Last thing you want is some ransomware or a guest with a compromised device to connect to your network and have access to the same LAN that the NAS is on.

I know I'll get some flack for this but if this is just for basic needs of 1 person, perhaps a Linux or Win 10/11 PC with secured shares on it. I'd also make contingency for offsite backup to cover you from theft, flood, Eskom or fire.

This does not have to be cloud storage, which can become super costly, but a secured spare PC + disks + network segmentation & something like Wireguard / talescale / zerotier or Cloudflare can acomplish this task securly.

Thanks man, I haven't really considered the security but will definitely have a look into it. For offsite backup I'm still trying to figure out a plan cause I work from home so don't really know where to put it. Might have to bite the bullet and go for the cloud option.

 

[Switch]

Yes, I plan on running RAID 5

Raid 10 is the safer option, there is a reason why raid 6 exists to replace raid 5 and even then raid 6 isn't generally recommended often.

 

[Lamberto]

Raid 10 is the safer option, there is a reason why raid 6 exists to replace raid 5 and even then raid 6 isn't generally recommended often.

Thanks, I don't actually know what raid 10 does so will go do some research on it.

 

[zakk]

@Switch literally almost every entry level enterprise SAN recommends RAID6.

@Lamberto

I think simplicity is key for your setup, since you want to manage it yourself.

Perhaps something as simple as 2 x 16TB or 2 x 18TB's in RAID1 is the way to go.

The enclosure / housing etc is your choice.

2.5GbE has become cheap (even an adapter for Mac) so a basic PC with a RAID1 setup might be a good bet.

There are many better options but not at R20k, I think like others have said invest in good drives (HGST UItraStar Helium has the lowest failure rate).

If you do RAID5/6 consider more smaller drives for faster rebuild speed.

With 4 drives RAID6 is safer, you can lose any 2 disks. In RAID10 if you lose the wrong 2 disks you are also screwed.

More than 4 disks RAID10 could be safer, but only if the 'correct' drives fails.

Rebuild on RAID10 is a lot better though, however RAID6 is technically still safer due to the rebuild strain/load.

 

[JollyJamma]

Hi everyone,

So the short and sweet is that I want to build a NAS for my video editing and I want some advice. I have a max budget of about R25k and would like it to be at least 20TB.

I have built quite a few pcs but never touched a NAS before so any help would be appreciated.

I’d just repurpose an old computer and use a chassis that can hold the number of drives you need to acquire the nett total storage.

An old AMD Ryzen 1600AF and a B450 or B550 motherboard with an SSD for cache and 16 to 32gigs off RAM would work well.

I’d go ATX so you can install a PCI-e to SATA adapter and LAN card.

LTR did a NAS setup a while ago (I watched it again recently) where they went through the reasons for using certain components and they used Intel Optane as the caching drive as it’s very durable as SSDs go.

I’ve done some research on using a NAS to backup Mac devices and all of the Reddit comments said how it works fine for a while but then becomes buggy and they moved to using a Mac Mini for a backup solution.

I know you aren’t specifically looking at backing up your Mac but if you did, it’s just a word of caution as it’s something I considered when researching NAS backup solutions.

 

[Billionairebum]

If what we are all saying is a bit far out of budget, consider using a 2tb SSD/nvme for your working pool and a simple 18tbx2 raid 1 for your offline pool. Do backups, syncs etc. cheapest solution, least expandable though.

Just buy a Lekker Petabyte SAN with tiered storage

 

[lowracer]

Stupid silly option.

Why not get a nas, or two, or three. for redundancy and security. it might be a liitle bit more work, but bear with me

Trick out your workstation with fast storage. not sure if you are on laptop or desktop, but that might affect other choices. if you are on a laptop, and you dont have an inverter solution, you might have to get one of the dc upses to run the first nas during loadshedding - straight 12v or 19v out from the dc ups.

Set up scheduled jobs to make backups from your workstation to the nas - this is a push of the files, and you need access to the nas's folder. Some software also allows for making a file copy of each change to the nas and some can integrate with apple time capsule or similiar.

now on your second nas, you set up scheduled copy jobs on the second nas, that will copy from nas 1 to nas 2. so you need access from nas 2 to nas 1, but your own computer doesnt have direct access to nas 2 - so ransomware might hit your computer, and then the drives on nas 1, but not nas 2. The copy jobs will be one of two types - copy only files i dont have, and always copy all files that has changed since last time. the first option will copy the original media - photo/video/etc, but not any subsequent changes. the second option will ensure that you have a recent version of "changeable" files like project files. you can set up multiple copy jobs to copy into monday/tuesday/etc folders, so only at day 7 will it start to overwrite. that gives you enough time to rollback to an older version. (with these, you can do it on one or split the load onto 2 distinct nases. if anything goes wrong, you still have another. )

yes, it will be a lot of redundancy and duplicates. what is the cost of a day's worth of lost time when you have to redo something and you have tight deadlines

(if you get another nas after this nas - it will be a mostly offline option - where you switch it on weekly, and copy whatever you want to copy, and switch it off again - or you can manually copy to a usb drive)

 

[pikolec]

All good advice above. You need to understand your use cases though. If you're using the NAS for storage, or if you intend to edit off the NAS are two very different use cases and they will have a big difference in cost. The upgrade from a normal 1G to 2.5G network can also have a noticeable improvement, but something that you can invest in later.

A 4 disk RAIDz2 configuration can easily saturate a 1Gb link for a single user.

If you can, try to get 5400RPM drives. The difference in the seek times between 5400 and 7200 isn't noticeable, but the amount of heat is (and heat kills drives). Unless you have AC . "NAS rated" 5400 drives are quite difficult to come by, I think only WD does them.

You max budget is also pretty tight. To get to about 20T with some redundancy:
3 x 12T (24 + 1 parity, minus losses) ~ 3 x R 5800 ~~ R17 400
5 x 6T (24 + 1 parity, minus losses) ! 6 x R3800 ~~ R22 800
... and you haven't got anything to put them in yet.

However, those are RAIDz1 configurations, RAIDz2 is recommended, so you end up with:
4 x 10Tb ~~R22k
6 x 6TB ~~ R27k

I'd recommend factoring in 2.5G networking, as previously mentioned, it is cost effective, but you would ideally want to factor in a switch if you don't/can't run a direct cable.

For NAS hardware, also consider the overall IDLE power consumption. Trying to keep an ex-desktop machine running during loadshedding (unless you have solar+batteries, etc) just gets annoying when something much lighter will do.

 

[Lamberto]

Sorry everyone, will respond sometime tomorrow hopefully. Got quite ill over the weekend and haven't had a chance to respond to messages.

 

[Lamberto]

So long story short I've been sick most of the week and what I've decided to do is build a pc in the define 7 XL and build an editing nas that is fairly simple.

It might not be the cheapest option but server noise and my space constraints make it the best option, and I also prefer to be able to work on the system if something goes wrong but I simply don't have near enough knowledge on servers.

So yeah, might not be the best overall choice but I think its the best for now.

 

[Billionairebum]

So long story short I've been sick most of the week and what I've decided to do is build a pc in the define 7 XL and build an editing nas that is fairly simple.

It might not be the cheapest option but server noise and my space constraints make it the best option, and I also prefer to be able to work on the system if something goes wrong but I simply don't have near enough knowledge on servers.

So yeah, might not be the best overall choice but I think its the best for now.

Well so far with what everyone has recommended this is actually a fantastic start. I looked for one for a long time without success. Send us some updates seeing as we all invested now.

 

[Mr.Bob]

Well so far with what everyone has recommended this is actually a fantastic start. I looked for one for a long time without success. Send us some updates seeing as we all invested now.

Yeah, I was shocked to see that those ones with terrible RMA policies, have it for 4k.

 

[warlock072]

Some considerations i would try keep upfront, ZFS and HBAs. Everything else is preference in my opinion. Video Editing i would consider 2.5Gb ethernet at least but 10Gb isnt far off cost wise. Depending on whether or not you editing directly on the NAS then you might wanna consider a small SSD/NVMe cache and use spinning disks for storage, to archive anything close to an SSD speed with cheap spinning disks you would need a Large number of disks! not worth it in my opinion.

 

[el3venth]

Checkout freenas. Try to get redundancy, a drive almost always fails. Also try to get red nas drives new, they usually come with a decently long warranty.

Yes to FreeNAS.

Hard NO to WD Red drives. Unless you are talking about red drives in some generic NAS storage capacity.

Western Digital NAS drives flash warnings after 3 years, even without problems

Some Western Digital NAS drives appear to be sending out "replace me!" warnings after an arbitrary time limit even in the absence of any actual hardware faults.

www.pcworld.com

Western Digital admits 2TB-6TB WD Red NAS drives use shingled magnetic recording – Blocks and Files

Some users are experiencing problems adding the latest WD Red NAS drives to RAID arrays and suspect it is because they are actually shingled magnetic recording drives submarined into the channel. Alan Brown, a network manager at UCL Mullard Space Science Laboratory, the UK’s largest...

blocksandfiles.com

Oh, and lastly. If you do build your own, then buy a second motherboard or RAID controller (and make sure it has the same firmware).
Everything else is easy to replace if it blows. But if a motherboard goes and your RAID configuration was not that standard, you might have a hard time to get everything back online.

But just swopping out the motherboard 1 for 1 is super easy. And you might save more in terms of time than trying to rebuild your NAS on other hardware.

 

[warlock072]

What i did a while ago is bought one of these small HP microservers and played around with various software options before actually seriously investing in hardware. I now have 2 xeon servers , 1 unraid , mostly for home stuff. 1 proxmox, lab stuff. A large combination of switches and stuff. This has just been the way ive developed my setup over time. I dont wanna think or have anything in writing about how much ive spent on it, this way the wife cannot hold me liable! lol

 

[Billionairebum]

What i did a while ago is bought one of these small HP microservers and played around with various software options before actually seriously investing in hardware. I now have 2 xeon servers , 1 unraid , mostly for home stuff. 1 proxmox, lab stuff. A large combination of switches and stuff. This has just been the way ive developed my setup over time. I dont wanna think or have anything in writing about how much ive spent on it, this way the wife cannot hold me liable! lol

A fellow r/homelab -er!

 

[Data Lab Stellenbosch]

A few wise words, NAS does not stand for backup !!

So you still need to look at a separate backup disconnected from the NAS.

 

[ettiene]

I want some clarity regarding Raid 5, is the following true?

When 1 drive fails you need to replace it with the exact same model, while your data will still be accessible because the controller will be rebuilding the data from the parity and remaining drives on the fly as its requested. Till you find a exact replacement you are running a high risk of losing the whole array and all your data if another drive fails. So you'll have to invest in a complete new array and move everything over, hoping nothing fails in the days it's being moved over.

When the controller fails you need to replace it with the exact same model. Other models/makes can't read the drives from another controller. If you can't find a replacement all you data is gone.

Due to bathtub failure curve the failure rate of all the drives is at approx the same time because they are bought at the same time and possibly from the same batch.
This does not bode well when your drives are a few years old and you got one failure.

 

[ettiene]

Just to add something totally different.
We had 2 break ins, at our house and company office. Many things where stolen, jewellery, name brand jeans, tvs, computer screens, laptops etc. On both occasions the Nas wasn't stolen because hp micro server doesn't really look like a computer.

In fact the one at home stood on the table in front of the window that they broke open to gain access, meaning they passed it many times stealing the more "valueble" items.