WTB: Large decent NAS storage device with back up and advice on transferring files

[Lawrence001]

So I've consistently had issues with copying ripped albums from my Zen device to my USB HDD with my laptop.

I think the first thing I need is to get a decent NAS storage device, and then find a way of transferring the files easier.

I probably need 10TB of storage to get futureproof, and I'd like a back up if not 2 of all that data. I'd also like reasonably fast access to the data and no problems streaming or getting a full list of the albums. I don't need it to stream HD videos it anything though so it doesn't have to be the latest spec.

Is this all reasonable or is it going to cost a small fortune to get this much storage, even given the cost is constantly falling.

I guess I don't mind a used device but I probably need little-used storage to last longer. Maybe I'll have to buy the drives themselves new as who's going to sell their nearly new storage (unless they drop an unfinished NAS project).

The advice I then need is how to easily copy thousands of albums from my Zen to my drives. When I've done it in the past I've used Windows file explorer on my laptop to select the albums I want on the Zen, copy and paste them to the HDD.

Unfortunately every 5 mins or so it throws an error, something like file not found and you've got to select try again, go to next one, or cancel or something like that. And a tick box to do this for all files.

As you can imagine, if it's going to take several days to copy and you have to sit there clicking try again every 5 minutes you're going to lose patience pretty quickly!

I thought about looking into DOS copy but I want to select the folders I want with the mouse, after sorting by last modified so I can select the batch of CD rips I'm backing up from the last one.

Maybe there's a back up option, but I'm worried it might delete the files on the HDD that aren't on the Zen. Ie the HDD should contain everything on the HDD before and/or what's on the Zen, not just become a copy of the Zen.

The other thing I want to do is quickly and easily get the MetaData and Cover Art for a big batch of albums already ripped, not while ripping. Ie assume each bottom most folder is a CD and work out, and add, the info for every folder and subfolder selected.

Any help much appreciated! It's got to the stage when I feel like giving up, or deleting the lot and starting again as the cover art and metadata is such a mixed bag, and there will be odd tracks and albums missing from the times I've had to click on ignore or cancel copy from the error messages. And I've done about 2000 albums so far!

[Edward]

to suggest a couple of things you mention Lawrence.

1. Seems like Windows can see/read the folders/files on your Zen. Try Teracopy (https://www.codesector.com/teracopy)to copy over files from your Zen to your Windows drives. Teracopy will retry problematic folders/files and keep going if it can't succeed. It will provide a log of all actions it takes.

2. To populate thousands of albums that are missing metadata (including album art) use JRiver. Make sure you toggle the setting in JRiver to embed the metadata/art in the files (its default is to retain that data in its library). But, if I were you (:)), any new rips you do I would at that time pull down metadata/art. Rippers like EAC pretty much do 90% of albums. From memory JRiver provides a generous full featured free trial (30 days?).

Sounds like you have a lot of already ripped albums. :eyebrows:

Added:

If you are using your laptop to copy files from your zen to a hard disc bear in mind the data is going from the Zen to your laptop and from your laptop to the HDD. This will be slow compared to doing a direct copy, especially if using WiFi on any of the legs. Try to find a way of doing a direct copy (zen to the HDD).

Why do you feel you need a NAS by the way? Perhaps consider getting a computer with sufficient storage.

[Lawrence001]

Thanks Edward there's some really useful advice there. That Teracopy looks like it might be just what I need to transfer the files without much hassle.

Jriver certainly sounds like it might be what I want, I'll have to look into the costs and see if it's worth paying for the number of albums I need to sort out.

Lastly, in terms of a NAS the advantages as I see them are:

Automatic back up (is that what a RAID array does?)

Expandable, although I think 10TB would future proof me forever as I can't see me expanding my collection much beyond what I have as I stream more and more (as in Qobuz etc).

Easy to play as a DLNA server type setup.

Is there anything I've missed, or misunderstood? I think you've had a similar setup for years now?

[Flashinthepan]

I went through this a few years ago, Lawrence, and researched and messaged... and have been happy with where I've landed. Synology NAS (DS212+ recently upgraded to DS220+), with 2 x Western Digital (WD) Red 3 TB drives now replaced by 2 x 10TB WD Red drives. In RAID 1 config, which means my two drives are mirrored so I have "only" 10TB of storage not 20TB. The Red drives are highly regarded but there was a controversial switch from one type to another a year or two ago so ask me (here or in the teepee) before you buy your drives if you go down this route (you want CMR not SMR).

I'm an Apple guy otherwise so probably of little help. I use ALAC not FLAC and iTunes for ripping but not for playback which is via Bluesound Node 2i using the excellent BluOS app.

My old DS212+ and different drives (2x 6TB drives, RAID 0 config giving 12TB capacity, convoluted story) will be in my internet-connected outbuilding and act as backup.

Hope this helps, happy to chat.

Nigel

[Flashinthepan]

We crossed in the post. RAID does not do backups, it's a disc/disk formatting thing. RAID 1 mirrors from one disc to another but that's in the same NAS so if your house gets struck by lightning...

[Lawrence001]

Thank Nigel that's more useful info. Synology has come up in my searches quite a lot, but when I research the ones that come up second hand they always seem to be old and have limited RAM or HDD expandability which puts me off.

Maybe it's no surprise that people only sell them when they're obsolescent as they're a fit and forget item. Hence my asking if anyone has bought one for a similar project that has then been abandoned, as it might be a more recent model and have fresh drives in.

One can live in hope...

[Lawrence001]

And re the struck by lightning thing, even if you have your backups on a separate NAS, if they're both on when the lightning strikes, are they not both at risk still?

[Edward]

Thanks Edward there's some really useful advice there. That Teracopy looks like it might be just what I need to transfer the files without much hassle.

Jriver certainly sounds like it might be what I want, I'll have to look into the costs and see if it's worth paying for the number of albums I need to sort out.

Lastly, in terms of a NAS the advantages as I see them are:

Automatic back up (is that what a RAID array does?)

Expandable, although I think 10TB would future proof me forever as I can't see me expanding my collection much beyond what I have as I stream more and more (as in Qobuz etc).

Easy to play as a DLNA server type setup.

Is there anything I've missed, or misunderstood? I think you've had a similar setup for years now?

Posts have been crossed with that of Nigel's so forgive any duplication.

As Nigel mentioned RAID is not backup. It simply protects against hard drive failure. Also it does not protect against failure of the RAID controller. If that goes you can't access your data unless you buy a new controller (which may or not be available when the controller fails). Best to use backup software and ideally backup software that copies existing folders/file structure rather than placing into different file structure (you need the same software to restore). Or look at solutions like Unraid or DrivePool.

DLNA works but can be flakey. Best to aim for a music renderer located just prior to your DAC. Many solutions out there. JRiver, Roon, LMS, Tidal, Spotify, Quboz etc. Control can be via dedicated or generic apps on your phone.

JRiver is free for first 30 days. Plenty of time to do a full metadata/art for thousands of albums missing that data. JRiver is also first class in connecting up things - but no Quboz, Tidal, Spotify, etc integration though.

Yes you've seen (heard) my setup at various times. But essentially: Windows 10 machine with multi tb storage (using DrivePool) running Roon Core (with Tidal integrated) streaming to devices on the LAN running Roon output (to DAC) and controlled by the relevant Roon app (on phone, laptop etc). But I hasten to add this is not to everyone's way of doing things as housekeeping is at times required (updating software, where to look to fix unexpected events etc). Previously I used JRiver which was great - but I dumped that as JRiver management refuses to integrate Tidal etc (their loss).

Hope above is somewhat useful.

E

[Lawrence001]

Yes indeed all information useful thanks.

So to summarise my plan, my "front end" will be my Liv Zen on which I rip my CDs, and having 3Tb of storage in theory will only run out of space when I've done an awful lot of CDs.

However I don't want to risk losing much ripping work so regular backups are required.

My current process involves taking the Zen from where it is (and is controlled via a cable into a powerline adaptor) and putting it near my router in the hall so that I can connect it directly with an Ethernet cable to do things as quickly as possible.

My storage at the moment is in a large USB HDD therefore I plug that into my laptop, and the laptop into the router with another Ethernet cable. I can then copy the files across in file explorer. This seems to take 24h+ for a few hundred at a time. I don't do it more frequently as the moving of the Zen if such a hassle.

Unfortunately as a certain young child in the house seeing said laptop when they wake up and come down at 6am (well before I stir) is guaranteed to try and play Roblox on it and cancel the file copy I can only do it when the kids are away. Which is only a few times a year.

This is even before I consider the issue of having to click on retry every 5 mins to keep the copy going.

So it's fair to say my current method is suboptimal and I need to improve it if I'm going to rip and regularly back up several thousand CDs.

Therefore the plan is to have a NAS drive thing plugged into the network and some way that the backups happen frequently, quickly and seamlessly. It will also allow me to select albums on the NAS and play them via the Squeezer app on my phone that controls the Zen, which can pick up DLNA servers on the network (I think!).

As I have no idea how LMS/Vortexbox/Linux backups work so this is a voyage of discovery to me.

[struth]

you can , once its all on hard drive, just copy it all over to another hd plugged in, then disconnect it and store it. Take it out every now and then and top up if youve bought any new music etc.
Jriver is about £30 odd and thats it. You get free updates until a new version comes out; but that version you have lasts for ever. Its great for ripping and altering metadata etc. I good player too. Lots you can do with it. I buy a new version regularly as you get a discount if you prebuy.
As for the kids, password protect and they wont be able to open pc .

[Edward]

I have no experience with the Zen box thing you have Lawrence but surely it has some way of backing up the data it contains to an external drive? Maybe the Liv Zen people can help? Can't you plug your USB HDD directly into the Liv Zen?

Does your router have a USB slot? If so perhaps consider simply plugging in a USB drive there? This would give you effective NAS functionality (assuming your Liv Zen 'sees' it).

In terms of connecting to your router with ethernet consider wiring directly. I find using flat cable such as this is (https://www.amazon.co.uk/gp/product/B07DDGXZWD/ref=ppx_yo_dt_b_asin_title_o08_s01?ie=UTF8&th=1)easy and provides for a neat finish.

[Lawrence001]

It may do, my concern with doing back ups is previous devices I've had eg Olive04HD, when you run a back up the first thing it does it format the HDD and then make it a mirror of the main drive. As I want to make incremental additions to my back up that would be a disaster!

Maybe I should do some more research. And also maybe I've made it much harder than it needs to be. That's probably because I'm more comfortable with a laptop screen and a Windows style file manager than something on a Linux based system managed on my phone. But if it did a the hard work for me then it would save a lot of stress and hassle.

[struth]

pick up a copy of acronis true image. One of the older ones will be cheaper. 2020 say. it will run and do a full copy of your system, and you can set it to update any way any time.

[Edward]

pick up a copy of acronis true image. One of the older ones will be cheaper. 2020 say. it will run and do a full copy of your system, and you can set it to update any way any time.

Grant I think Acronis only works on Microsoft or Mac devices, Lawrence's streamer is Linux based.

[struth]

Grant I think Acronis only works on Microsoft or Mac devices, Lawrence's streamer is Linux based.

ah ok... disregard..lol

[Lawrence001]

Here's a link to the Vortexbox back up instructions. It says once you've done the back up you can do it incrementally. But it doesn't explain what will happen if it finds files on the back up that aren't on the Zen.

My concern is it will refuse to back up or worse still remove the files that aren't on the Zen.

https://wiki.vortexbox.org/backing_up_the_data_on_your_vortexbox

[struth]

sounds like all you need is a usb ex hdd plugged into vortebox and it has its own backup system on it. it will wipe and format the new drive then copy over.. it will take a while.

[Lawrence001]

That would work if the drive on my Zen contained all the music I ripped, but I want to build up a large drive with up to 10tb of music eventually. So this back up method would not allow me to do this, and I'll end up where I am now transferring large amounts of data between two hard drives with my laptop in the middle.

Maybe that's the only way of doing it, I didn't realise ripping CD collections and going file based was such a pain.

[struth]

according to what i thought i read, as long as the hdd is big enough i think it updates as it goes... but let edward read it. he is better at these things.
im more windows.
Ive 3 hdd's all with same films and music on them. I copy them over as i go now but initially i just plugged them into same pc.. The drives are currently on different ones but will copy over the air. a cd or even a film will copy over wifi fairly quick.
I also have a backup software(acronis) attached to smaller drives for backing up my system which it does incrementally every month. I also have a couple of flash drives with the boot details.
I use plex, jriver and roon.

[Edward]

Lawrence and I have been having a whatsapp exchange on this.

The Vortexbox (what's the connection with Liv Zen btw?) backup instruction is confusing. 1/ It says it there is an option to format the drive. Does not say what happens if one chooses not to format. Does it use free space and if so where does it place its backup files? 2/ It is not clear if the backup will create a unique file structure, but it is implied that it is simply NTFS which, of course, can be read by a Microsoft machine (as MS invented NTFS). I think Mac can read/write NTFS devices.

Anyway my suggestion is to take a spare usb drive (a flash drive for example) and experiment with that and see how one goes.

Alternatively to using the backup routine provided by Vortexbox (Liv Zen) use the Linux CP command with appropriate parameters to recursively copy subfolders etc. But surely Lawrence the Liv Zen comes with some sort of file manager?

But I recommend (as per our whatsapp messages) skipping the Liv Zen unit for ripping (mainly as it is no longer supported by Innous and it is clearly not pulling down metadata/art). Rather use more modern tools such as a desktop or laptop. Or use something like this (https://store.acronova.com/nimbie-usb-plus-bd-cd-dvd-autoloader-nb21.html). :lol:

But to your original request Lawrence for a NAS. Consider something like the Western Digital MY Cloud device.

[Lawrence001]

I'll address your questions/points below thanks Edward


what's the connection with Liv Zen btw?

Before Innuos, Nuno set up Liv and sold units that were basically the same as Vortexbox's own but in nice boxes and with slot drives, using the freeware that company offered. (I don't know any of the history of when Vortexbox became a retailer of devices and where the software came from etc. It uses LMS as the player and they added the ripping bit on top. So it's actually freeware on freeware if that makes sense)

I think there was some criticism of the prices of Liv's devices as they didn't add much more than a fancy case to the Vortexbox device which was much cheaper.

Maybe as a result of this (but pure speculation on my part) Nuno closed or rebranded Liv as Innuos, and added a layer of branded software over the Vortexbox software. I think initially this was very little extra, just a prettier interface, but over time it improved so that, for example, you didn't have to go through the rather painful way of adding a DACs using codes like "hw:0,0 -r -192000" and fiddling around with different parameters (with no idea what they mean) until the sound comes out (or not knowing if the DAC is actually working or just not compatible with the Zen).

I don't know what the latest models are like but I'm sure they're more intuitive than these early ones or they'd have gone bust by now!!

but it is implied that it is simply NTFS which, of course, can be read by a Microsoft machine (as MS invented NTFS). I think Mac can read/write NTFS devices.

I think @disarmamant told me it's formatted as Linux storage so not sure how compatible it is with Windows.

Alternatively to using the backup routine provided by Vortexbox (Liv Zen) use the Linux CP command with appropriate parameters to recursively copy subfolders etc.

I don't want to do copies of everything on the Zen, just copy the folders that have been added since last time. Not sure how easy this is with Linux CP. Also, it's a pain to get into the underlying code as the startup goes straight to Vortexbox, and you need to plug in a keyboard and monitor to control it which is a pain as it's sat in the Hifi cupboard.

But surely Lawrence the Liv Zen comes with some sort of file manager?

Not as such as using Windows on a network attached laptop is the recommended way to manage files.

Unfortunately Linux files are not quite the same as Windows files, as Windows doesn't allow file names as long as Linux (and many classical albums it picks up off free db breach the windows length in some tracks so it refuses to copy the file and you have to skip them. Also Linux is case sensitive (or Windows I forget which) so many albums show up as duplicates in file manager when they are the same file - this is how the accidental mass delete I did came about.)

Rather use more modern tools such as a desktop or laptop.

I don't want to buy a computer just for this when I have 2 Zens, and also the one thing the Zen does well is rips CDs without you having to make any selections, and my lifestyle means I have no time to sit at a screen picking the right album name or picture every 5 mins, then I've not found a device that works as well.

I do have time to spend a few hours every so often cleaning up the small number of albums that didn't get the right data, but not for confirming every album is correct or not when doing a ripping session while doing other things like working or looking after the kids

But to your original request Lawrence for a NAS. Consider something like the Western Digital MY Cloud device.

Thanks I will look into that

[Edward]

Thanks Lawrence. I hope you manage to find a solution that works for you.

[Lawrence001]

Thanks Edward just researching WE NAS's.

[Lawrence001]

I've just been looking at some NAS and Synology look good. The reviews mention about getting a fast Ethernet switch and not plugging into the ISP supplied router. What's that all about?

[Edward]

I imagine that they are referring to the fact isps use to provide routers limited to 100 MB. Typically these days routers are GB (1024*MB). It is best to have this speed as you want max speed between devices on your Lan. You definitely do not want to move data around your network at only 100MB.


Note that this does not refer to the speed of your incoming internet connection. That will be anywhere between say 0.5 MB to GB.

§

[Lawrence001]

I'll have a look at mine. I wonder if that's the reason it took several days to copy files over from my Zen, my BT home hub may have been the limiting factor, it hadn't occurred to me.

[struth]

if copying using what i think is called smb3 now it is slow; its not like plugging in 2 drives to a pc, even using usb2.

[Lawrence001]

Yes this is 2 drives over the network as you can't plug the Zen into the laptop as it's a Linux computer.

[Lawrence001]

Update, I've found out I can plug one HDD into my BT router via USB and the Zen into the Ethernet port, then copy and paste the files across the network on my laptop.

I'm going to do this as my main method of copying rips from the Zen. I've bought a second (10Tb rather than 8) HDD to back up the first one. The first one has a USB port and claims to be a hub, so I'm hoping I can plug the second one into that and backup without having to plug them both into the laptop.

I've probably saved myself about £600 for a big NAS doing it this way. Of course the laptop has to remain on throughout all this managing the copies but hopefully it's just click and forget.

[Lawrence001]

Hmm been going 11 hours, so far 4% complete. Time remaining showing as "more than 1 day.". I should think so!!

Do backups on the network always take this long?? Speed is fluctuating around the 3MB/s level. I think I've got 2.4Tb in total to back up in this batch.