Friday, 1 February 2008

File storage.

I've been somewhat lax with my attempts to post regularly on here, and still have a number of posts to do at some point, but I've been tired and busy and am now going to be away all of this weekend, so if my silence was a bother, it will continue to be, and if it was a relief, enjoy it while it lasts.

For now, I thought I'd put up an interesting and irritating problem I've run into with my computer. I've reached the point where the amount of downloaded TV I have can no longer fit on a single 500GB hard drive. This is, naturally, a bit annoying, as I'm going to have to move stuff around and find somewhere else to start storing it. The more interesting problem is that I want to keep the downloads in a form in which I can find a specific episode of a specific program. At the moment, this means I've got them in folders according to program and subfolders according to series, so Lost > Season 1 > Episode 3.

The easiest way I can think to try and maintain this when starting to move some of the videos to a new hard drive is to move the first half of the folders (alphabetically) onto the new drive, and continue to add files to the appropriate drive, depending on the name of the series. There is no problem with this, apart from that it's not a permanent solution, and as I download more, I will end up having to constantly shift round files from disk to disk.

The alternative is to find some sort of indexing program that would act in the way that iTunes does with ripped/downloaded music and add another layer of abstraction to the file system. This would allow me to dump the files wherever I liked without having to worry about keeping them in a human-readable order, since I could always access them through the indexing layer.

The second option appeals to me in a logical way, since it seems like the most efficient way to allow my collection to grow, but it bugs me that if I do for whatever reason need to find a file (if my indexing program/system breaks or crashes) it'll be far more difficult than it should.

So, I'd be interested to hear anyone's thoughts on this - which system should I go with, and, if the second one, can you recommend an indexing program?


Andy J. Wotherspoon said...

I'm mildly confused by what you mean by an indexing layer etc.

Also could you get another drive and raid it with your existing one to create an even larger disk, meaning you could then just put them in a single folder which would span them across the two drives. I'm not sure if that is even possible as I have no experience of raid drives but got the idea this is what could be done.

TheTelf said...

An indexing layer would be a program or an interface that stores pointers to the file locations. Instead of having to look through the filesystem to play a video, I can find it using a search or other structure within the list of pointers.

E.g in iTunes, when you download podcasts, they aren't stored in distinct folders or labelled with human-readable names, because you find them using iTunes itself, which knows where they are, and can sort them for you any way you want.

It mirrors the way the actual filesystem works too, in the sense that data is stored randomly on the hard drive, but you don't need to know the exact memory address of the data you want, because Windows (or whatever OS) stores and organises the pointers for you.

RAIDing (or I think there are other, software, ways of doing it too) is an option, but I don't think you can do it infititely - I think there's still a limit on the number of disks you can join.

If I'm still not making sense, then say so :D

joebloggs said...

Though my first thought was also the RAID option, onthinking about it, this is basically equivalent to just buying a bigger hard drive!

I'm not sure whether or not RAID is infinite, I doubt it, but then you're not going to find an infinite amount of hard drive space anyway!

I think any indexing layer would be fine, the best option though is to go for something that adds meta tags to the original files, because other wise if you lose the software you lose all your indexing work.