The Linux File Stack... Idea.

AutoDMC · Post by **AutoDMC** » Tue Apr 19, 2005 9:19 pm

So, the idea I have is for an old PC (I got about a million of them), and a gaggle of old hard drives (I have a gazillion of them) which are really too small individually to use.

So the idea is to take an old computer, hook it full of hard drives to the gills. The computer runs linux (of course), and emulates an over-the-usb hard drive. The linux program handles the fact that it doens't have one big, reliable hard drive, but many small, unreliable ones. It does all the "raid" style duplication of data, it puts the most used inodes on the fastest drives, it optomizes itself, etc. etc.

This came to me in a dream one night, and I was wondering if anyone had ever heard of something like it, or maybe seen a project like it.

(I know you can mount different drives to different nodes on the Linux File System, but this isn't the same thing. There, the drives are still separate drives. Here, they would all emulate a huge drive.)

keless · Post by **keless** » Tue Apr 19, 2005 9:37 pm

there is a type of RAID that allows you to emulate one file system over multiple hard drives (there are different types of RAID: 0, 1, 5, etc) just look for that.

meanwhile, its not typically done because older hard drives fail, heat up, and have miniscule space. for the effort you could spend very little money instead and get one larger, faster, more recent drive and not have to do all the work.

hybrid · Post by **hybrid** » Tue Apr 19, 2005 9:46 pm

You only have 4 IDE ports available in a typical computer, thus number of hard drives is quite restricted. Otherwise you would have to spent some money for additional controller cards which are usually not that cheap.
I guess you're better off using a NAS drive, or, as you suggested USB, take a little cheaper USB portable harddisk.

AutoDMC · Post by **AutoDMC** » Thu Apr 21, 2005 1:57 pm

Yah, I know about RAID, however, when I looked, I couldn't find one that spanned disks (just didn't find it, I guess). And anyway, I don't have a RAID controller.

I know that Motherboards have only 4 IDE ports, but I get every piece of discarded computer equipment in my town, and I have boxes of IDE controller cards. I also have boxes of in-good-repair-but-slightly-less-than-huge drives. The idea was to take all the crap I have stored in my workshop and build something useful out of it. If the "filestack" worked, I could then slowly upgrade the drives inside to bigger-and-better-and-spiffier drives, which would simply "expand" the existing drive without messing with the contents.

And if it didn't work, I would break anything worth money

Guest · Post by **Guest** » Thu Apr 21, 2005 2:12 pm

AutoDMC wrote:Yah, I know about RAID, however, when I looked, I couldn't find one that spanned disks (just didn't find it, I guess). And anyway, I don't have a RAID controller.

I know that Motherboards have only 4 IDE ports, but I get every piece of discarded computer equipment in my town, and I have boxes of IDE controller cards.

You don't need a RAID controller. Set up a software RAID and Linux will care for all that RAID style duplication

There are different solutions for such disk arrays, e.g. LVM and md. I think the LVM has been replaced in 2.6 by a more flexible solution which should be the thing you're looking for. Don't remember its name right now, but should be easy to google for.
If you have the IDE cards - and the cards are capable of using IRQs other than 14 and 15, and even better they have some DMA support then it should be fairly simple to setup such disk arrays. I read about such a system serving as a video storage.

Guest · Post by **Guest** » Fri Apr 22, 2005 2:01 pm

I'm sure you can find this information somewhere else, and far more complete at that, but:

Raid 0: striping - i.e. two disks are combined to make one big virtual disk.
if one disk fails, the virtual disk is no longer available.

Raid 1: mirroring - i.e two disks contain the exact same data. One can fail but the virtual disk remains available. Typically you can resync a replacement disk while the virtual disk is running, so any downtime is limited to the physical replace.

Raid 1+0: combination of 0 and 1

Raid 3: don't quite remember, but is either similar to 4 or requires about 10 disks.

Raid 4: uses multiple disks plus one disk (so at least 3 in total) to contain redundancy information. One disk can fail and the virtual disk is still available.

Raid 5: same as 4 except the redundancy is on a different disk for every "sector"

RAID drives emulate real drives. I.o.w. those drives can still be partitioned.

The thing to remember when using any of these multidisk solutions is that the disks containing data all have to be the same size. E.g. if you have a bunch of 8GB disks and one 5GB, the other disks will be used as if they are also 5GB). The exception is the parity disk on RAID 4.

Also a word of warning: when connecting a new disk on a cable used by the RAID solution, be VERY certain that you have the ID set correctly. You wouldn't be the first to need to restore his data from a backup due to a conflict.

firefly2442 · Post by **firefly2442** » Sat Apr 23, 2005 12:53 am

If you have a bunch of old computers sitting around you could try setting up a small cluster. Try this software.

http://openmosix.sourceforge.net