Confused by partitioning

[xerai]

Dear all, I hope someone can take pity on me and give me some help/advice as although I am able to install Amahi. I have very limited experience of Linux and I am confused about how Amahi works with Greyhole and if I am partitioning and preparing my drives correctly.

Firstly I should say that all I want to get from my server (at this point) is store and share files over my LAN with Windows machines (desktops and a HTPC). Despite my modest demands I find that I have wiped and installed Amahi three times whilst attempting to set it up ‘correctly’. Each time I change how I partition the drives, but I’m still not sure I’m doing it ‘right’.

My Hardware:
Atom 330 CPU, 2GB RAM, sda 320GB (2.5” sata), sdb 1.5T (3.5” sata), sdc 1.5T (3.5” sata) – room for 2 more drives to add sometime in the future
NB: Im using the Fedora14 64bit install DVD

My main confusion is about the landing zone, exactly what it does and how to set it up. I’ve been following
this youtube guide and partitioned my drives as follows during the install, but I think its incorrect?

Sda1 300GB (set as / during install)
Sda2 ~6GB swap
Sdb1 Max (set as /var/hda/files during install)
Sdc1 Max

With the above when Ive finished the install I end up with a 1.5T filesystem

I have read the greyhole wiki but I’m still unclear about the landing zone. My understanding is that when I copy files to a share over my LAN the files go to ‘/var/hda/files’ initially then get moved to the greyhole area later – is that correct? As a result you must make the ‘/var/hda/files’ large enough to hold the capacity of files you might upload. I have a large block of data ~550GB which I want to upload, but I don’t want to find I have to do half and then wait a whole day to upload the second half. What Im also unclear about is whether the ‘/var/hda/files’ contains something of the uploaded files afterwards or will eventually revert to being empty (after Greyhole has processed it). Am I making my ‘/var/hda/files’ far too big, could I get away with putting it on Sda and using all of Sdb and Sdc as the Greyhole filesystem? I just read a post whis says ideally the ‘/var/hda/files’ should be on a seperate drive, so maybe not?

In ‘Computer’ I can see a ‘1.5T TB filesystem’ which contains just a folder called lost+found. Im not sure if this is the ‘/var/hda/files’ or the Greyhole area. There is also a folder call ‘file system’ which appears to be the contents of my Sda1. Where is the Greyhole area and where is the ‘/var/hda/files’ or is one of them not accessible?

Via ‘System monitor’ I can see the three drives. Sda1 is ‘/’, Sdc1 is ‘/media/xxxxxx’ and Sdb1 is ‘/var/had/files’
And in ‘Resources’ I can see the ‘swap’ is setup and working.

Im thinking about repartitioning again with the following layout, is this a good idea or can you suggest something else?

Sda1 50GB (set as / during install)
Sda2 ~6GB swap
Sda3 Max remaining
Sdb1 600GB (set as /var/hda/files during install)
Sdb2 Max remaining
Sdc1 Max

Or maybe

Sda1 50GB (set as / during install)
Sda2 ~6GB swap
Sda3 Max remaining (set as /var/hda/files during install) ~250GB
Sdb1 Max
Sdc1 Max

Thanks in advance for any help provided

[xerai]

Hmm, i guess the lack of replies means nobody knows of a fomulae to work out how big to set the greyhole landing zone?

I read the greyhole wiki again and it explains what the landzone does, but doesnt explain what a suitable size is. Just as a example: If I am uploading 500GB of data in one go does that mean the landing zone needs to be 500GB or can it be smaller? If it can be smaller how small can it be, if it has to be bigger how much bigger?

I dont particularly want to waste space in my system and I would rather not setup a landing zone taking a big chunk out of storage space when I could have got away with a much smaller one. Someone must know something about this technology?

[bigfoot65]

The longer the post, the less response you will get typically. This is a user based forum, so activity level varies.

Also, keep in mind set up is a personal preference, so not really a formula per say to share. No need to make it complicated.

My setup is as follows:

- 500GB hard drive for OS/Greyhole Landing zone.
- 2 1TB drives for Greyhole
- 1 2TB drive for Greyhole

All my files are on the OS drive, but the 3 other drives I have added to the pool and enabled max on shares. This puts a copy on each as a backup.

No need to dedicate 300GB to root. The OS and apps do not take up much space. I would use the 300GB for root, swap, and /var/hda/files and use the other drives for Greyhole.

If you want to know more about Greyhole, check out the wiki or the Greyhole site.

http://wiki.amahi.org/index.php/Greyhole

[sgtfoo]

TO understand the workings of this whole system it's best to set aside what you already know about windows's simpler partitioning schemes.

You may see a "Filesystem" drive in the Fedora OS, but in this case, it's just a container for all the things on the server and the partitions on the system, used by amahi are mounted as folders somewhere within. With Linux, you can 'mount' folders to encompass entire partitions, and Linux will just see it as a folder. This allows for clever usage of drives or partitions to separate data physically. I would suggest to just avoid the "filesystem" in the Fedora OS to not confuse yourself.

Amahi uses it's own unique scheme to mount partitions for use as shares for the server.
everything inside the server filesystem under /var/hda/files/ is your shares.
Please remember: don't change folders in there from your server desktop.

For another example, here's my layout, first physically, and then, how it's virtually allocated to partitions...
(Keep in mind, Linux needs at very least 2 partitions at install, the root "/" for everything, and the 'swap' partition)
Taking advantage of this, it's also a decent idea to have a "/home" partition to keep intact between installs of Fedora, that will never be formatted because it can contain any files you want to carry across installs of Fedora (such as when Amahi eventually moves to a newer release of Fedora)
Keep in mind, the only drives I kept connected during Fedora & Amahi install are the 74GB drive and one 1TB drive

* 1x 74 GB drive [sda1] for OS "/" (10GB) and [sda2] "/home" (remaining GB) and [sda3] "swap"(6GB) partitions. (Amahi will never use this for shares)

* 1x 1TB drive [sdb1] -- I decided to use this as my LANDING ZONE. I figure I'll never need more than 1TB of space for transfering thigns to the server. This is mounted during install as /var/hda/files/

* 1 x 1.5TB [sdc1] drive -- first share drive, mounted AFTER install as: /var/hda/files/drives/drive01 (and selected to use in Greyhole pool)
* 1x 1TB [sdd1] drive -- second share drive, mounted AFTER install as: /var/hda/files/drives/drive02 (and selected to use in Greyhole pool)
* 1x 2TB [sde1] drive -- third share drive, mounted AFTER install as: /var/hda/files/drives/drive03 (and selected to use in Greyhole pool)

In the end I also added the first drive to the pool, allowing for some occasionally faster transfers of files. Although it is sometimes advised to NOT add the LZ drive to the pool. In your case I would suggest not to.

As for waiting a whole day to transfer files, you don't have to because you can manually run

Code: Select all

sudo greyhole -f

in the terminal to have the pool allocated from the LZ to the pool drives right away.

As for your suggested setups.....

Sda1 50GB (set as / during install)
Sda2 ~6GB swap
Sda3 Max remaining
Sdb1 600GB (set as /var/hda/files during install)
Sdb2 Max remaining
Sdc1 Max

Or maybe

Sda1 50GB (set as / during install)
Sda2 ~6GB swap
Sda3 Max remaining (set as /var/hda/files during install) ~250GB
Sdb1 Max
Sdc1 Max

Both are quite fine, but take into account that if a LZ partitions lives with a pool drive physically, what happens when that whole drive dies? You will need to recreate both partitions. I would vote for whichever sertup has the LZ (/var/hda/files) on a seperate drive entirely, as it is the most used drive in the pool essentially.

Have fun!

[ghstryder]

And just to prove that there is no "one size fits all" solution, I have a 73Gb drive which contains /, swap and the LZ. Data lives in an external enclosure.

I was able to fill the LZ when I tested Amahi/Greyhole by doing things I would never attempt in practice. In daily use, it has never even come close to filling the LZ.

The system drive and LZ see plenty of use - those are 2 Seagate Cheetahs (15,000 rpm Enterprise Class SCSIs) running in RAID 1. The enclosure is connected via a pair of Infiniband cables.

[xerai]

Many thanks for all the replies. I agree my first post was too long, but I was confused and didnt really know what I wanted to ask

Im a bit clearer on what Im trying to ask now, and from what Im learning I think the following is correct:

Partition 1: Small root (/) 50Gb is fine
Partition 2: Small swap 4GB should be OK
Partition 3: Creating a /home partition on a seperate drive than root is also a good idea
Partition 4: Landing zone on a seperate drive
Partition 5,6 etc: Used by Greyhole
NB: Above numbering does not reflect any specific order

The thing Im still not clear about is the minimum size of the landing zone you can get away with. Is it absolutely set in stone that the landing zone size must exceed the size of your expected transfer?

If for example I had a landing zone of 200GB and I started copying 500GB across the network to the server what would happen?

I guess Greyhole doesnt start processing the files automatically so the landing zone would run out of space?
What would happen if you try to run 'sudo greyhole -f ' as the transfer is in progress would that enable you to have a smaller landing zone or wont it work like that.

I am expecting the answer to be that Greyhole cant be run in that way, but copying files across a LAN takes quite a long time and I would expect (in theory) Greyhole could keep ahead of the files coming in and so live with a small landing zone.

Sorry for all these questions, Im almost resigned to having a 600GB landing zone, but want to make sure there arent any 'tricks' that might enable me to have a smaller one. I guess I could test these things myself, but hoping someone else has already tried or knows the answers

[sgtfoo]

There's pretty much no reason to have a seperate drive for the /home folder. It's not used by Amahi. It's something that every linux install needs in order to put user files somewhere. Since this is a server, you'll seldom use the /home folder. Just leave it on the OS drive as the remaining size of the drive after the other partitions.

As for the size of your LZ, just take into account the fact that you may want a separate drive or partition for it. Then understand that you can eventually add it to the pool to use the space, but it will apparently gradually shrink in size as it is used.

Think about the largest amount of data you'd copy/move to your server in a day. Account for maybe 15% more GB to be safe and then call that the size of your LZ. Be realistic. You'll also be limited by how fast things CAN move to the LZ. Many of us cap out at 50MB/s. It's a necessary consideration if you're using greyhole.
Personally, I find it worth the purchase of a 1TB HDD to make my LZ 1TB big. It's the most commonly used drive aside from the OS drive.

[prodriguezii]

Sorry to revive and old thread but...

I recently did a fresh Ubuntu Server 12.04 install with Amahi. Below is my partitioning setup:

• sda (1TB):
  8 GB swap
  10 GB /home
  20 GB /
  Balance /var/hda/files/drives/sda
  
  sdb (1.5TB):
  Max /var/hda/files/drives/sdb
  
  sdc (1TB):
  Max /var/hda/files/drives/sdc

As you can see I failed to make a separate mount/partition for the Greyhole LZ (/var/hda/files) so it is on the root (20GB) partition.

I added the drives to the Greyhole storage pool and enabled Greyhole for all my shares. Then, in my ignorance of workings of Greyhole, I proceeded to copy 400GB of data from my Windows machine to the shares. Within 24 hours, the transfer was complete. I checked my shares and all the files seem to be there, sym-linked by Greyhole and can be accessed without any noticeable corruption.

How is this possible/why did this work?

[bigfoot65]

Seems like a Greyhole question. Recommend next time it be addressed in that area as the author will more likely reply with the best answer.

Of course if it was me, I would not worry about it Why question something that works, only worry about those that do not. But that is me.