Shares are filling up...

[rkillcrazy]

I've been playing with this for a couple weeks and I'm liking what Amahi has to offer. I'm wanting to move off of my Windows Home Server but this is holding me up...

The default location for shares seem to be on the system drive - /var/hda/files/movies, /var/hda/files/music, /var/hda/files/pictures, et cetera. I have several other drives set up and Greyhole is using them correctly - I think. The shares are set to use the pool and when I copy over one or two ISOs, they enter the share's default path and then Greyhole moves them to the pool. This seems to be how it's supposed to work and I'm cool up to that point.

Here's the issue... When I try to copy over a few hundred ISOs, the system drive fills up and I get an error stating there's no more room left on the share. When I jump into the HDA, I see the system drive is full but the pooled drives have plenty of room. It looks like I'm copying the files to the share faster than Greyhole can move them to the pool. In turn, this creates a bottleneck and I have to wait until Greyhole has processed existing files before I can dump more to it.

Have I done something wrong? Is this a bug? Is there a config I can tweak so Greyhole moves the files to the pool faster? I know I can manually run Greyhole but that's a bit of a pain for hundreds of files. Furthermore, I think Greyhole starts the process about 10 seconds after the files reach the share anyway so running Greyhole manually doesn't seem like it would do much good.

[gboudreau]

Indeed, running Greyhole manually won't help.

If GH can't copy the files locally faster than you can copy them over the network, it probably means it need to create multiple copies (creating 2+ copies locally can be slower than one copy over the network), or that one of the drive in the pool is very slow, and when GH decides to use that one, file operations are slower than over the network.

What you can do is move the default shares into one of the big drives that are included in the pool.

For example, if you added /var/hda/files/drives/sdb1 in the pool, you could move the /var/hda/files/movies (etc.) in there.
To do so:
- Stop Samba and Greyhole from the Amahi dashboard: uncheck the watchdog and run at boot checkboxes, and stop the services in the Setup > Apps > Servers page (only visible when Advanced options are enabled)
- From the command line, or HDA desktop, manually move the shared folders from /var/hda/files/ into the root of the big drive you want to use, /var/hda/files/drives/sdb1/ in the above example.
From the command line, that would be (as root):

Code: Select all

mv /var/hda/files/movies /var/hda/files/drives/sdb1/

- Finally, in the Amahi dashboard, edit all your shares, and change their paths to reflect their new location. Make sure the path you enter is where you moved the files, i.e. /var/hda/files/drives/sdb1/movies for the Movies share, for example.
- Restart Greyhole and Samba from the dashboard. Re-check the checkboxes you disabled, and start the services.

You might also want to make sure the drive you've chosen will always have enough free space on it for you to do your copying. To do so, you'll need to edit a value in MySQL.
Install PHPMyAdmin from the Apps page of the dashboard, and use that (username: root, password: hda) to go in the hda_production database, and browse the disk_pool_partitions table.
Find the row for your big drive, and edit it.
Change the min_free value to the number of GB you want Greyhole to keep free.
GH will do it's best to keep that much space free (it will only use it if your pool is almost full).

Hope this helps.

Good day.

[rkillcrazy]

Indeed, running Greyhole manually won't help.

If GH can't copy the files locally faster than you can copy them over the network, it probably means it need to create multiple copies (creating 2+ copies locally can be slower than one copy over the network), or that one of the drive in the pool is very slow, and when GH decides to use that one, file operations are slower than over the network.

What you can do is move the default shares into one of the big drives that are included in the pool.

For example, if you added /var/hda/files/drives/sdb1 in the pool, you could move the /var/hda/files/movies (etc.) in there.
To do so:
- Stop Samba and Greyhole from the Amahi dashboard: uncheck the watchdog and run at boot checkboxes, and stop the services in the Setup > Apps > Servers page (only visible when Advanced options are enabled)
- From the command line, or HDA desktop, manually move the shared folders from /var/hda/files/ into the root of the big drive you want to use, /var/hda/files/drives/sdb1/ in the above example.
From the command line, that would be (as root):

Code: Select all

mv /var/hda/files/movies /var/hda/files/drives/sdb1/

- Finally, in the Amahi dashboard, edit all your shares, and change their paths to reflect their new location. Make sure the path you enter is where you moved the files, i.e. /var/hda/files/drives/sdb1/movies for the Movies share, for example.
- Restart Greyhole and Samba from the dashboard. Re-check the checkboxes you disabled, and start the services.

You might also want to make sure the drive you've chosen will always have enough free space on it for you to do your copying. To do so, you'll need to edit a value in MySQL.
Install PHPMyAdmin from the Apps page of the dashboard, and use that (username: root, password: hda) to go in the hda_production database, and browse the disk_pool_partitions table.
Find the row for your big drive, and edit it.
Change the min_free value to the number of GB you want Greyhole to keep free.
GH will do it's best to keep that much space free (it will only use it if your pool is almost full).

Hope this helps.

Good day.

I'll try this when I get home tonight....

Was I correct when I said GH begins the pooling process about 10 seconds after the file hits the share? If that's correct, I'd like to think the HDDs could keep up, as they are all WD20EARS and, while they're not rolling at 10,000 RPMs, I'd like to think they can get things done faster than I can upload the files. My upload speed is about 20-MB/s - which is horrible for a 1,000-mb/s network.

[gboudreau]

Was I correct when I said GH begins the pooling process about 10 seconds after the file hits the share?

Almost. GH sleeps for 10 seconds when it has nothing to do. So it should take between 0 and 10 seconds for the files to start getting moved.

Was I correct when I said GH begins the pooling process about 10 seconds after the file hits the share? If that's correct, I'd like to think the HDDs could keep up, as they are all WD20EARS and, while they're not rolling at 10,000 RPMs, I'd like to think they can get things done faster than I can upload the files. My upload speed is about 20-MB/s - which is horrible for a 1,000-mb/s network.

I/O priority is given to other processes than GH (GH daemon I/O priority is set to 'idle'). So when processes are 'battling' for I/O resources, all other processes will always have the upper hand. BTW 'idle' priority doesn't mean it will only work when disks are idle, contrary to what one might think. But it's the lowest possible priority for a process on I/O resources.
This was done in order to prevent Greyhole from slowing down other processes. You'd probably wouldn't be happy to not be able to stream a movie while GH works because it takes all the resources.
That being said, I didn't experiment much with 'ionice' (what I use to set the GH daemon I/O priority, in the init script). You could, if you want to. I think there are multiple other values than 'idle' you could play with, and see if you prefer other ones (man ionice).

You can calculate how fast GH is working by looking at the /var/log/greyhole.log file:

Code: Select all

grep -A 1 "Copying file to\|File changed\|File created" /var/log/greyhole.log | grep -v "Will use source file" | grep -v "Loading "

Look for a big file in there, and look at the following 2 lines to know how long it took to copy (GH uses rsync to copy btw).
Here's an example from my log:

--
Jun 1 08:50:12 6 write: File changed: CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf - 1.82GB
--
Jun 1 08:50:12 7 write: Copying file to /mnt/hdd0/gh/CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf
Jun 1 08:51:09 7 write: Saving 2 tombstones for CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf
--

So it took 57 seconds (08:51:09 minus 08:50:12) to copy 1.82GB = 32.7MB/s.

[rkillcrazy]

Was I correct when I said GH begins the pooling process about 10 seconds after the file hits the share?

Almost. GH sleeps for 10 seconds when it has nothing to do. So it should take between 0 and 10 seconds for the files to start getting moved.

Was I correct when I said GH begins the pooling process about 10 seconds after the file hits the share? If that's correct, I'd like to think the HDDs could keep up, as they are all WD20EARS and, while they're not rolling at 10,000 RPMs, I'd like to think they can get things done faster than I can upload the files. My upload speed is about 20-MB/s - which is horrible for a 1,000-mb/s network.

I/O priority is given to other processes than GH (GH daemon I/O priority is set to 'idle'). So when processes are 'battling' for I/O resources, all other processes will always have the upper hand. BTW 'idle' priority doesn't mean it will only work when disks are idle, contrary to what one might think. But it's the lowest possible priority for a process on I/O resources.
This was done in order to prevent Greyhole from slowing down other processes. You'd probably wouldn't be happy to not be able to stream a movie while GH works because it takes all the resources.
That being said, I didn't experiment much with 'ionice' (what I use to set the GH daemon I/O priority, in the init script). You could, if you want to. I think there are multiple other values than 'idle' you could play with, and see if you prefer other ones (man ionice).

You can calculate how fast GH is working by looking at the /var/log/greyhole.log file:

Code: Select all

grep -A 1 "Copying file to\|File changed\|File created" /var/log/greyhole.log | grep -v "Will use source file" | grep -v "Loading "

Look for a big file in there, and look at the following 2 lines to know how long it took to copy (GH uses rsync to copy btw).
Here's an example from my log:

--
Jun 1 08:50:12 6 write: File changed: CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf - 1.82GB
--
Jun 1 08:50:12 7 write: Copying file to /mnt/hdd0/gh/CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf
Jun 1 08:51:09 7 write: Saving 2 tombstones for CrashPlan/Nicole/413973645040209040/cpbf0000000000000000000/cpbdf
--

So it took 57 seconds (08:51:09 minus 08:50:12) to copy 1.82GB = 32.7MB/s.

All good info! I'll take a look at that too.

[moredruid]

test raw disk throughput:

Code: Select all

dd if=/dev/zero of=1GiBfile bs=1M count=1000

this draws zeroes from the fastest resource of your system (/dev/zero) and puts blocks of 1MB zeroes x 1000 in the file 1GiBfile

[rkillcrazy]

test raw disk throughput:

Code: Select all

dd if=/dev/zero of=1GiBfile bs=1M count=1000

this draws zeroes from the fastest resource of your system (/dev/zero) and puts blocks of 1MB zeroes x 1000 in the file 1GiBfile

Whoa, whoa, whoa.... Slow down; I'm no Linux guru.

Explain that again?

[moredruid]

LOL

what happens is this:
the front fan of your system draws fresh air into your system. as you know, air is empty. this empty air is then drawn into your processor by your CPU fan, the CPU then converts the air into digital copies of the empty air: zeroes. These zeroes are stored in a special device called /dev/zero. If you ever run out of zeroes you should hear the fans whirr louder to fill the zeroes back up quickly. you then ask /dev/zero (if=/dev/zero) to deliver 1MB (bs=1M) of zeroes and put it in a file (in this case called 1GiBfile: of=1GiBfile). It does that a thousand times (count=1000); 1000x1MB=1GiB.

capiche?

you end up with a 1GiB file that's basically just zeroes (if you ever want to zero out your disk/reset all bits to 0 you can use this mechanism). if you compress it it should compress ridiculously well (on my system gzip compression creates an archive of 994KB from the 1000MB file; bz2 compression gets it down to 839 bytes!).