Error when copying files

[siam]

Hello,

First please excuse for my bad english, I'm french.

I've begin to move my datas from a 2Tb NTFS disk to my greyhole storage pool and converted greyhole to MySQL.

When it copy files it's very slow, 3 operations/s and give me that error for each file

Code: Select all

Dec 16 08:46:44 3 write: Can't find how much free space is left on /var/hda/files/drives/drive3/gh. This partition will never be used! Dec 16 08:46:44 3 write: Please report this using the Issues tab at http://greyhole.pommepause.com. You should include the following information in your ticket: Dec 16 08:46:44 3 write: ===== Error report starts here ===== Dec 16 08:46:44 3 write: Unknown free space for partition: /var/hda/files/drives/drive3/gh Dec 16 08:46:44 3 write: df_command: df -k '/var/hda/files/drives/drive3/gh' '/var/hda/files/drives/drive4/gh' '/var/hda/files/drives/drive1/gh' '/var/hda/files/drives/drive2/gh' 2>&1 | grep '%' | grep -v "^df: .*: No such file or directory$" | awk '{print $(NF),$(NF-2)}' Dec 16 08:46:44 3 write: Result of df_command: array ( 0 => '/var/hda/files/drives/drive3 907011908', 1 => '/var/hda/files/drives/drive4 1733738132', 2 => '/var/hda/files/drives/drive1 911862016', 3 => '/var/hda/files/drives/drive2 907869140', ) Dec 16 08:46:44 3 write: Result of df -k: array ( 0 => 'Sys. de fich. 1K-blocs Occupé Disponible Capacité Monté sur', 1 => '/dev/mapper/vg_siam-lv_root', 2 => ' 303395200 3922080 284061512 2% /', 3 => 'tmpfs 1028712 284 1028428 1% /dev/shm', 4 => '/dev/sda1 198337 23959 164138 13% /boot', 5 => '/dev/sdb1 961432904 732848 911862016 1% /var/hda/files/drives/drive1', 6 => '/dev/sde1 961432572 4725408 907869140 1% /var/hda/files/drives/drive2', 7 => '/dev/sdf1 961432572 5582640 907011908 1% /var/hda/files/drives/drive3', 8 => '/dev/sdc1 1922859824 91446020 1733738132 6% /var/hda/files/drives/drive4', 9 => '/dev/sdd1 1953512444 1611735604 341776840 83% /media/Saves', ) Dec 16 08:46:44 3 write: ===== Error report ends here ===== Dec 16 08:46:44 7 write: Drives with available space: /var/hda/files/drives/drive4/gh (1.60TB avail) - /var/hda/files/drives/drive1/gh (860GB avail) - /var/hda/files/drives/drive2/gh (856GB avail)

uname -r; rpm -q samba hda-greyhole

Code: Select all

2.6.32.26-175.fc12.x86_64 samba-3.4.9-60.fc12.x86_64 hda-greyhole-0.7.5-1.x86_64

yum -y install fpaste; fpaste /etc/samba/smb.conf; fpaste /etc/greyhole.conf

Code: Select all

Modules complémentaires chargés : fastestmirror, presto, refresh-packagekit Loading mirror speeds from cached hostfile * fedora: fr2.rpmfind.net * updates: fr2.rpmfind.net Configuration du processus d'installation Le paquet fpaste-0.3.4-1.fc12.noarch est déjà installé dans sa dernière version Rien à faire Uploading (5.0K)... http://fpaste.org/j8ps/ Uploading (1.5K)... http://fpaste.org/Jvhy/

mount; fdisk -l; df -h; greyhole --stats

Code: Select all

/dev/mapper/vg_siam-lv_root on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/sdb1 on /var/hda/files/drives/drive1 type ext4 (rw) /dev/sde1 on /var/hda/files/drives/drive2 type ext4 (rw) /dev/sdf1 on /var/hda/files/drives/drive3 type ext4 (rw) /dev/sdc1 on /var/hda/files/drives/drive4 type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) gvfs-fuse-daemon on /home/admin/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=admin) /dev/sdd1 on /media/Saves type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096,default_permissions) Disque /dev/sda: 320.1 Go, 320072933376 octets 255 têtes, 63 secteurs/piste, 38913 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x963bda34 Périphérique Amorce Début Fin Blocs Id Système /dev/sda1 * 1 26 204800 83 Linux La partition 1 ne se termine pas sur une frontière de cylindre. /dev/sda2 26 38913 312363841 8e Linux LVM Disque /dev/sdb: 1000.2 Go, 1000204886016 octets 255 têtes, 63 secteurs/piste, 121601 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x000d19f7 Périphérique Amorce Début Fin Blocs Id Système /dev/sdb1 1 121602 976760832 83 Linux Disque /dev/sdc: 2000.4 Go, 2000398934016 octets 255 têtes, 63 secteurs/piste, 243201 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x1c0edce8 Périphérique Amorce Début Fin Blocs Id Système /dev/sdc1 1 243202 1953513472 83 Linux Disque /dev/sdd: 2000.4 Go, 2000398934016 octets 255 têtes, 63 secteurs/piste, 243201 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x7011c0ec Périphérique Amorce Début Fin Blocs Id Système /dev/sdd1 1 243202 1953512448 7 HPFS/NTFS Disque /dev/sde: 1000.2 Go, 1000203804160 octets 255 têtes, 63 secteurs/piste, 121601 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x000c8700 Périphérique Amorce Début Fin Blocs Id Système /dev/sde1 1 121602 976760503+ 83 Linux Disque /dev/sdf: 1000.2 Go, 1000203804160 octets 255 têtes, 63 secteurs/piste, 121601 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x00028d17 Périphérique Amorce Début Fin Blocs Id Système /dev/sdf1 1 121602 976760503+ 83 Linux Disque /dev/dm-0: 315.6 Go, 315629764608 octets 255 têtes, 63 secteurs/piste, 38373 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x00000000 Le disque /dev/dm-0 ne contient pas une table de partition valide Disque /dev/dm-1: 4227 Mo, 4227858432 octets 255 têtes, 63 secteurs/piste, 514 cylindres Unités = cylindres de 16065 * 512 = 8225280 octets Identifiant de disque : 0x00000000 Le disque /dev/dm-1 ne contient pas une table de partition valide Sys. de fich. Tail. Occ. Disp. %Occ. Monté sur /dev/mapper/vg_siam-lv_root 290G 3,9G 271G 2% / tmpfs 1005M 284K 1005M 1% /dev/shm /dev/sda1 194M 24M 161M 13% /boot /dev/sdb1 917G 811M 870G 1% /var/hda/files/drives/drive1 /dev/sde1 917G 4,6G 866G 1% /var/hda/files/drives/drive2 /dev/sdf1 917G 5,4G 865G 1% /var/hda/files/drives/drive3 /dev/sdc1 1,8T 106G 1,6T 7% /var/hda/files/drives/drive4 /dev/sdd1 1,9T 1,5T 345G 82% /media/Saves Greyhole Statistics =================== Storage Pool Total - Used = Free + Attic = Possible /var/hda/files/drives/drive3/gh: N/A - N/A = N/A + N/A = N/A /var/hda/files/drives/drive4/gh: 1834G - 106G = 1635G + 0G = 1635G /var/hda/files/drives/drive1/gh: 917G - 1G = 870G + 0G = 870G /var/hda/files/drives/drive2/gh: 917G - 5G = 866G + 0G = 866G

mysql -u root -phda -e "select * from disk_pool_partitions" hda_production

Code: Select all

+----+------------------------------+--------------+---------------------+---------------------+ | id | path | minimum_free | created_at | updated_at | +----+------------------------------+--------------+---------------------+---------------------+ | 3 | /var/hda/files/drives/drive3 | 10 | 2010-12-15 13:22:05 | 2010-12-15 13:22:05 | | 4 | /var/hda/files/drives/drive4 | 10 | 2010-12-15 13:35:52 | 2010-12-15 13:35:52 | | 5 | /var/hda/files/drives/drive1 | 10 | 2010-12-15 21:43:05 | 2010-12-15 21:43:05 | | 6 | /var/hda/files/drives/drive2 | 10 | 2010-12-15 21:48:38 | 2010-12-15 21:48:38 | +----+------------------------------+--------------+---------------------+---------------------+

mysql -u root -phda -e "select concat(path, '/gh') from disk_pool_partitions" hda_production | grep -v 'concat(' | xargs ls -la | fpaste:
http://fpaste.org/GCF6/

greyhole --view-queue

Code: Select all

Greyhole Work Queue Statistics ============================== This table gives you the number of pending operations queued for the Greyhole daemon, per share. Write Delete Rename Data 0 0 0 Old Linux 516201 0 0 Photos 09 2010 235 0 0 Photos Corse 2010 1596 0 0 Tri Old Linux 2407 0 0 Videos 545 0 0 ================= Total 520984+ 0 + 0 = 520984

[gboudreau]

The problem comes from the french localization of your Amahi server.
Greyhole expects one more line in the result of that big df -k command, and it isn't there in french.
I'll make the necessary change to fix that in Greyhole, and will provide you commands to patch your Greyhole until an official fix is released though the auto-updates.

[gboudreau]

Code: Select all

wget http://dl.dropbox.com/u/3022105/Greyhole/greyhole-french.patch patch -p0 < greyhole-french.patch rm greyhole-french.patch service greyhole restart

[siam]

Thanks,

this work fine

But is it normal that greyhole write only 3 files per second?

the files are very small and all my HDDs are in S-ATA.

[gboudreau]

It depends on the number of extra copies you specified, and other factors.
On my system, processing 10 x 2 bytes files took 4 seconds.

For files with no extra copies, it shouldn't take long, but still, for a lot of small files, the overhead of handling tasks and metadata is not negligible , since each write is handled independently. That means spawning at least two processes to check for file locks, and then rsync the file into the pool, for each file.

Greyhole handles much better big files, so if you don't really need those files like this, I suggest you archive them (zip/rar/whatever).

[siam]

Thanks for this explanation

Those files are from save of my WHS so I will wait 3 days and when copy will end I will try to archive some.

Thank you for your fast reply