Greyhole - Missing Files From Shares

Antonis78
Posts: 18
Joined: Tue Jul 19, 2011 12:28 pm

Greyhole - Missing Files From Shares

Postby Antonis78 » Wed Mar 19, 2014 3:58 pm

Hello,

My OS drive failed on me and I reinstalled Amahi. First I installed the Amahi 7 but after I had some trouble with greyhole I installed the Amahi 6. My configuration is

1x160GB drive IDE -> OS
1x320 GB drive IDE -> LZ
2x500GB drives -> POOL
4x1TB drives -> POOL

What I did to transfer my data was, I connected one drive at a time, copied the data in the LZ, while greyhole was not running and after I activated greyhole, run fsck and waited until greyhole transfered the files from the LZ to the pool.

Now that I finished I find several files missing (I have 2 for sure but I know there are others too), from the shares (when I access them through my Windows machine) but physically the files are there in the drives. I tried greyhole -f -o (find orphaned files) but with no luck.

Any other options I could try?

1. What version of Fedora, Samba & Greyhole are you running?
2.6.35.14-106.fc14.x86_64
samba-3.5.11-79.fc14.x86_64
package amahi-greyhole is not installed

2. The content of the /etc/samba/smb.conf & /etc/greyhole.conf files (provide paste URLs):
When I run this: fpaste /etc/samba/smb.conf and fpaste /etc/greyhole.conf​ I only get this http://fpaste.org/

3. The result of the following commands:
mount

Code: Select all

/dev/sdg3 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sdg1 on /boot type ext3 (rw) /dev/sdh1 on /var/hda/files/drives/drive1 type ext4 (rw) /dev/sda1 on /var/hda/files/drives/drive2 type ext4 (rw) /dev/sdb1 on /var/hda/files/drives/drive3 type ext4 (rw) /dev/sdc1 on /var/hda/files/drives/drive4 type ext4 (rw) /dev/sdd1 on /var/hda/files/drives/drive5 type ext4 (rw) /dev/sde1 on /var/hda/files/drives/drive6 type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/sdf1 on /var/hda/files/drives/drive17 type ext4 (rw)
fdisk -l

Code: Select all

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 63 1953525167 976762552+ 83 Linux Partition 1 does not start on physical sector boundary. Disk /dev/sdb: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdf5 Device Boot Start End Blocks Id System /dev/sdb1 63 976773167 488386552+ 83 Linux Disk /dev/sdc: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdee Device Boot Start End Blocks Id System /dev/sdc1 63 976773167 488386552+ 83 Linux Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xce2f6faa Device Boot Start End Blocks Id System /dev/sdd1 63 1953525167 976762552+ 83 Linux Disk /dev/sde: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xce2f6fac Device Boot Start End Blocks Id System /dev/sde1 63 1953525167 976762552+ 83 Linux Disk /dev/sdf: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdd5 Device Boot Start End Blocks Id System /dev/sdf1 63 1953525167 976762552+ 83 Linux Disk /dev/sdg: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0009cb7f Device Boot Start End Blocks Id System /dev/sdg1 * 2048 411647 204800 83 Linux /dev/sdg2 411648 12208127 5898240 82 Linux swap / Solaris /dev/sdg3 12208128 312580095 150185984 83 Linux Disk /dev/sdh: 320.1 GB, 320072933376 bytes 255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x6e306628 Device Boot Start End Blocks Id System /dev/sdh1 63 625142447 312571192+ 83 Linux [root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sdg3 141G 1.9G 132G 2% / tmpfs 1.9G 0 1.9G 0% /dev/shm /dev/sdg1 194M 24M 160M 14% /boot /dev/sdh1 294G 893M 278G 1% /var/hda/files/drives/drive1 /dev/sda1 917G 803G 69G 93% /var/hda/files/drives/drive2 /dev/sdb1 459G 368G 68G 85% /var/hda/files/drives/drive3 /dev/sdc1 459G 368G 68G 85% /var/hda/files/drives/drive4 /dev/sdd1 917G 803G 68G 93% /var/hda/files/drives/drive5 /dev/sde1 917G 200M 871G 1% /var/hda/files/drives/drive6 /dev/sdf1 917G 560G 311G 65% /var/hda/files/drives/drive17 [root@localhost ~]# mount /dev/sdg3 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sdg1 on /boot type ext3 (rw) /dev/sdh1 on /var/hda/files/drives/drive1 type ext4 (rw) /dev/sda1 on /var/hda/files/drives/drive2 type ext4 (rw) /dev/sdb1 on /var/hda/files/drives/drive3 type ext4 (rw) /dev/sdc1 on /var/hda/files/drives/drive4 type ext4 (rw) /dev/sdd1 on /var/hda/files/drives/drive5 type ext4 (rw) /dev/sde1 on /var/hda/files/drives/drive6 type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/sdf1 on /var/hda/files/drives/drive17 type ext4 (rw) [root@localhost ~]# fdisk -l Disk /dev/sda: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 63 1953525167 976762552+ 83 Linux Partition 1 does not start on physical sector boundary. Disk /dev/sdb: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdf5 Device Boot Start End Blocks Id System /dev/sdb1 63 976773167 488386552+ 83 Linux Disk /dev/sdc: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdee Device Boot Start End Blocks Id System /dev/sdc1 63 976773167 488386552+ 83 Linux Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xce2f6faa Device Boot Start End Blocks Id System /dev/sdd1 63 1953525167 976762552+ 83 Linux Disk /dev/sde: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xce2f6fac Device Boot Start End Blocks Id System /dev/sde1 63 1953525167 976762552+ 83 Linux Disk /dev/sdf: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeb3acdd5 Device Boot Start End Blocks Id System /dev/sdf1 63 1953525167 976762552+ 83 Linux Disk /dev/sdg: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0009cb7f Device Boot Start End Blocks Id System /dev/sdg1 * 2048 411647 204800 83 Linux /dev/sdg2 411648 12208127 5898240 82 Linux swap / Solaris /dev/sdg3 12208128 312580095 150185984 83 Linux Disk /dev/sdh: 320.1 GB, 320072933376 bytes 255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x6e306628 Device Boot Start End Blocks Id System /dev/sdh1 63 625142447 312571192+ 83 Linux
df -h

Code: Select all

Filesystem Size Used Avail Use% Mounted on /dev/sdg3 141G 1.9G 132G 2% / tmpfs 1.9G 0 1.9G 0% /dev/shm /dev/sdg1 194M 24M 160M 14% /boot /dev/sdh1 294G 893M 278G 1% /var/hda/files/drives/drive1 /dev/sda1 917G 803G 69G 93% /var/hda/files/drives/drive2 /dev/sdb1 459G 368G 68G 85% /var/hda/files/drives/drive3 /dev/sdc1 459G 368G 68G 85% /var/hda/files/drives/drive4 /dev/sdd1 917G 803G 68G 93% /var/hda/files/drives/drive5 /dev/sde1 917G 200M 871G 1% /var/hda/files/drives/drive6 /dev/sdf1 917G 560G 311G 65% /var/hda/files/drives/drive17
greyhole --stats

Code: Select all

Greyhole Statistics =================== Storage Pool Total - Used = Free + Attic = Possible /var/hda/files/drives/drive2/gh: 917G - 802G = 68G + 91G = 159G /var/hda/files/drives/drive3/gh: 458G - 367G = 68G + 2G = 70G /var/hda/files/drives/drive4/gh: 458G - 367G = 68G + 1G = 69G /var/hda/files/drives/drive5/gh: 917G - 803G = 68G + 4G = 72G ========================================== Total: 2751G - 2339G = 272G + 99G = 371G
greyhole --debug 2\ Fast\ 2\ Furious\ \(2003\).mkv

Code: Select all

Debugging file operations for file named "/2 Fast 2 Furious (2003).mkv" From DB ======= [2014-03-11 22:17:47] Task ID 816838: write Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv [2014-03-14 06:12:55] Task ID 855592: write Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv [2014-03-17 12:50:12] Task ID 1061115: fsck_file Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv [2014-03-17 12:51:33] Task ID 1126190: fsck_file Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv [2014-03-17 12:51:59] Task ID 1175500: fsck_file Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv From logs ========= From filesystem =============== Landing Zone: lrwxrwxrwx 1 root root 90 Mar 17 19:34 /var/hda/files/drives/drive1/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv -> /var/hda/files/drives/drive2/gh/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv Graveyard: File copies: -rwxrw-r-- 1 admin users 8533317306 Jul 14 2009 /var/hda/files/drives/drive2/gh/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).mkv
Note: for the specific file I found a tombstone in drive2/gh_graveyard and another in drive3/gh_graveyard_backup

User avatar
bigfoot65
Project Manager
Posts: 11450
Joined: Mon May 25, 2009 4:31 pm

Re: Greyhole - Missing Files From Shares

Postby bigfoot65 » Wed Mar 19, 2014 4:15 pm

Are you running Fedora 14? We no longer support that OS.

As root user, do greyhole and report what version you are using. Chances are there is an update that fixes the issues you have now.

You will get the best results with Amahi 7. The wiki guidance for installing Greyhole and configuring it have been simplified, so should be fairly straight forward to set up.

We would be happy to help you with if it needed. Not sure how much we can do if you are using Fedora 14.
ßîgƒσστ65
Applications Manager

Antonis78
Posts: 18
Joined: Tue Jul 19, 2011 12:28 pm

Re: Greyhole - Missing Files From Shares

Postby Antonis78 » Wed Mar 19, 2014 11:40 pm

Hello,

I am using Greyhole 0.9.2. Should I update?

I installed the Amahi 7 and after following the wiki for greyhole, it just would not work. So I decided to go back to Amahi 6 and wait until the greyhole app is available.

If I can't sort this out and I decide to go to Amahi 7 again, how will I transfer my data? It took me 7 days to finish and I cannot do that again!

Thanks

User avatar
bigfoot65
Project Manager
Posts: 11450
Joined: Mon May 25, 2009 4:31 pm

Re: Greyhole - Missing Files From Shares

Postby bigfoot65 » Thu Mar 20, 2014 5:20 am

I would recommend you upgrade Greyhole.

If you install Amahi 7, we can assist with any problems you might have with getting Greyhole working. Took me 10 minutes to get it working with 3 drives.

Personally, I would not spend a lot of time on Amahi 6 as support is limited to maintenance fixes which typically happen. No new apps or features will be added. While Greyhole is nice to have, the same type of file redundancy can be done without it.

If Greyhole setup was your only reason for not using Amahi 7, might not have been the best choice. Just my opinion of course :)

Migrating the Greyhole drives to Amahi 7 is not too difficult. It's just a matter of duplicating the shares and info from /etc/greyhole.conf file to the new install. There is wiki guidance for doing this here.

https://wiki.amahi.org/index.php/Reconn ... orage_pool

When you add the drives and tell Greyhole there are there, it will start putting symlinks in the shares. It works fine as long as you duplicate the share and storage pool setup as you have now.
ßîgƒσστ65
Applications Manager

Antonis78
Posts: 18
Joined: Tue Jul 19, 2011 12:28 pm

Re: Greyhole - Missing Files From Shares

Postby Antonis78 » Thu Mar 20, 2014 6:02 am

I would recommend you upgrade Greyhole.
Could you point me to guide about this?
If you install Amahi 7, we can assist with any problems you might have with getting Greyhole working. Took me 10 minutes to get it working with 3 drives.

Personally, I would not spend a lot of time on Amahi 6 as support is limited to maintenance fixes which typically happen. No new apps or features will be added. While Greyhole is nice to have, the same type of file redundancy can be done without it.
My intention is to wait until the greyhole is integrated in the web interface, like it is in the Amahi 6 and then make the jump.
If Greyhole setup was your only reason for not using Amahi 7, might not have been the best choice. Just my opinion of course :)
You mean that I did not make right choice by abandoning Amahi 7 due to Greyhole? I will agree with you, but I needed my machine up and running ASAP and after I lost so much time trying to make Greyhole work, I just gave up and went back to something that I knew would work.
Migrating the Greyhole drives to Amahi 7 is not too difficult. It's just a matter of duplicating the shares and info from /etc/greyhole.conf file to the new install. There is wiki guidance for doing this here.

https://wiki.amahi.org/index.php/Reconn ... orage_pool

When you add the drives and tell Greyhole there are there, it will start putting symlinks in the shares. It works fine as long as you duplicate the share and storage pool setup as you have now.
If I do this now, with the files missing, wouldn't just transfer the problem to the new installation as well? I could try Amahi 7 again over the weekend, but I think that I should solve the problem now and then try to upgrade.

User avatar
bigfoot65
Project Manager
Posts: 11450
Joined: Mon May 25, 2009 4:31 pm

Re: Greyhole - Missing Files From Shares

Postby bigfoot65 » Thu Mar 20, 2014 7:05 am

Agreed. Fix one problem at a time.

The guidance is in the wiki.
https://wiki.amahi.org/index.php/Greyhole_updating
ßîgƒσστ65
Applications Manager

Antonis78
Posts: 18
Joined: Tue Jul 19, 2011 12:28 pm

Re: Greyhole - Missing Files From Shares

Postby Antonis78 » Thu Mar 20, 2014 7:24 am

Ok,

at the wiki I see this
Fedora
Amahi releases new versions of Greyhole (GH) after they've been tested. Official Greyhole for Amahi 7 on Fedora 19 is 0.9.44
The ONLY officially supported Greyhole package for Amahi 7 is available from the Amahi repository. The Greyhole web site no longer provides updates for Amahi.
So, I should upgrade to the 0.9.44 version (or whatever is the latest version in Amahi repository) or should I get the latest version from the Greyhole website?

How can I upgrade from the Amahi repository?

yum install hda-greyhole gives me this

Code: Select all

Loaded plugins: fastestmirror, langpacks, presto, refresh-packagekit Adding en_US to language list Loading mirror speeds from cached hostfile * fedora: mirror.fraunhofer.de * updates: mirror.fraunhofer.de Setting up Install Process Package hda-greyhole-0.9.2-1.x86_64 already installed and latest version Nothing to do
Thank you for your time, by the way :)

User avatar
bigfoot65
Project Manager
Posts: 11450
Joined: Mon May 25, 2009 4:31 pm

Re: Greyhole - Missing Files From Shares

Postby bigfoot65 » Thu Mar 20, 2014 7:42 am

Note that the statement says fort Fedora 19. It also provides guidance for Ubuntu. If you are using Fedora 14, you will have to go to the Greyhole web site and download hda-greyhole RPM from there.

We no longer support anything with Fedora 14. It's been at end of life for years and a potential security risk to use.
ßîgƒσστ65
Applications Manager

Antonis78
Posts: 18
Joined: Tue Jul 19, 2011 12:28 pm

Re: Greyhole - Missing Files From Shares

Postby Antonis78 » Thu Mar 20, 2014 8:41 am

I upgraded Greyhole to version 0.9.9 and I run a fsck -o. I am still waiting to end so I will post the results.

In case that it doesn't fix the problem, if I delete all the tombstones and force Greyhole to recreate them will that help? Is there an easy way to do this? Of course with over 3TB of data I imagine that the fsck will take a long time but I have no problem if this means that my files will be correct.

If I fix this issue I think that I will give Amahi 7 another try over the weekend.

EDIT: still no files! I deleted the tombstone files for the file '2 Fast 2 Furious (2003).mkv', in drive2/gh/.gh_graveyard and drive3/gh/.gh_graveyard_backup and run fsck again with find orphaned files. That did not work either. I can see the file in drive2/gh/Movies/ but not when I access it through samba.

I think I will install Amahi 7 and transfer everything over the network and hope for the best.

User avatar
bigfoot65
Project Manager
Posts: 11450
Joined: Mon May 25, 2009 4:31 pm

Re: Greyhole - Missing Files From Shares

Postby bigfoot65 » Thu Mar 20, 2014 1:55 pm

I upgraded Greyhole to version 0.9.9 and I run a fsck -o. I am still waiting to end so I will post the results.
The latest version is 0.9.41 which should correct the problem.

I agree Amahi 7 would be best, but if you want to resolve the issue first then get hda-greyhole version .0.9.41 and try it.

http://www.greyhole.net/releases/rpm/x8 ... x86_64.rpm
or
http://www.greyhole.net/releases/rpm/i3 ... 1.i386.rpm
depending on your architecture.
ßîgƒσστ65
Applications Manager

Who is online

Users browsing this forum: No registered users and 2 guests