new file creation went wrong

[NeverSimple]

Hello,

(Sorry for the long post....)

yesterday evening I created two new documents, a Powerpoint (.pptx) file and a Word (.docx) file on a HDA share included in the Greyhole pool. Worked on these for a couple of hours, saved, and shutdown my computer. This morning i planned on working some more on these files. But something 'horrible' has happened: The Powerpoint document isn't there anymore and the Word document has a size of 0 bytes. After looking I found (lucky me!) the correct copy of the Word document in the 'attic' of a pool drive. This looks exactly like I saved it for the last time. The Powerpoint doucument seems to have renamed to 3F8B3536.TMP after I renamed that to 'something.pptx' I could open it and again it seems to look like I last saved it.

Looking through the saved 'greyhole.log' (attached) I see that from the time of creation until I closed Word and Powerpoint, for every 10 seconds for 2 and a half hours, greyhole (or samba?) is unable to work on these files, because they are locked by another process. That's scary by itself. I pressed 'save' a number of times. I assume this is a Windows 'fault' but can't a file really not be saved when it is open? What if my program or computer crashes?

(I had to split the greyhole.log file attached, because of size limitations)

Looking at the end of the log it looks like the order of the tasks is wrong (summary, powerpoint):

Code: Select all

Dec 9 23:55:34 7 rename: Now working on task ID 471756: rename Docs/NetwerkPresentatie/C3EFF84A.tmp -> Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/C3EFF84A.tmp -> /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Loading tombstones for Docs/NetwerkPresentatie/C3EFF84A.tmp... Got 0 tombstones. Dec 9 23:55:34 7 rename: Loading tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx... Got 0 tombstones. Dec 9 23:55:34 7 rename: Loading tombstones for Docs/NetwerkPresentatie/C3EFF84A.tmp... Got 0 tombstones. Dec 9 23:55:34 6 rename: File created: Docs/NetwerkPresentatie/AVESIS 16-9.pptx - 239KB Dec 9 23:55:34 7 rename: Loading tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx... Got 0 tombstones. Dec 9 23:55:34 7 rename: Drives with available space: /var/hda/files/drives/sdc1/gh (146GB avail) - /var/hda/files/drives/sdb1/gh (146GB avail) Dec 9 23:55:34 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Copying file to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Creating symlink in share pointing to the above file copy. Dec 9 23:55:34 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Copying file to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 7 rename: Now working on task ID 471758: rename Docs/NetwerkPresentatie/5017881E.tmp -> Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:34 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/5017881E.tmp -> /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/5017881E.tmp... Got 0 tombstones. Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/5017881E.tmp... Got 0 tombstones. Dec 9 23:55:35 6 rename: File changed: Docs/NetwerkPresentatie/AVESIS 16-9.pptx - 239KB Dec 9 23:55:35 7 rename: Will use source file: /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx... Got 2 tombstones. Dec 9 23:55:35 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: File copy at /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx is already up to date. Dec 9 23:55:35 7 rename: Copying file to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Now working on task ID 471762: rename Docs/NetwerkPresentatie/AVESIS 16-9.pptx -> Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx -> /var/hda/files/docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx... Got 2 tombstones. Dec 9 23:55:35 7 rename: Renaming copy at /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Updating symlink at /var/hda/files/docs/NetwerkPresentatie/3F8B3536.tmp to point to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Renaming copy at /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/AVESIS 16-9.pptx to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Removing tombstones for Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Removed tombstone at /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Removed tombstone at /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 rename: Removing deprecated symlink at /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Now working on task ID 471763: rename Docs/NetwerkPresentatie/E4CA29C9.tmp -> Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/E4CA29C9.tmp -> /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/E4CA29C9.tmp... Got 0 tombstones. Dec 9 23:55:35 7 rename: Loading tombstones for Docs/NetwerkPresentatie/E4CA29C9.tmp... Got 0 tombstones. Dec 9 23:55:35 7 rename: File write: /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx doesn't exist anymore. Skipping. Dec 9 23:55:35 7 unlink: Now working on task ID 471764: unlink Docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 6 unlink: File deleted: /var/hda/files/docs/NetwerkPresentatie/3F8B3536.tmp Dec 9 23:55:35 7 unlink: File still exists in landing zone; a new file replaced the one deleted here. Skipping. Dec 9 23:55:35 7 write: Now working on task ID 471765: write Docs/NetwerkPresentatie/AVESIS 16-9.pptx Dec 9 23:55:35 7 write: File write: /var/hda/files/docs/NetwerkPresentatie/AVESIS 16-9.pptx doesn't exist anymore. Skipping. Dec 9 23:55:35 7 simplify_tasks: Simplifying pending tasks.

Of course I didn't rename these files myself to something like 3F8B3536.TMP. Don't know where that comes from.

summary, word document:

Code: Select all

Dec 9 23:55:57 7 write: Now working on task ID 471786: write Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 6 write: File created: Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx - 17.6KB Dec 9 23:55:57 7 write: Loading tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx... Got 0 tombstones. Dec 9 23:55:57 7 write: Drives with available space: /var/hda/files/drives/sdc1/gh (146GB avail) - /var/hda/files/drives/sdb1/gh (146GB avail) Dec 9 23:55:57 7 write: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Copying file to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Creating symlink in share pointing to the above file copy. Dec 9 23:55:57 7 write: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Copying file to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 write: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 rename: Now working on task ID 471789: rename Docs/NetwerkPresentatie/9F751EDE.tmp -> Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/9F751EDE.tmp -> /var/hda/files/docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:57 7 rename: Loading tombstones for Docs/NetwerkPresentatie/9F751EDE.tmp... Got 0 tombstones. Dec 9 23:55:58 7 rename: Loading tombstones for Docs/NetwerkPresentatie/9F751EDE.tmp... Got 0 tombstones. Dec 9 23:55:58 6 rename: File changed: Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx - 17.6KB Dec 9 23:55:58 7 rename: Will use source file: /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Loading tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx... Got 2 tombstones. Dec 9 23:55:58 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: File copy at /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx is already up to date. Dec 9 23:55:58 7 rename: Copying file to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Now working on task ID 471790: rename Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx -> Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 6 rename: File renamed: /var/hda/files/docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx -> /var/hda/files/docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Loading tombstones for Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx... Got 2 tombstones. Dec 9 23:55:58 7 rename: Loading tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx... Got 2 tombstones. Dec 9 23:55:58 7 rename: Moved copy from /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx to attic: /var/hda/files/drives/sdb1/gh/.gh_attic/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Moved copy from /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx to attic: /var/hda/files/drives/sdc1/gh/.gh_attic/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Removing tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Removed tombstone at /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Removed tombstone at /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Renaming copy at /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Updating symlink at /var/hda/files/docs/NetwerkPresentatie/Netwerk_aantekeningen.docx to point to /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Renaming copy at /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Removing tombstones for Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx Dec 9 23:55:58 7 rename: Removed tombstone at /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx Dec 9 23:55:58 7 rename: Removed tombstone at /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx Dec 9 23:55:58 7 rename: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:55:58 7 rename: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Now working on task ID 471798: write Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 6 write: File changed: Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx - 0.00B Dec 9 23:56:19 7 write: Will use source file: /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Loading tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx... Got 2 tombstones. Dec 9 23:56:19 7 write: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: File copy at /var/hda/files/drives/sdb1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx is already up to date. Dec 9 23:56:19 7 write: Copying file to /var/hda/files/drives/sdc1/gh/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Saving 2 tombstones for Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Saving tombstones in /var/hda/files/drives/sdb1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx Dec 9 23:56:19 7 write: Saving tombstones in /var/hda/files/drives/sdc1/gh/.gh_graveyard/Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx

The taskID's are, of course, handled in the correct order but the tasks themselfs seem to be in the wrong order. In the .docx case it started with saving the correct name and size (File created: Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx - 17.6KB) but in a later task it replaced this one with the file that Word made when creating the file (File renamed: /var/hda/files/docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx -> /var/hda/files/docs/NetwerkPresentatie/Netwerk_aantekeningen.docx). For the .pptx file it seems the same, but there I've got no idea what the .TMP files are.

Got no idea what happened. Didn't notice anything strange when working on the documents. Got me pretty worried though and lost a sizeable amount of hours trying to shed some light on this (and recovering the files). It could have been worse however, thankfully Greyhole uses the 'attic'.

I don't know if more info is needed/useful. If so i'll try to provide it

Could be important: I'm using a modified Greyhole: http://forums.amahi.org/viewtopic.php?f=39&t=2118 (instead of rsync > rsync -t)


versions;
2.6.32.14-127.fc12.i686
samba-3.4.9-60.fc12.i686
hda-greyhole-0.6.28-1.i386

smb.conf
Uploading (5.5K)...
http://fpaste.org/mE9P/

greyhole.conf:
Uploading (1.3K)...
http://fpaste.org/b9c4/

Code: Select all

[root@localhost NetwerkPresentatie]# mount; fdisk -l; df -h; greyhole --stats /dev/sda1 on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sdb1 on /var/hda/files/drives/sdb1 type ext3 (rw) /dev/sdc1 on /var/hda/files/drives/sdc1 type ext3 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) Disk /dev/sda: 163.9 GB, 163927556096 bytes 255 heads, 63 sectors/track, 19929 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x81320a85 Device Boot Start End Blocks Id System /dev/sda1 * 1 19672 158015277 83 Linux /dev/sda2 19672 19929 2064384 82 Linux swap / Solaris Disk /dev/sdb: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x000ddb5e Device Boot Start End Blocks Id System /dev/sdb1 1 60801 488384001 83 Linux Disk /dev/sdc: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00032237 Device Boot Start End Blocks Id System /dev/sdc1 1 60801 488384001 83 Linux Filesystem Size Used Avail Use% Mounted on /dev/sda1 149G 13G 129G 9% / tmpfs 502M 0 502M 0% /dev/shm /dev/sdb1 459G 279G 157G 65% /var/hda/files/drives/sdb1 /dev/sdc1 459G 279G 157G 65% /var/hda/files/drives/sdc1 Greyhole Statistics =================== Storage Pool Total - Used = Free + Attic = Possible /var/hda/files/drives/sdb1/gh: 458G - 279G = 156G + 0G = 157G /var/hda/files/drives/sdc1/gh: 458G - 279G = 156G + 0G = 157G

Code: Select all

[root@localhost NetwerkPresentatie]# mysql -u root -phda -e "select * from disk_pool_partitions" hda_production +----+----------------------------+--------------+---------------------+---------------------+ | id | path | minimum_free | created_at | updated_at | +----+----------------------------+--------------+---------------------+---------------------+ | 1 | /var/hda/files/drives/sdb1 | 10 | 2010-08-19 18:55:47 | 2010-08-19 18:55:47 | | 2 | /var/hda/files/drives/sdc1 | 10 | 2010-08-19 18:55:48 | 2010-08-19 18:55:48 | +----+----------------------------+--------------+---------------------+---------------------+

Code: Select all

[root@localhost NetwerkPresentatie]# mysql -u root -phda -e "select concat(path, '/gh') from disk_pool_partitions" hda_production | grep -v 'concat(' | xargs ls -la | fpaste Uploading (2.0K)... http://fpaste.org/NdaV/

Code: Select all

[root@localhost NetwerkPresentatie]# greyhole --view-queue Greyhole Work Queue Statistics ============================== This table gives you the number of pending operations queued for the Greyhole daemon, per share. Write Delete Rename Books 0 0 0 Docs 0 0 0 HAT 0 0 0 HomerBackup 0 0 0 Movies 0 0 0 Music 0 0 0 Pictures 0 0 0 SyncTest 0 0 0 =========== Total 0 + 0 + 0 = 0

Richard

[gboudreau]

Locking:
This is Greyhole telling you it won't move that file into the pool, or create extra copies (if you asked for some), until that other process release that file.
It's not that it can't, it's that it's not safe to do so.
Word could be writing to the file while Greyhole is copying it, which would result in an corrupted copy of the file.
And so, Greyhole waits until the other process release the file to work on it.
Even so, your file is still saved on your Samba share this whole time.

I'll have a look at your log.
My guess is that the TMP file comes from PowerPoint auto-save feature.

There is probably a bug in Greyhole that caused the operations to be processed in the wrong order, probably something to do with the file being locked for so long.
But I don't see how any of that would make the files disappear from the share.
More on that later.

[NeverSimple]

Locking:
This is Greyhole telling you it won't move that file into the pool, or create extra copies (if you asked for some), until that other process release that file.
It's not that it can't, it's that it's not safe to do so.
Word could be writing to the file while Greyhole is copying it, which would result in an corrupted copy of the file.
And so, Greyhole waits until the other process release the file to work on it.
Even so, your file is still saved on your Samba share this whole time.

Ok, thanks for clearing that up.

I'll have a look at your log.
My guess is that the TMP file comes from PowerPoint auto-save feature.

There is probably a bug in Greyhole that caused the operations to be processed in the wrong order, probably something to do with the file being locked for so long.
But I don't see how any of that would make the files disappear from the share.
More on that later.

They didn't disappear, maybe I was unclear about that. It's about two files: a powerpoint file and a word document
- The Powerpoint file got renamed to 3F8B3536.tmp, so at first it looked like it disappeared.
- The Word file got replaced with a copy containing 0 bytes.

So they both were present.

Thanks for looking into it.


Richard

[gboudreau]

Here's what PowerPoint did (very fast) that confused Greyhole:

- write E4CA29C9.tmp
- rename AVESIS 16-9.pptx -> 3F8B3536.tmp
- rename E4CA29C9.tmp -> AVESIS 16-9.pptx
- delete 3F8B3536.tmp

I'll create a test case to reproduce, and find a safe way to fix GH to handle those situations correctly.
I'll have a look at the .docx problem next.

[gboudreau]

I committed a fix for the above; it will be included in the next version of GH.

The problem with the docx was lock-related.
What happened is the lock was released when GH was working on those two tasks:
- rename Docs/NetwerkPresentatie/Nieuw - Microsoft Word-document.docx -> Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx
- write Docs/NetwerkPresentatie/Netwerk_aantekeningen.docx

The lock was still there when GH tried to execute the rename, so it postponed that task, then the lock was released, then GH executed the write, which worked. Finally, GH executed the postponed rename tasks, and boom, 0 bytes file.

To fix, I'll make sure locks are remembered long enough to insure tasks are always executed in the proper order.
(i.e. always postpone all tasks related to a file X, when it's found to be locked during a task execution.)

[NeverSimple]

Hello Guillaume,

Thanks for looking into that so thoroughly. It's even better that you found real causes and are able to develop a fix for that. Things like this tend to get me very worried, because the reason I'm using GH is NOT to loose information ...

I'm convinced that you know a lot (and I mean a LOT) more about your software than I do, but I'm still curious:

If I understand the '.docx' case correct, in the new GH version, if a file is locked all tasks concerning that file are queued and when the lock is released, the queue get's emptied starting with the oldest task in the queue. What I don't understand is the reason for the queue. You said that GH waits until the process working on a file releases it, but during the lock Samba still saves the latest changes. So, when the lock is released, the file on the share (landingzone) is the latest, actual file you want. Why is it important to remember what happened to it in the past? (ie. renames, size change). The file GH wants to work on is on the share with the correct name, date/time and size. Or isn't it? I'm not even sure how GH identifies a locked file it has to work on during renames. Maybe i-node numbers? Anyway, this is completely out of my league.

Good luck with the fix,


Richard

[gboudreau]

Re-playing the file operations in the same order as they happened on the shares is one way to do what GH does.
There are indeed other options, like the one you mentioned, where GH tries to only replicate the final state of the files on your shares.
But this is more complicated than it looks, and has some downsides versus the current algorithm.
I'm still thinking about what's the best way to do it, and haven't come up with anything better that the current algorithm. I might, at some point.