SOLVED: Greyhole disabled; mysql tables corrupted?

[gstreet]

I would be grateful for some assistance - note that I am only a casual Linux user.

I installed Amahi 6 (on a server running desktop Ubuntu 12.04) nearly three years ago and it has worked great for me all this time with no real problems.

Recently, I ended up filling up my system (root) partition - long story that I won't go into. Although I have fixed this, it appears to have caused issues with greyhole and the mysql tables.

Code: Select all

gstreet@gstreet-MicroServer:~$ sudo greyhole --stats Can't describe settings with query: DESCRIBE settings - Error: Table './greyhole/settings' is marked as crashed and last (automatic?) repair failed

I found some commands in another thread that I tried to use to repair the table

Code: Select all

gstreet@gstreet-MicroServer:~$ mysql -u root -phda -e "repair table tasks" greyhole +----------------+--------+----------+----------+ | Table | Op | Msg_type | Msg_text | +----------------+--------+----------+----------+ | greyhole.tasks | repair | status | OK | +----------------+--------+----------+----------+

I also tried this command which I left running overnight but my remote session timed out and I don't know if it was successful or the response.

Code: Select all

mysql -u root -phda -e "repair table tasks_completed" greyhole

Finally, I tried the following and received an error:

Code: Select all

gstreet@gstreet-MicroServer:~$ mysql -u root -phda -e "repair table settings" greyhole +-------------------+--------+----------+------------------------------------------------------+ | Table | Op | Msg_type | Msg_text | +-------------------+--------+----------+------------------------------------------------------+ | greyhole.settings | repair | error | Can't create new tempfile: './greyhole/settings.TMD' | | greyhole.settings | repair | status | Operation failed | +-------------------+--------+----------+------------------------------------------------------+

I'm not sure what to do next.
Grateful for any guidance or instruction you might be able to give me. Please be explicit given my novice Linux skills. Thanks heaps in advance!

[bigfoot65]

have you checked Greyhole troubleshooting wiki guidance?

[gstreet]

I have, thanks. But I will run through it again tonight when I get off work and post results / feedback.

[bigfoot65]

Sounds good.

It's probably something simple that needs fixed.

[gstreet]

Okay; I worked through the Greyhole Troubleshooting Wiki again and was not successful.

So I will walk though it here and post the responses. Sorry in advance for the long post. From the top of the GH Troubleshooting Wiki:

1. What version of OS, Samba & Greyhole are you running?

Code: Select all

gstreet@gstreet-MicroServer:~$ uname -r; dpkg -s samba greyhole 3.2.0-85-generic Package: samba Status: install ok installed Priority: optional Section: net Installed-Size: 22864 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Architecture: amd64 Version: 2:3.6.3-2ubuntu2.12 Replaces: samba-common (<= 2.0.5a-2) Depends: samba-common (= 2:3.6.3-2ubuntu2.12), libwbclient0 (= 2:3.6.3-2ubuntu2.12), libacl1 (>= 2.2.51-5), libattr1 (>= 1:2.4.46-5), libc6 (>= 2.15), libcap2 (>= 2.10), libcomerr2 (>= 1.01), libcups2 (>= 1.4.0), libgssapi-krb5-2 (>= 1.10+dfsg~), libk5crypto3 (>= 1.6.dfsg.2), libkrb5-3 (>= 1.10+dfsg~), libldap-2.4-2 (>= 2.4.7), libpam0g (>= 0.99.7.1), libpopt0 (>= 1.14), libtalloc2 (>= 2.0.4~git20101213), libtdb1 (>= 1.2.7+git20101214), zlib1g (>= 1:1.1.4), debconf (>= 0.5) | debconf-2.0, upstart-job, libpam-runtime (>= 1.0.1-11), libpam-modules, lsb-base (>= 3.2-13), procps, update-inetd, adduser, samba-common-bin Recommends: logrotate, tdb-tools Suggests: openbsd-inetd | inet-superserver, smbldap-tools, ldb-tools, ctdb, ufw Conflicts: samba4 (<< 4.0.0~alpha6-2) Conffiles: /etc/ufw/applications.d/samba 9d474788f9a1955e16134f2794be473d /etc/init/nmbd.conf 737e820c320801a2a0fc9e9c84d43fc0 /etc/init/smbd.conf ef97a6e85e7f13aba5e63321990a31a4 /etc/logrotate.d/samba 9a7687a2498ae9bfbe6c1cc252818113 /etc/cron.daily/samba f6519535df7964f95cdd7db501bf3ad2 /etc/network/if-up.d/samba 2c617f0d1baebc4ba4177087a7eefc57 Description: SMB/CIFS file, print, and login server for Unix Samba is an implementation of the SMB/CIFS protocol for Unix systems, providing support for cross-platform file and printer sharing with Microsoft Windows, OS X, and other Unix systems. Samba can also function as an NT4-style domain controller, and can integrate with both NT4 domains and Active Directory realms as a member server. . This package provides the components necessary to use Samba as a stand-alone file and print server. For use in an NT4 domain or Active Directory realm, you will also need the winbind package. . This package is not required for connecting to existing SMB/CIFS servers (see smbclient) or for mounting remote filesystems (see cifs-utils). Homepage: http://www.samba.org Original-Maintainer: Debian Samba Maintainers <pkg-samba-maint@lists.alioth.debian.org> Package: greyhole Status: install ok installed Priority: extra Section: net Maintainer: Guillaume Boudreau <gb@greyhole.net> Architecture: amd64 Version: 0.9.22-1 Depends: libc6 (>= 2.4), samba (>= 3.4.3), samba-common, samba-common-bin, php5-cli (>= 5), php5-mysql, php-pear, mysql-server, rsync, sysstat, php5-intl Conffiles: /etc/init.d/greyhole c3dd1b5a5c1d2501835889558f0274ab /etc/init/greyhole.conf 980ae3ac61009bb174d89d7f429dd5e5 /etc/greyhole.conf 336d1b91df2ea4257a8565bf44ac47db /etc/logrotate.d/greyhole 7f403f4ce8557356157f4aa56f7387c4 /etc/cron.weekly/greyhole e168797a3cab5801e528a7b7bde21e31 /etc/cron.d/greyhole 643183bf54d24f9a86cb3b5214af1d43 /etc/cron.daily/greyhole d8914c41957535d54232c5ac71ad1009 Description: Drive pooling technology for Samba Allows you to create a storage pool, accessible from Samba shares, that offers data redundancy and JBOD concatenation. Homepage: http://www.greyhole.net

2. The content of the /etc/samba/smb.conf & /etc/greyhole.conf files (provide paste URLs):

Code: Select all

gstreet@gstreet-MicroServer:~$ pastebinit /etc/samba/smb.conf http://paste.ubuntu.com/12875600/

Code: Select all

gstreet@gstreet-MicroServer:~$ pastebinit /etc/greyhole.conf http://paste.ubuntu.com/12875608/

3. The result of the following commands:

Code: Select all

gstreet@gstreet-MicroServer:~$ mount /dev/sde1 on / type ext4 (rw,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) none on /sys/fs/fuse/connections type fusectl (rw) none on /sys/kernel/debug type debugfs (rw) none on /sys/kernel/security type securityfs (rw) udev on /dev type devtmpfs (rw,mode=0755) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755) none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880) none on /run/shm type tmpfs (rw,nosuid,nodev) /dev/sda1 on /var/hda/files/drives/drive1 type ext4 (rw) /dev/sdb1 on /var/hda/files/drives/drive2 type ext4 (rw) /dev/sdc1 on /var/hda/files/drives/drive3 type ext4 (rw) /dev/sdd1 on /var/hda/files/drives/drive4 type ext4 (rw) /dev/sde3 on /home type ext4 (rw,nosuid,nodev) /dev/sde4 on /media/OS_Img_backups type ext4 (ro,noexec,nosuid,nodev) gvfs-fuse-daemon on /var/lib/lightdm/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=lightdm) //127.0.0.1/Backup_Carl_Macbk on /mnt/samba/Backup_Carl_Macbk type cifs (rw) //127.0.0.1/Backup_Coolermaster on /mnt/samba/Backup_Coolermaster type cifs (rw) //127.0.0.1/Backup_Corsair on /mnt/samba/Backup_Corsair type cifs (rw) //127.0.0.1/Backup_Drew_Macbk on /mnt/samba/Backup_Drew_Macbk type cifs (rw) //127.0.0.1/Backup_Jessica_Macbk on /mnt/samba/Backup_Jessica_Macbk type cifs (rw) //127.0.0.1/Backup_microserver on /mnt/samba/Backup_microserver type cifs (rw) //127.0.0.1/Books on /mnt/samba/Books type cifs (rw) //127.0.0.1/Docs on /mnt/samba/Docs type cifs (rw) //127.0.0.1/Guitar on /mnt/samba/Guitar type cifs (rw) //127.0.0.1/Movies on /mnt/samba/Movies type cifs (rw) //127.0.0.1/Music on /mnt/samba/Music type cifs (rw) //127.0.0.1/Pictures on /mnt/samba/Pictures type cifs (rw) //127.0.0.1/mythtv on /mnt/samba/mythtv type cifs (rw) gvfs-fuse-daemon on /home/gstreet/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=gstreet)

Code: Select all

gstreet@gstreet-MicroServer:~$ sudo fdisk -l Disk /dev/sda: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x18b6bf46 Device Boot Start End Blocks Id System /dev/sda1 2048 3907028991 1953513472 83 Linux Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00052b3e Device Boot Start End Blocks Id System /dev/sdb1 2048 3907028991 1953513472 83 Linux Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x000dabbe Device Boot Start End Blocks Id System /dev/sdc1 2048 3907028991 1953513472 83 Linux Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x0008deae Device Boot Start End Blocks Id System /dev/sdd1 2048 3907028991 1953513472 83 Linux Disk /dev/sde: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000f2a57 Device Boot Start End Blocks Id System /dev/sde1 * 2048 40962047 20480000 83 Linux /dev/sde2 455628800 488396799 16384000 5 Extended /dev/sde3 42737664 412618751 184940544 83 Linux /dev/sde4 412618752 455628799 21505024 83 Linux /dev/sde5 455630848 488396799 16382976 82 Linux swap / Solaris Partition table entries are not in disk order

Code: Select all

gstreet@gstreet-MicroServer:~$ sudo df -h df: `/sys/kernel/debug': Function not implemented Filesystem Size Used Avail Use% Mounted on /dev/sde1 20G 7.9G 11G 44% / udev 1.9G 4.0K 1.9G 1% /dev tmpfs 383M 1.4M 382M 1% /run none 5.0M 0 5.0M 0% /run/lock none 1.9G 216K 1.9G 1% /run/shm /dev/sda1 1.8T 853G 888G 49% /var/hda/files/drives/drive1 /dev/sdb1 1.8T 858G 883G 50% /var/hda/files/drives/drive2 /dev/sdc1 1.8T 915G 827G 53% /var/hda/files/drives/drive3 /dev/sdd1 1.8T 861G 880G 50% /var/hda/files/drives/drive4 /dev/sde3 174G 1.5G 164G 1% /home /dev/sde4 21G 11G 8.4G 57% /media/OS_Img_backups //127.0.0.1/Backup_Carl_Macbk 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_Carl_Macbk //127.0.0.1/Backup_Coolermaster 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_Coolermaster //127.0.0.1/Backup_Corsair 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_Corsair //127.0.0.1/Backup_Drew_Macbk 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_Drew_Macbk //127.0.0.1/Backup_Jessica_Macbk 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_Jessica_Macbk //127.0.0.1/Backup_microserver 7.2T 3.8T 3.4T 53% /mnt/samba/Backup_microserver //127.0.0.1/Books 7.2T 3.8T 3.4T 53% /mnt/samba/Books //127.0.0.1/Docs 7.2T 3.8T 3.4T 53% /mnt/samba/Docs //127.0.0.1/Guitar 7.2T 3.8T 3.4T 53% /mnt/samba/Guitar //127.0.0.1/Movies 7.2T 3.8T 3.4T 53% /mnt/samba/Movies //127.0.0.1/Music 7.2T 3.8T 3.4T 53% /mnt/samba/Music //127.0.0.1/Pictures 7.2T 3.8T 3.4T 53% /mnt/samba/Pictures //127.0.0.1/mythtv 7.2T 3.8T 3.4T 53% /mnt/samba/mythtv

...and here is where I see the Greyhole error I first posted about:

Code: Select all

gstreet@gstreet-MicroServer:~$ greyhole --stats Can't describe settings with query: DESCRIBE settings - Error: Table './greyhole/settings' is marked as crashed and last (automatic?) repair failed

4. The list drives in your storage pool (per Amahi platform):

Code: Select all

gstreet@gstreet-MicroServer:~$ mysql -u root -phda -e "select * from disk_pool_partitions" hda_production +----+------------------------------+--------------+---------------------+---------------------+ | id | path | minimum_free | created_at | updated_at | +----+------------------------------+--------------+---------------------+---------------------+ | 1 | /var/hda/files/drives/drive1 | 10 | 2012-10-30 10:00:54 | 2012-10-30 10:00:54 | | 2 | /var/hda/files/drives/drive2 | 10 | 2012-10-30 10:00:55 | 2012-10-30 10:00:55 | | 3 | /var/hda/files/drives/drive3 | 10 | 2012-10-30 10:00:56 | 2012-10-30 10:00:56 | | 4 | /var/hda/files/drives/drive4 | 10 | 2012-10-30 10:00:58 | 2012-10-30 10:00:58 | +----+------------------------------+--------------+---------------------+---------------------+

5. A list of the directories on the root of the drives included in your storage pool, obtained with the following command (provide a paste URL):

Code: Select all

gstreet@gstreet-MicroServer:~$ mysql -u root -phda -e "select concat(path, '/gh') from disk_pool_partitions" \hda_production | grep -v 'concat(' | xargs ls -la | pastebinit http://paste.ubuntu.com/12875651/

6. The Greyhole work queue:

Code: Select all

gstreet@gstreet-MicroServer:~$ sudo greyhole --view-queue Can't describe settings with query: DESCRIBE settings - Error: Table './greyhole/settings' is marked as crashed and last (automatic?) repair failed

Failed again - same error

Code: Select all

tail /var/log/greyhole.log Note that greyhole.log file is empty but previous one is compressed - following are the last few lines: Oct 13 20:00:25 4 daemon: PHP Warning [2]: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld.sock' (2) in /usr/bin/greyhole on line 427; BT: greyhole[L1935] db_connect() => greyhole[L427] mysql_connect(localhost,greyhole,greyhole) Oct 13 20:00:25 2 daemon: Can't connect to mysql database. Oct 13 20:01:55 4 daemon: PHP Warning [2]: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /usr/bin/greyhole on line 427; BT: greyhole[L1935] db_connect() => greyhole[L427] mysql_connect(localhost,greyhole,greyhole) Oct 13 20:01:55 2 daemon: Can't connect to mysql database. Oct 13 20:03:25 4 daemon: PHP Warning [2]: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /usr/bin/greyhole on line 427; BT: greyhole[L1935] db_connect() => greyhole[L427] mysql_connect(localhost,greyhole,greyhole) Oct 13 20:03:25 2 daemon: Can't connect to mysql database.

So this is all the basic info. I tried again to run through the procedures in the Wicki for
1)Greyhole Work Queue
2)Database Table Error
and I was not successful. Let me know if you want me to post the info when I try this.

I'm not knowledgeable enough to troubleshoot based on the information above. Grateful if you could suggest next steps.

Thanks again for taking the time to assist.

[bigfoot65]

Wonder if MySQL is running. Try this as root user:

Code: Select all

sudo service mysqld status

If it's not running, then replace status with start.

[gstreet]

I assume you had a typo "mysqld" above?

Code: Select all

gstreet@gstreet-MicroServer:~$ sudo service mysql status mysql start/running, process 1642

Looks like it is running. Next idea?

Thanks

[bigfoot65]

Sorry about that, it's mysqld for Fedora

Wonder is this is related to your issue?

viewtopic.php?t=4402

I am at a loss. Only advice I can give is to drop the Greyhole tables and reload them as described in the wiki here.

This is something specific to Greyhole, so you may want to contact the author. His website is https://www.greyhole.net.

Sorry I cannot be of more assistance.

[gstreet]

Thanks for your help; I'll take it over to the Greyhole site and see if anyone can help me there. If I sort it out, I'll update this thread.

I guess if worse comes to worst, I might use this as an opportunity to rebuild the server under Fedora so I can run the latest Amahi which appears to be moving down that path long term. I cringe a bit at that as it was a heap of work to get everything set up three years ago but until now I have almost run problem free.

[bigfoot65]

Hope you get it sorted out. Would be interested in the solution.

Good luck!