Sudden major greyhole problems

[acemilo]

Is it possible greyhole and mount_shares_locally are trying to start before mysql?

If you start these services in order

Code: Select all

service greyhole start service mount_shares_locally start

do they start OK? Can you access the shares?

After boot finishes, then yes starting greyhole and starting the mount shares script works fine, it's just during bootup that it fails.

[uslacker]

It still sounds like mysql is not up when greyhole tries to start, there for it fails.

Can you look in /etc/rc4.d for the S??mysqld entry? What number is in the name of mysqld, greyhole and mount_share_locally.

As for why this is just happening.
Perhaps adding the 6th drive is slowing down the greyhole startup.

You might also ask this on the greyhole IRC if you have chance.

[acemilo]

It still sounds like mysql is not up when greyhole tries to start, there for it fails.

Can you look in /etc/rc4.d for the S??mysqld entry? What number is in the name of mysqld, greyhole and mount_share_locally.

As for why this is just happening.
Perhaps adding the 6th drive is slowing down the greyhole startup.

You might also ask this on the greyhole IRC if you have chance.

MySQL is S64
Greyhole is S92
Mount_shares_locally is also S92

Does this have something to do with it?

[gboudreau]

fpaste-ing your complete boot.log might help.
It's definitely a problem with your MySQL; it should start before Greyhole and mount_shares_locally, but it either doesn't, or the init.d script returns before the mysql daemon is ready to receive connections.
Also look for mysql logs in /var/log/ or /var/lib/mysql/

[acemilo]

It appears that for some reason, MySQL is NOT starting before the script like it's supposed to. Here's the fpaste link: http://fpaste.org/AUcC/. MySQL is starting long after the script. What's weird is that the same thing is happening on my f14 install as my f12. How do we tell fedora to start mysql before the script? In the script, it says mysql must be started first, so I don't know why it's executing before mysql is started. Looks like we're getting closer to a fix!

[gboudreau]

Code: Select all

fpaste /etc/inittab ls -laR /etc/rc*.d/ | fpaste

[acemilo]

Code: Select all

fpaste /etc/inittab ls -laR /etc/rc*.d/ | fpaste

First command: http://fpaste.org/sZWO/
Second command: http://fpaste.org/sUN1/

[gboudreau]

In /etc/rc3.d/, you have 3 symlinks that start with "S-1"
Those will be started before everything else, and Greyhole and mount_shares_locally are there.
Remove those (rm); they are also symlinks at their proper location (S92), so just deleting the S-1 will fix your problems.
No idea what added those there, but this is the root cause of all your problems.

[acemilo]

I think we got it! I deleted the entries and rebooted. My server is back in my closet so it's headless right now but I went to my /mnt/samba directory and did an ls on a directory and it didn't work. Navigating away and back gave me a list! I don't know why it does that, but I noticed it does sometimes. Also, checking the status of greyhole shows it to be running! I think what happened was when I manually updated greyhole it must have recreated those links for some reason and made them higher priority. I don't know why, but that must have been it.

You guys are awesome, thanks for the help again. If I notice the problem is still present (shouldn't be) I'll report back, but for now, all looks good! Thanks again!

[gboudreau]

I doubt this happened during a manual Greyhole updated: Greyhole doesn't know about & doesn't touch crashplan and mount_shares_locally, both of which also had the weird symlink.