CLOSED: More Greyhole Write Issues - WARN write: No metadata files could be created -- not creating 'gh' subfolder?

[taa]

Hi All,

I'm receiving the same "No metadata files could be created" error that johnny5shared describes in this post but there are enough differences in other info reported there that a new topic seems warranted.

So here's my tale of woe. I'm in the process of upgrading my physical HDA server from Amahi 6 to Amahi 9. The last piece is to rebuild the greyhole storage pools. I have not installed the Greyhole UI app yet because I became quite nervous about the HDA OS Migration Guide and differences I noticed there in the greyhole setup. I'm in the same situation as johnny5shared when he said, "It's just that I stand to lose a lot of data if this isn't implemented properly...."

I decided to set up a Virtualbox VM, host-only network + NAT, so I could experiment and make sure I understood how share paths and the drive pools would be assigned. After over a couple dozen fresh Amahi 9 installs in vbox, I believe I have a reliable kickstart file that I can use to duplicate the setup. I am using the Greyhole UI on the VM. Basic VM stats: 1GB RAM, OS drive 8GB, what I hope will be the main drive for the landing zone is 1G, and three pool drives at 500MB each.

I made one manual tweak to an otherwise "normal" install using the Disk Wizard and Greyhole UI: in greyhole.conf I changed min_free from 10gb to 0gb (see https://www.greyhole.net/irc/ at timestamp 17:58:53) to stop the no-disk-space-available type messages.

What I think is the issue is that Greyhole UI is not creating the /gh subfolders and when the greyhole daemon doesn't find them, it spews out the "No metadata files could be created" error. I don't want to manually create them if I can avoid it -- I'm expecting the Disk Wizard and/or Greyhole UI to take the necessary steps. No files are getting duplicated as far as I can tell.

I know about manually adding a disk drive but just trying to get greyhole going with the disk wizard and automated steps has been almost too much already. Besides, I would think doing the wizard/automated steps would be preferable anyway.

I have not done anything to the mariadb files at all. I don't even know what all is in them. I'm blissfully unaware.

I know I need to (and do) stop samba and greyhole servers before I mess with this stuff.

I've had some contact with gboudreau about this, and after seeing the greyhole log, he wrote:

mkdir(/var/hda/drives/pooldisk3/gh/.gh_metastore_backup)


That seems to indicate the folder /var/hda/drives/pooldisk3/gh doesn't exist. The folders you specify as storage_pool_drive in greyhole.conf needs to exist.

Output of gboudreau's script is here: http://ix.io/1PDn

Output of apaste --sysinfo is here: http://tinyurl.com/jqqb3hy

Other info requested by Greyhole Troubleshooting is here: http://tinyurl.com/zgs5sw8

generated by this frikkin' long command:

Code: Select all

( x="=======================================" ; echo -e "$x\nStatus of mariadb\n$x\n"; systemctl status mariadb ; echo -e "\n$x\nAmahi-greyhole version\n$x\n" ; rpm -q amahi-greyhole ; echo -e "\n$x\nfdisk output\n$x\n" ; fdisk -l ; echo -e "\n$x\nmysql query #1\n$x\n"; mysql -t -u root -phda -e "select * from disk_pool_partitions" hda_production ; echo -e "\n$x\nmysql query #2\n$x\n"; mysql -u root -phda -e "select concat(path, '/gh') from disk_pool_partitions" hda_production | grep -v 'concat(' | xargs ls -la 2>&1) | apaste

The fdisk output shows a /dev/sdf1 partition. It is a 2GB flash drive I use for temporarily saving files. It will not be part of a permanent setup.

[bigfoot65]

I don't want to manually create them if I can avoid it -- I'm expecting the Disk Wizard and/or Greyhole UI to take the necessary steps.

Disk Wizard does not know the drive is being used for Greyhole, nor should it. Creating those folders is the job of Greyhole.

I've had some contact with gboudreau about this, and after seeing the greyhole log

If he indicated the problem is the missing /gh folder, then I don't see the need for the long post. He knows Greyhole better than anyone.

Unfortunately sometimes the /gh folder does not get created automatically. Create it and restart Greyhole. That should resolve the issue.

No need to make this more complicated than necessary.

As with anything, we always encourage users to back up their data. If you have multiple Greyhole drives with the same data, install one and get things going. Then if anything goes wrong, you still have a backup with the other drives.

[taa]

Creating those folders is the job of Greyhole.

Actually, no, it's not -- it's the responsibility of apps that integrate with Greyhole, in this case the Greyhole UI App. This is stated quite clearly in the Greyhole documentation "How to add drives to your storage pool":

Code: Select all

3. Create an empty folder, let's say gh, in the root of the partition: mkdir /mnt/hdd5/gh

The next step in the documentation states:

Code: Select all

4. Add the path to the folder you just created in /etc/greyhole.conf: storage_pool_drive = /mnt/hdd5/gh, min_free: 10gb

In my testing, the Greyhole UI App does update greyhole.conf as expected.

If he indicated the problem is the missing /gh folder, then I don't see the need for the long post. He knows Greyhole better than anyone.

I'm glad we're in agreement that gboudreau is the expert, but there seems to be a disconnect here: After acknowledging that gboudreau stated the problem is the missing /gh folder, and acknowledging that if anyone would know gboudreau would, these two points seem to be ignored by stating it is Greyhole's job to create it. I'm sure you didn't intend for this to come across as finger-pointing, but that is how it feels.

I'm surprised at the label of "long post" because I carefully read through this similar thread, all 18 posts spread over two browser pages, and in creating my original post on this topic I tried to be concise in presenting answers to the anticipated questions. Unlike that other thread, I did not post the complete text of the various information gathering pieces and instead posted URLs to the gh-infos and apaste results. For future reference, I don't mind writing topics differently but I'll need to know precisely what info to leave out so that it will still be complete (answers the questions Amahi Support will likely ask) and so it won't be considered "long." This post unfortunately seems necessary, and long, to clear up misunderstandings.

I don't understand what is meant by "complicated." To me it's a matter of creating the missing subfolder. If the steps needed for the Greyhole UI App to create them are complicated, I wouldn't know.

From my perspective, the Greyhole UI App should take care of creating the gh subfolders. I paid for it and it is non-refundable. Having to manually do the steps the app should be doing defeats the purpose of having the app. I'm in the middle here. Right now I'm asking Amahi to accept ownership of this issue and work with gboudreau directly if needed, but it doesn't seem like that is going to happen. I'm willing to continue as the person of contact between the two parties. If it becomes apparent that that's not going to work, then at the very least please provide me details on where I can find and modify the Greyhole UI App on my HDA. Will I have additional questions and need to ask for help from Amahi Support? Maybe, maybe not. I am aware and quite agree with your prior concerns that I'm not an advanced user, and even though I don't think I'm obligated to fix the app, I am willing to try.

[bigfoot65]

Actually, no, it's not -- it's the responsibility of apps that integrate with Greyhole, in this case the Greyhole UI App. This is stated quite clearly in the Greyhole documentation "How to add drives to your storage pool":

Not sure how recent that wiki guidance is though. Prior to the creation of the greyhole-ui App, those directories were automatically created when starting the Greyhole daemon based on my experience. That may have changed and we were not aware.

these two points seem to be ignored by stating it is Greyhole's job to create it. I'm sure you didn't intend for this to come across as finger-pointing, but that is how it feels.

No that was not the intent. As I mentioned previously, in the past Greyhole took care of that task. When installing Greyhole manually (without the app) the /gh was created automatically.

I'm surprised at the label of "long post"

I am sorry if I offended you as that was not the intent. My point was if all that was needed was to create the /gh folder, then the other additional info was no needed. Anytime there is the potential of there being a bug with Amahi or an app, we ask that users submit a bug report vs posting in the forums. The main reason as the developer does not monitor forums and sometimes I go days before I respond to posts. This ensures it's captured and we can prioritize our workload.

I don't understand what is meant by "complicated." To me it's a matter of creating the missing subfolder. If the steps needed for the Greyhole UI App to create them are complicated, I wouldn't know.

My point was if it's just the need for creating the /gh, then seemed like a simple step and all the details were not needed. Just create the folder.

From my perspective, the Greyhole UI App should take care of creating the gh subfolders. I paid for it and it is non-refundable. Having to manually do the steps the app should be doing defeats the purpose of having the app.

Well if Greyhole no longer ensures the /gh folder is created, then yes we would need to have the app take care of it. The app is intended as a GUI front end for Greyhole, preventing the user from having to manually modify the greyhole.conf file. It was not designed to create the /gh folder from what I know since that was not a requirement at the time. If it is now, then we will have to make a change.

Right now I'm asking Amahi to accept ownership of this issue and work with gboudreau directly if needed, but it doesn't seem like that is going to happen.

It was not stated that Amahi would do nothing. Please don't assume based on one response. We will need to converse with gboudreau to determine if this is indeed an Amahi issue or not. Since this has not been a problem in the past, we need to ensure the app requires an update before doing so.

I don't think I'm obligated to fix the app, I am willing to try.

That is correct, you are not. We appreciate your feedback and need to determine if the app requires an update. Again, sorry if I offended you or gave you the impression we would not look into this matter. We will need to investigate more.

Please submit a bug report with a summary of the issue along with a link to this forum thread. If you want to wait for resolution, please understand it may be a while. Sorry for the long response but I wanted to ensure I addressed all your concerns.

[taa]

Thanks for your reply and I appreciate the time you took in doing so. I'm working with the Greyhole UI on my live setup now and want to wait a few days before creating the bug report just in case something else pops up.

[bigfoot65]

Fair enough.

Will watch for the bug report.

[taa]

I figured out the cause. See bug 2209.

[bigfoot65]

We will take a look.

Thanks for following up.

Will mark this thread as closed. Further tracking will be done in the bug.

[astinusofmetate]

Rather than start a new thread for possibly the exact same issue, I want to know I've got the same problem that requires the same bandaid solution.

Used Disk Wizard as described, and then followed the Greyhole UI steps to add the drives to the pool.

Nothing appeared to happen best I can tell. I mean, I copied test files to what I thought was a "pooled" share, Greyhole status indicated nothing was happening, and the free space on the root drive got smaller. Hit the little "Run FSCK" button and nothing happened.

Tried rebooting and the Greyhole status web page now says "Currently working on task ID...". That sounds like progress! So I looked at the Greyhole log file via the app "Greyhole LogMon". It's now stuck in a loop with the phrase:

No metadata files could be created. Will wait until metadata files can be created to work on this file.

I used the app Extplorer to quickly browse around. The 2 new drives are showing as subfolders under the /drives folder (I assume those are some sort of link/reference thing that greyhole/linux does?).

Should I create a "gh" subfolder under each drive folder there using this app?

Did I miss some part of the setup step or is this part of the bug previously submitted?

[bigfoot65]

The 2 new drives are showing as subfolders under the /drives folder (I assume those are some sort of link/reference thing that greyhole/linux does?).

That is correct. It's called a mount point. Think of it as basically a folder where all the files on the drive reside.

Should I create a "gh" subfolder under each drive folder there using this app?

That is the fix. It should be done automatically and we are unsure why. Only happens to some users.

Did I miss some part of the setup step or is this part of the bug previously submitted?

See bug 2209 for details. It appears it may be a side affect of using the disk-wizard App.