Rebuild Greyhole Incrementally or All at Once?

jkbuckethead
Posts: 78
Joined: Sat Jan 31, 2015 5:35 pm

Rebuild Greyhole Incrementally or All at Once?

Postby jkbuckethead » Mon Oct 05, 2015 9:00 am

I need to rebuild my Greyhole storage pool after upgrading to Amahi 8. I have 8 data drives, containing 18-19 TB of data, mostly video and photos. Currently, I am not planning to start with my old GH config files because quite frankly I have no idea how to transfer the files from my old system disk to the new. I'm a total linux newbie so copying files from a hard drive to a flash drive and back again, something I can easily do in DOS, is beyond me. If someone can point me toward a good tutorial on how to mount and copy to/from a flash drive, I might go in another direction, but for now I am rebuilding GH from scratch.

Should I install all the drives at once and rebuild in one shot, or should I add one or two drives at a time and rebuild incrementally?

It seems like quite a big task to index all the disks in one shot. It seems a little safer to index in smaller batches. Is an incremental rebuild even possible?

Another option I'm considering is a brute force approach. I have a couple of blank drives available. I've thought about creating a new GH storage pool then copying files from the existing GH disks using another computer. As each disk is copied, I'll reformat it and add it to the storage pool to increase capacity as I go. This will take some time, and a fair bit of manual intervention, but it seems like it should work.

Thoughts and advice appreciated.

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Rebuild Greyhole Incrementally or All at Once?

Postby bigfoot65 » Mon Oct 05, 2015 9:55 am

I typically rebuild mine all at once.

Keep in mind that if all your files are in the storage pool, they are on the drives in the gh folder. When rebuilding them, all that is typically done is creating the symbolic links that would be missing in your shares (LZ).

Very little if any file copying should take place unless you have files that are not part of the storage pool.

Does that make sense?

Are you following the Reconnect existing Greyhole storage pool guidance?

While I don't have near as much data as you, it appeared to be the best way to rebuild. May take a while since you have a lot of data. The key is to recreate the shares and number of copies to match what you had in Amahi 7.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

jkbuckethead
Posts: 78
Joined: Sat Jan 31, 2015 5:35 pm

Re: Rebuild Greyhole Incrementally or All at Once?

Postby jkbuckethead » Mon Oct 05, 2015 1:57 pm

Keep in mind that if all your files are in the storage pool, they are on the drives in the gh folder. When rebuilding them, all that is typically done is creating the symbolic links that would be missing in your shares (LZ).

Very little if any file copying should take place unless you have files that are not part of the storage pool.

Does that make sense?
Yes, that makes sense.
Are you following the Reconnect existing Greyhole storage pool guidance?
No, I am not following the wiki at this time. The referenced wiki section starts with "back up the GH config files somewhere safe." I didn't know how to do Step #1, so I wasn't sure what else would apply. I still have my Amahi 7 system disk so I do have the files, I just don't know how to transfer them.

Using online references, I have tried to figure out how to copy files from disk to disk, but so far I've had no luck. Back in the day I was pretty good with DOS, but linux is proving a tough nut to crack. The linux terminal just makes my head hurt. I guess 20-some years of GUI computing has made me soft. It's funny I'm having so much trouble with the text interface since my first computing experience ever was on a paper terminal VAX. Definitely no mouse in those days.

I have a computer with Ubuntu that I set up just to try out linux. I may try attaching the drives to it and see if I can figure out how to transfer the files that way. The GUI should help me through the rough parts.

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Rebuild Greyhole Incrementally or All at Once?

Postby bigfoot65 » Mon Oct 05, 2015 2:06 pm

There are many ways to transfer the files, but if you were using Greyhole there is no need to transfer them. Just follow the wiki guidance. Moving them manually is the long way and will take quite a long time. There is a better way to do it. You will also find a lot of guidance in the wiki, such as HDA OS Migration guide that might help and be easier to follow.

Only files that were not part of the storage pool remain on your old system. You could run both HDAs at the same time, using samba to transfer files or plug in the old disks to the new HDA and mount them to pull off files.

As I said, there are many ways to do this one. I personally would start with the first recommendation, then see what is missing if anything. You can then transfer those files.

I presume your setup made use of Greyhole for all your shares, correct?
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

jkbuckethead
Posts: 78
Joined: Sat Jan 31, 2015 5:35 pm

Re: Rebuild Greyhole Incrementally or All at Once?

Postby jkbuckethead » Mon Oct 05, 2015 6:04 pm

I'm not trying to copy all of my data, that would be a last resort only. I was talking about transferring the GH config files that are referenced in Step 1 of the wiki. I haven't figured out how to back up these files from the old system drive to the new one. Copy from A: drive to B: drive I can do, but I can't figure out how to do the same in linux. I think the problem is I haven't yet grasped the concept of mount points.

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Rebuild Greyhole Incrementally or All at Once?

Postby bigfoot65 » Mon Oct 05, 2015 6:16 pm

What Greyhole config files are you referring to? If you mean greyhole.conf, you won't need it if you are using the Greyhole UI. It will overwrite anything you manually set in that file.

As for mount points, think of them as a shortcut to a directory. When you mount a drive to the the directory, it makes all files on that drive available in that directory. For example, mounting /dev/sdb1 to /var/hda/files/drives/drive1 is nothing more than creating a shortcut for the drive (sdb1) to the drive1 directory.

Does that help?

Here's a good link that might shed some more light on the subject.
http://www.linuxnix.com/what-is-a-mount ... linuxunix/
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

Who is online

Users browsing this forum: No registered users and 15 guests