Running Amahi on Fedora 12, with a 500 GB internal drive, and a 5.5TB Rosewill enclosed attached via eSata and mounted at /var/hda/files/disks/drive1. Both drives are part of Greyhole and all of my shares are configured to use the pool.
I am running Sabnzbd with Couch Potato and Sick Beard and this morning was checking into why some downloads weren't there and saw that Sabnzbd was reporting that there was 0.0GB available and the download was froze.
I switch over to Amahi web util and check my drives. The internal drive is reporting that there is 20GB available while the 5.5TB is reporting that 4.7TB is available.
At this point with the exception of my stalled download all other services (VPN, SMB, etc) are working without issue.
I ssh'ed into my box as root (I'm at work right now) and initiated a greyhole --fsck at which point it stalled out, and all services became inaccessible and disconnected.
I'm at a loss for what I need to do to get everything back up and running. Thanks.
Server locked up, primary drive full.
Re: Server locked up, primary drive full.
Open up a terminal on your hda and type: df -h (this will display all your partitions and the amount of space on them)
Copy the output and post it here. Perhaps we can help.
Copy the output and post it here. Perhaps we can help.
-
- Posts: 5
- Joined: Thu Oct 06, 2011 10:03 pm
Re: Server locked up, primary drive full.
So I am apparently back in the situation.
Last time this happened, by the time that I returned home and began to look as to what was going on, the fsck had finished, everything was balanced out and after a reboot everything came back up no issue. Everything has been working fine, have downloaded lots of file and haven't had any issues, until today, I am once again showing 0 GB free on my main drive and nothings working again.
I am currently running greyhole --balance and will follow up with a greyhole --fsck reboot and see what happens, but I would like to find a permanent fix so I'm not doing this once a month.
As requested below is the output when I run df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_amahi-lv_root
457G 434G 0 100% /
tmpfs 502M 0 502M 0% /dev/shm
/dev/sda1 194M 22M 163M 12% /boot
/dev/sdb 5.5T 1.3T 4.0T 25% /var/hda/files/drives/sdb
Thanks
Last time this happened, by the time that I returned home and began to look as to what was going on, the fsck had finished, everything was balanced out and after a reboot everything came back up no issue. Everything has been working fine, have downloaded lots of file and haven't had any issues, until today, I am once again showing 0 GB free on my main drive and nothings working again.
I am currently running greyhole --balance and will follow up with a greyhole --fsck reboot and see what happens, but I would like to find a permanent fix so I'm not doing this once a month.
As requested below is the output when I run df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_amahi-lv_root
457G 434G 0 100% /
tmpfs 502M 0 502M 0% /dev/shm
/dev/sda1 194M 22M 163M 12% /boot
/dev/sdb 5.5T 1.3T 4.0T 25% /var/hda/files/drives/sdb
Thanks
Re: Server locked up, primary drive full.
login with ssh (as root or become root)
you'll probably see /var being very large
you'll probably notice /log being very large
you can remove all log files (or just 1 big log file) in /var/log and run "logrotate -f /etc/logrotate.conf" to get everything back in order.
Code: Select all
cd /
du -kh --max-depth=1 .
Code: Select all
cd /var
du -kh --max-depth=1 .
you can remove all log files (or just 1 big log file) in /var/log and run "logrotate -f /etc/logrotate.conf" to get everything back in order.
echo '16i[q]sa[ln0=aln100%Pln100/snlbx]sbA0D2173656C7572206968616D41snlbxq' | dc
Galileo - HP Proliant ML110 G6 quad core Xeon 2.4GHz, 4GB RAM, 2x750GB RAID1 + 2x1TB RAID1 HDD
Galileo - HP Proliant ML110 G6 quad core Xeon 2.4GHz, 4GB RAM, 2x750GB RAID1 + 2x1TB RAID1 HDD
-
- Posts: 5
- Joined: Thu Oct 06, 2011 10:03 pm
Re: Server locked up, primary drive full.
Greyhole is what is taking up a majority of my 500gb drive.
I removed that drive from the greyhole pool and ran --fsck and --balance, but the files that were stored on that partition were no longer accessible.
How do I remove the 500gb drive leaving the 5.5tb raid array and move the files to it? Because that would obviously solve my drive space issues on the 500gb
I removed that drive from the greyhole pool and ran --fsck and --balance, but the files that were stored on that partition were no longer accessible.
How do I remove the 500gb drive leaving the 5.5tb raid array and move the files to it? Because that would obviously solve my drive space issues on the 500gb
Re: Server locked up, primary drive full.
It is not recommended to use Raid and Greyhole. If you removed the drive from the pool, the files are still on the drive in the directory. You will have to remove that directory manually.
Since you have removed it from the pool, it will not longer be used. If there were files on that drive that were not anywhere else, you might wanna manually copy them to a different drive.
The best way to remove a drive from the pool, is to run the command as root user:
That will ensure any files that exist only on that drive are moved to another one. Then after that, do the following:
Then you can remove it from the pool. That is how I do it.
This is a topic best addressed in the Greyhole topic area for future. There are advanced users who watch that area and can provide assistance.
Since you have removed it from the pool, it will not longer be used. If there were files on that drive that were not anywhere else, you might wanna manually copy them to a different drive.
The best way to remove a drive from the pool, is to run the command as root user:
Code: Select all
greyhole --going /var/hda/files/drives/drive1
Code: Select all
greyhole --gone /var/hda/files/drives/drive1
This is a topic best addressed in the Greyhole topic area for future. There are advanced users who watch that area and can provide assistance.
ßîgƒσστ65
Applications Manager
My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2
Applications Manager
My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2
Re: Server locked up, primary drive full.
moredruid, I'm trying to follow your approach and am not having much luck.
I've run "logrotate -f /etc/logrotate.conf" and here is the result of the two du commands:
[root@localhost ~]# cd /
[root@localhost /]# logrotate -f /etc/logrotate.conf
[root@localhost /]# du -kh --max-depth=1
25M ./root
34M ./etc
18M ./boot
16K ./lost+found
8.0K ./1. Waiting to Encode
4.0K ./srv
1.5T ./home
14M ./sbin
4.0K ./cgroup
11T ./var
du: cannot access `./proc/11077/task/11077/fd/4': No such file or directory
du: cannot access `./proc/11077/task/11077/fdinfo/4': No such file or directory
du: cannot access `./proc/11077/fd/4': No such file or directory
du: cannot access `./proc/11077/fdinfo/4': No such file or directory
du: cannot access `./proc/27167': No such file or directory
du: cannot access `./proc/27185': No such file or directory
0 ./proc
0 ./sys
8.0K ./opt
8.4M ./bin
122M ./lib
4.0K ./selinux
4.0K ./mnt
4.0K ./media
104K ./tmp
1.8G ./usr
3.3M ./.gem
612K ./dev
12T .
[root@localhost /]# cd /var
[root@localhost var]# du -kh --max-depth=1
12K ./yp
1.2M ./www
252K ./run
11T ./hda
4.0K ./account
20K ./lock
402M ./spool
4.0K ./preserve
4.0K ./opt
4.0K ./local
353M ./log
4.0K ./gdm
20K ./db
330M ./lib
4.0K ./games
456M ./cache
4.0K ./nis
4.0K ./tmp
8.0K ./empty
4.0K ./report
220K ./named
11T .
[root@localhost var]#
And, here's my available space after this:
[root@localhost var]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
50G 48G 0 100% /
tmpfs 1.8G 0 1.8G 0% /dev/shm
/dev/sda1 485M 34M 426M 8% /boot
/dev/mapper/VolGroup-lv_home
1.8T 1.5T 170G 90% /home
/dev/sdb1 1.8T 1.6T 171G 91% /var/hda/drives/drive1
/dev/sdc1 1.8T 1.6T 171G 91% /var/hda/drives/drive3
/dev/sdg3 1.8T 1.6T 172G 91% /var/hda/drives/drive4
/dev/sdd1 1.8T 1.5T 282G 84% /var/hda/drives/drive5
/dev/sde1 1.8T 1.5T 282G 84% /var/hda/drives/drive6
/dev/sdi1 917G 700G 171G 81% /var/hda/drives/drive7
/dev/sdf1 917G 589G 282G 68% /var/hda/drives/drive8
/dev/sdj1 917G 700G 171G 81% /var/hda/drives/drive9
/dev/sdh1 917G 701G 170G 81% /var/hda/drives/drive10
[root@localhost var]#
Any other suggestions on how I can either make space available on my root partition or how I can increase it's size without creating more issues?
Until I fix this available space issue I don't think I can fix the "ruby on rails"/greyhole database errors I'm getting.
Thanks.
I've run "logrotate -f /etc/logrotate.conf" and here is the result of the two du commands:
[root@localhost ~]# cd /
[root@localhost /]# logrotate -f /etc/logrotate.conf
[root@localhost /]# du -kh --max-depth=1
25M ./root
34M ./etc
18M ./boot
16K ./lost+found
8.0K ./1. Waiting to Encode
4.0K ./srv
1.5T ./home
14M ./sbin
4.0K ./cgroup
11T ./var
du: cannot access `./proc/11077/task/11077/fd/4': No such file or directory
du: cannot access `./proc/11077/task/11077/fdinfo/4': No such file or directory
du: cannot access `./proc/11077/fd/4': No such file or directory
du: cannot access `./proc/11077/fdinfo/4': No such file or directory
du: cannot access `./proc/27167': No such file or directory
du: cannot access `./proc/27185': No such file or directory
0 ./proc
0 ./sys
8.0K ./opt
8.4M ./bin
122M ./lib
4.0K ./selinux
4.0K ./mnt
4.0K ./media
104K ./tmp
1.8G ./usr
3.3M ./.gem
612K ./dev
12T .
[root@localhost /]# cd /var
[root@localhost var]# du -kh --max-depth=1
12K ./yp
1.2M ./www
252K ./run
11T ./hda
4.0K ./account
20K ./lock
402M ./spool
4.0K ./preserve
4.0K ./opt
4.0K ./local
353M ./log
4.0K ./gdm
20K ./db
330M ./lib
4.0K ./games
456M ./cache
4.0K ./nis
4.0K ./tmp
8.0K ./empty
4.0K ./report
220K ./named
11T .
[root@localhost var]#
And, here's my available space after this:
[root@localhost var]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
50G 48G 0 100% /
tmpfs 1.8G 0 1.8G 0% /dev/shm
/dev/sda1 485M 34M 426M 8% /boot
/dev/mapper/VolGroup-lv_home
1.8T 1.5T 170G 90% /home
/dev/sdb1 1.8T 1.6T 171G 91% /var/hda/drives/drive1
/dev/sdc1 1.8T 1.6T 171G 91% /var/hda/drives/drive3
/dev/sdg3 1.8T 1.6T 172G 91% /var/hda/drives/drive4
/dev/sdd1 1.8T 1.5T 282G 84% /var/hda/drives/drive5
/dev/sde1 1.8T 1.5T 282G 84% /var/hda/drives/drive6
/dev/sdi1 917G 700G 171G 81% /var/hda/drives/drive7
/dev/sdf1 917G 589G 282G 68% /var/hda/drives/drive8
/dev/sdj1 917G 700G 171G 81% /var/hda/drives/drive9
/dev/sdh1 917G 701G 170G 81% /var/hda/drives/drive10
[root@localhost var]#
Any other suggestions on how I can either make space available on my root partition or how I can increase it's size without creating more issues?
Until I fix this available space issue I don't think I can fix the "ruby on rails"/greyhole database errors I'm getting.
Thanks.
Re: Server locked up, primary drive full.
Are you running Greyhole in addition to using the logical-volume manager?
Post the output of this command:
sudo lvs
This will show all of the logical volumes you have.
I'm having a hard time understanding where all the / space went.
Post the output of this command:
sudo lvs
This will show all of the logical volumes you have.
I'm having a hard time understanding where all the / space went.
Re: Server locked up, primary drive full.
If I am running LVM it isn't on purpose. I'm running what I thought was a standard install from the install CD.
Here is what I get from sudo LVM
[root@localhost /]# sudo lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
lv_home VolGroup -wi-ao 1.76t
lv_root VolGroup -wi-ao 50.00g
lv_swap VolGroup -wi-ao 5.19g
You'll notice that it isn't quite consistent with the first listings I had above as I've found some files that appear to be on my landing zone and not moved to the correct drives. I've been removing those files and have freed up a bit of space.
Any further help you can provide would be great.
Richard
Here is what I get from sudo LVM
[root@localhost /]# sudo lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
lv_home VolGroup -wi-ao 1.76t
lv_root VolGroup -wi-ao 50.00g
lv_swap VolGroup -wi-ao 5.19g
You'll notice that it isn't quite consistent with the first listings I had above as I've found some files that appear to be on my landing zone and not moved to the correct drives. I've been removing those files and have freed up a bit of space.
Any further help you can provide would be great.
Richard
Re: Server locked up, primary drive full.
I'm not a greyhole expert, but it sounds like you must have your landing zone on your OS drive.
Search around the forums, there might be a way to move your landing zone to a different drive.
If your LZ fills up, there is no room left for the OS which can lead to problems.
I don't run greyhole myself, so I won't be much help to you.
Good luck.
Search around the forums, there might be a way to move your landing zone to a different drive.
If your LZ fills up, there is no room left for the OS which can lead to problems.
I don't run greyhole myself, so I won't be much help to you.
Good luck.
Who is online
Users browsing this forum: No registered users and 10 guests