CLOSED: ROOT is Full After Installing/Using Transmission

[BlueJacket1992]

I'm not a N00B to Linux; I've mainly worked with CentOS but have dabbled in Fedora for years. Recently I installed and used Transmission and now I'm having issues:

1. I cannot login to my Dashboard. Using a browser to go to http://hda/ yields the "Whoopsie..." page
followed by the "HTTP 500 Internal Server Error"

2. Via CLI, I check my file systems and get:

DF-ScreenShot.jpg (108.58 KiB) Viewed 4314 times

[root@localhost ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 909G 863G 0 100% /
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 872K 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs 3.9G 20K 3.9G 1% /tmp
/dev/sdc1 2.7T 1.1T 1.6T 41% /var/hda/drives/drive2
/dev/sdb1 2.7T 1.1T 1.6T 41% /var/hda/drives/drive1
/dev/sdd1 2.7T 1.1T 1.6T 41% /var/hda/drives/drive3
/dev/sda1 477M 62M 387M 14% /boot
none 4.0M 0 4.0M 0% /var/spool/greyhole/mem

I know the ROOT being full is my issue. I'm certain, too, that it's due to the GH LZ that was AUTOMATICALLY configured when I installed the plugin. However, I don't know how to move the LZ to the drive pool.

I know my ROOT is not included in the GH drive pool...though I can't post a pic of the Dashboard GH Config screen.

Any help would be greatly appreciated!

[BlueJacket1992]

BTW...I've checked my greyhole.conf and greyhole.yml files to ensure nothing is pointed to the ROOT directory. I have copies of these and my smb.conf file if someone would want to see them. Just PM me and I can send them to you (I'm not too comfortable posting my Shares config/info online.)

[bigfoot65]

The LZ is by default on the root partition. To change that to a different drive, check out Greyhole Landing Zone wiki article.

[BlueJacket1992]

Bigfoot, I don't think that's my only issue. I've done some digging in my greyhole, monit, and boot logs. Here's what I have:

In the greyhole.log:
Feb 25 08:09:52 CRITICAL initialize: Can't connect to database: SQLSTATE[HY000] [2002] No such file or directory

In the monit.log:
[EST Feb 24 03:33:03] info : Reinitializing monit daemon
[EST Feb 24 03:33:04] info : Starting monit HTTP server at [localhost:2812]
[EST Feb 24 03:33:04] info : monit HTTP server started
[EST Feb 24 03:33:04] info : 'system_localhost' Monit reloaded
[EST Feb 24 03:33:04] error : 'mysql' process is not running
[EST Feb 24 03:33:04] info : 'mysql' trying to restart
[EST Feb 24 03:33:04] info : 'mysql' start: /usr/bin/systemctl
[EST Feb 24 03:33:34] error : 'mysql' failed to start
[EST Feb 24 03:33:34] error : 'apache' process is not running
[EST Feb 24 03:33:34] info : 'apache' trying to restart
[EST Feb 24 03:33:34] info : 'apache' start: /usr/bin/systemctl
[EST Feb 24 03:34:04] error : 'apache' failed to start
[EST Feb 24 03:34:04] error : 'amahi-greyhole' process is not running
[EST Feb 24 03:34:04] info : 'amahi-greyhole' trying to restart
[EST Feb 24 03:34:04] info : 'amahi-greyhole' start: /usr/bin/systemctl
[EST Feb 24 03:34:04] error : 'amahi-anywhere' process is not running
[EST Feb 24 03:34:04] info : 'amahi-anywhere' trying to restart
[EST Feb 24 03:34:04] info : 'amahi-anywhere' start: /usr/bin/systemctl
[EST Feb 24 03:34:34] error : 'amahi-anywhere' failed to start

When I run df-h:

FileSystem01.jpg (75.31 KiB) Viewed 4311 times

So, ROOT isn't completely full...though it's not at a healthy level. I ran the yum updater to see if I've missed anything...I'm just not sure where to start. With MySQL, Apache, and greyhole not running/starting on which one should I focus my attention?

[BlueJacket1992]

And systemctl1 indicates:

[root@localhost /]# /usr/bin/systemctl
UNIT LOAD ACTIVE SUB DESCRIPTION
proc-sys-f...nfmt_misc.automount loaded active waiting Arbitrary Executable File Formats
sys-device...-sound-card1.device loaded active plugged High Definition Audio Controller
sys-device...-sound-card0.device loaded active plugged 5 Series/3400 Series Chipset High
sys-device...ock-sdb-sdb1.device loaded active plugged WDC_WD30EFRX-68EUZN0
sys-device...:0-block-sdb.device loaded active plugged WDC_WD30EFRX-68EUZN0
sys-device...ock-sdc-sdc1.device loaded active plugged WDC_WD30EFRX-68EUZN0
sys-device...:0-block-sdc.device loaded active plugged WDC_WD30EFRX-68EUZN0
sys-device...ock-sdd-sdd1.device loaded active plugged WDC_WD30EZRX-00MMMB0
sys-device...:0-block-sdd.device loaded active plugged WDC_WD30EZRX-00MMMB0
sys-device...:0-block-sr1.device loaded active plugged HL-DT-STDVD-RAM_GH22NP21
sys-device...00.0-net-em1.device loaded active plugged RTL8111/8168/8411 PCI Express Gig
sys-device...ock-sde-sde1.device loaded active plugged Cruzer
sys-device...:0-block-sde.device loaded active plugged Cruzer
sys-device...ock-sda-sda1.device loaded active plugged WDC_WD1001FALS-00E8B0
sys-device...ock-sda-sda2.device loaded active plugged WDC_WD1001FALS-00E8B0
sys-device...ock-sda-sda3.device loaded active plugged WDC_WD1001FALS-00E8B0
sys-device...:0-block-sda.device loaded active plugged WDC_WD1001FALS-00E8B0
sys-device...:0-block-sr0.device loaded active plugged ATAPI_iHES208_2
sys-device...50-tty-ttyS1.device loaded active plugged /sys/devices/platform/serial8250/
sys-device...50-tty-ttyS2.device loaded active plugged /sys/devices/platform/serial8250/
sys-device...50-tty-ttyS3.device loaded active plugged /sys/devices/platform/serial8250/
sys-device...0a-tty-ttyS0.device loaded active plugged /sys/devices/pnp0/00:0a/tty/ttyS0
sys-device...ual-net-tun0.device loaded active plugged /sys/devices/virtual/net/tun0
sys-module-configfs.device loaded active plugged /sys/module/configfs
sys-subsys...-devices-em1.device loaded active plugged RTL8111/8168/8411 PCI Express Gig
sys-subsys...devices-tun0.device loaded active plugged /sys/subsystem/net/devices/tun0
-.mount loaded active mounted /
boot.mount loaded active mounted /boot
dev-hugepages.mount loaded active mounted Huge Pages File System
dev-mqueue.mount loaded active mounted POSIX Message Queue File System
sys-kernel-config.mount loaded active mounted Configuration File System
sys-kernel-debug.mount loaded active mounted Debug File System
tmp.mount loaded active mounted Temporary Directory
var-hda-drives-drive1.mount loaded active mounted /var/hda/drives/drive1
var-hda-drives-drive2.mount loaded active mounted /var/hda/drives/drive2
var-hda-drives-drive3.mount loaded active mounted /var/hda/drives/drive3
var-spool-greyhole-mem.mount loaded active mounted /var/spool/greyhole/mem
systemd-as...sword-plymouth.path loaded active waiting Forward Password Requests to Plym
systemd-ask-password-wall.path loaded active waiting Forward Password Requests to Wall
amahi-anywhere.service loaded failed failed Amahi Anywhere service
amahi-greyhole.service loaded failed failed Greyhole for Amahi
amahi-plex.service loaded active running Plex Media Server for Amahi
amahi-transmission.service loaded active running SYSV: This is a daemon which runs
auditd.service loaded active running Security Auditing Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
chronyd.service loaded active running NTP client/server
crond.service loaded active running Command Scheduler
dbus.service loaded active running D-Bus System Message Bus
dnsmasq.service loaded active running DNS caching server.
fedora-readonly.service loaded active exited Configure read-only root support
getty@tty1.service loaded active running Getty on tty1
hda-ctl.service loaded active running Amahi HDA controller
hddtemp.service loaded active running Hard drive temperature monitor da
httpd.service loaded active running The Apache HTTP Server
monit.service loaded active running Pro-active monitoring utility for
mount_shares_locally.service loaded active exited LSB: mount Samba shares locally
mysqld.service loaded failed failed MariaDB database server
network.service loaded failed failed LSB: Bring up/down networking
NetworkManager.service loaded active running Network Manager
nmb.service loaded active running Samba NMB Daemon
openvpn@amahi.service loaded active running OpenVPN Robust And Highly Flexibl
polkit.service loaded active running Authorization Manager
rsyslog.service loaded active running System Logging Service
sendmail.service loaded active running Sendmail Mail Transport Agent
sm-client.service loaded active running Sendmail Mail Transport Client
smb.service loaded active running Samba SMB Daemon
sshd.service loaded active running OpenSSH server daemon
sysstat.service loaded failed failed Resets System Activity Logs
systemd-fsck-root.service loaded active exited File System Check on Root Device
systemd-fs...a514e97b0e9.service loaded active exited File System Check on /dev/disk/by
systemd-fs...96b0b877cfd.service loaded active exited File System Check on /dev/disk/by
systemd-fs...e02580ad196.service loaded active exited File System Check on /dev/disk/by
systemd-fs...6e4460a8b8b.service loaded active exited File System Check on /dev/disk/by
systemd-journald.service loaded active running Journal Service
systemd-logind.service loaded active running Login Service
systemd-re...ead-collect.service loaded active exited Collect Read-Ahead Data
systemd-readahead-replay.service loaded active exited Replay Read-Ahead Data
systemd-remount-fs.service loaded active exited Remount Root and Kernel File Syst
systemd-sysctl.service loaded active exited Apply Kernel Variables
systemd-tm...s-setup-dev.service loaded active exited Create static device nodes in /de
systemd-tmpfiles-setup.service loaded active exited Recreate Volatile Files and Direc
systemd-udev-trigger.service loaded active exited udev Coldplug all Devices
systemd-udevd.service loaded active running udev Kernel Device Manager
systemd-update-utmp.service loaded active exited Update UTMP about System Reboot/S
systemd-user-sessions.service loaded active exited Permit User Sessions
systemd-vconsole-setup.service loaded active exited Setup Virtual Console
avahi-daemon.socket loaded active listening Avahi mDNS/DNS-SD Stack Activatio
dbus.socket loaded active running D-Bus System Message Bus Socket
syslog.socket loaded active running Syslog Socket
systemd-initctl.socket loaded active listening /dev/initctl Compatibility Named
systemd-journald.socket loaded active running Journal Socket
systemd-shutdownd.socket loaded active listening Delayed Shutdown Socket
systemd-udevd-control.socket loaded active listening udev Control Socket
systemd-udevd-kernel.socket loaded active running udev Kernel Socket
dev-sda2.swap loaded active active /dev/sda2
basic.target loaded active active Basic System
cryptsetup.target loaded active active Encrypted Volumes
getty.target loaded active active Login Prompts
local-fs-pre.target loaded active active Local File Systems (Pre)
local-fs.target loaded active active Local File Systems
multi-user.target loaded active active Multi-User System
network.target loaded active active Network
paths.target loaded active active Paths
remote-fs.target loaded active active Remote File Systems
sockets.target loaded active active Sockets
sound.target loaded active active Sound Card
swap.target loaded active active Swap
sysinit.target loaded active active System Initialization
timers.target loaded active active Timers
systemd-readahead-done.timer loaded active elapsed Stop Read-Ahead Data Collection 1
systemd-tmpfiles-clean.timer loaded active waiting Daily Cleanup of Temporary Direct

LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.

111 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

[bigfoot65]

Have you rebooted?

[BlueJacket1992]

Yes and I get the "Woopsie..." page each time I try to login to the HDA.

And when I login to Amahi.org it indicated my HDA is STOPPED...

HDA01.png (82.38 KiB) Viewed 4304 times

[bigfoot65]

I would recommend you SSH into your HDA if you can. Then try and remove some files from the shares in /var/hda/files/sharename.

Although one screen shot shows root full and another shows space, this could be the cause of all the services failing to start.

Once you free up some space, a reboot should fix the problem and all the services would start. That appears to be what has caused all the issues. As I stated, the LZ is the root partition, so adding several files that are part of the Greyhole storage pool can easily overload it.

Did you do any manual changes to the HDA?

You can also boot a Live CD on the HDA and clean up some files on the OS disk as well.

[BlueJacket1992]

So, to answer your question: Yes, I did some manual changes to the HDA. Specifically, I deleted cache files and some libraries from the Plex application. But somehow I think I've bungled something other than those distinct libraries. So...

Since so much is going wrong, i.e., Apache, MySQL, Amahi-Greyhole, Amahi-Anywhere, I suppose I should attempt to repair my OS disk with the Live-CD.

A couple questions, then:

1) I have a significant amount of data currently in my HDA shares and Greyhole manages the disk pool on which the shares are stored. Will I lose my data if I re-load/repair the OS disk?! I am so nervous about that...
2) Any suggestions for a "process/procedure" for repairing the OS disk? I assume it will overwrite many of the config files and I'll neeed to rebuild those...

THANK YOU for all your help!

[bigfoot65]

Ok, so let me see if I understand this correctly.

You are using Greyhole for your files. Are all your shares part of the Greyhole storage pool? If so, then disconnecting those drives should preserve your data.

Also trying to reverse what you did may not work. Recommend you reinstall.