SOLVED: Can't find where it's copied my files to!

Rok
Posts: 12
Joined: Sun Jan 11, 2015 12:06 pm

SOLVED: Can't find where it's copied my files to!

Postby Rok » Sun Apr 22, 2018 12:00 pm

Hi,

My Amahi setup has an internal HDD and two external USB HDDs. I recently started hearing an ominous clicking noise from one of the externals so invested in a 4TB external to serve as a local backup for all the existing HDDs. I already have cloud backup for the very important stuff, but a local backup seems/feels like a nice plan.

I plugged the new HDD in and used the Amahi Disk Wizard application. This seemed to go fine and I was able to format and then create a shared folder on the new drive.

Using a terminal via WinSCP I set up the initial backup of the clicking drive using something along the lines of -

rsync -avz /var/hda/files/drives/media/ /var/hda/files/drives/backup/media/

Everything seemed to be going fine until an error popped up in the console saying that it had run out of disk space. This struck me as slightly odd seeing as it was backing up a partially filled 2TB drive to an empty 4TB one.

Looking at the drives using the Amahi dashboard it wasn't actually showing the new drive at all but was showing the internal to have completely filled up. The partition was showing for the new drive but the actual drive wasn't showing at all. I rebooted the server and could now see the new drive.

My problem is that the files must have copied from the clicking external drive to the internal drive as that has completely filled up, but I cant seem to locate them to remove them. They're not in /var/hda/files/drives/backup as that is showing as being empty.

This is clearly a Linux newbie issue and I'm sure there's a straight forward and obvious solution. It's just the it isn't straight forward and obvious to me right now. I did a file search using WinSCP for one of the files that was definitely copied but it didn't locate it on the internal drive. Any help and/or advice would be greatly appreciated!

Rok
Posts: 12
Joined: Sun Jan 11, 2015 12:06 pm

Re: Can't find where it's copied my files to!

Postby Rok » Fri Apr 27, 2018 12:37 am

I'm not sure what the policy is on these forums for bumping posts but this is the one and only time I'll do it. I just have no idea how to locate where the files have gone and right now I'm stuck with a drive that's completely full when it really shouldn't be.

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Can't find where it's copied my files to!

Postby bigfoot65 » Fri Apr 27, 2018 9:09 am

I'm not sure what the policy is on these forums for bumping posts but this is the one and only time I'll do it.
We request users not bump posts. This is a community supported forum, so if you don't get a reply it's because someone doesn't have the experience to assist or they have not seen your post yet.

So I can try to assist. First of all, are you using Greyhole?

Can you provide the URL for:

Code: Select all

apaste --sysinfo
Once I see your setup and such, hope to be able to determine the issue.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

Rok
Posts: 12
Joined: Sun Jan 11, 2015 12:06 pm

Re: Can't find where it's copied my files to!

Postby Rok » Sun Apr 29, 2018 1:02 am

Thank you for offering to help.

First of all, are you using Greyhole?

I'm not using Greyhole. I have an Acer Aspire Revo running Amahi on it's internal HDD. Attached to it, I have three external USB HDDs (one of them being the new 4TB drive that I added and attempted to copy to). I access the drives via shares that I set up (such as /var/hda/files/drives/backup)

Can you provide the URL for:
CODE: SELECT ALL

apaste --sysinfo
Once I see your setup and such, hope to be able to determine the issue.

When I attempt to do this, I get the following output from the PuTTY session -

Code: Select all

Gathering system info ...............................Uploading (10.6KiB)...Error Uploading: Internal Server Error

I don't know if this is related but I'm also having trouble connecting with the dashboard. I'm generally getting a warning message that "The Dashboard has encountered an exception!" and that it could be to do with very low disk in the root partition. This seems likely seeing as it filled up with backed up files that I can't locate :)

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Can't find where it's copied my files to!

Postby bigfoot65 » Mon Apr 30, 2018 5:44 pm

I'm generally getting a warning message that "The Dashboard has encountered an exception!" and that it could be to do with very low disk in the root partition.
This is likely that the root drive is full. Best way to clear space is to log into the HDA using terminal. Then remove some of the large files in your shares.

If you can still access the share Backup, you may be able to remove files that way as well.

Either way, until you clear some space on root, you will have issues. All your files should live in the /var/hda/files directory.

BTW, when you add drives to the HDA you have to mount them using the disk-wizard App or follow the manual guidance.

Linux doesn't play as well with USB drives like Windows.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

Rok
Posts: 12
Joined: Sun Jan 11, 2015 12:06 pm

Re: Can't find where it's copied my files to!

Postby Rok » Mon Apr 30, 2018 11:07 pm

This is likely that the root drive is full. Best way to clear space is to log into the HDA using terminal.

I've removed some files from the root drive now and can again access the dashboard. I'm still having no luck with apaste --sysinfo though.

If you can still access the share Backup, you may be able to remove files that way as well.

Removing files from the Backup share isn't the problem. After rebooting following adding the drive using the disk-wizard app, the drive works exactly as expected. The problem seems to have stemmed from writing to the drive before performing the reboot. This is the process which caused the root drive to fill up (rather than the new backup drive).

BTW, when you add drives to the HDA you have to mount them using the disk-wizard App or follow the manual guidance.

Yup - that's the process I followed. But before rebooting, I rsync'd to /var/hda/files/drives/backup. It seemed to be working but apparently it was writing to the root drive rather than the new USB drive (presumably because it wasn't mounted until after the reboot?)

I've searched the root drive for some of the files that would have been copied there but am unable to find them. There was a lot of space on the root drive prior to this operation so they must be there somewhere. Or the drive at least thinks they're there somewhere. I would estimate that in excess of 100GB has vanished somewhere during this process and I can't find the files anywhere on the root drive to delete them! There should be a lot of mp4 files, for example, but searching the entire drive (recursively from root using WinSCP) isn't showing them anywhere on the root drive. And yet in excess of 100GB of files must be there somewhere for the drive to now be full.

Could the folder/directory/file information have gotten corrupted during this process so that the drive thinks it's full?

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Can't find where it's copied my files to!

Postby bigfoot65 » Tue May 01, 2018 6:26 am

I'm still having no luck with apaste --sysinfo though.
It seems to be temperamental. You might be able to redirect it to a file:

Code: Select all

printf --sysinfo > sysinfo.txt
Then move the file to here.
Could the folder/directory/file information have gotten corrupted during this process so that the drive thinks it's full?
It's possible. Also check /tmp to see if they might have gotten written there.

There are scripts you can find on the internet that will help you find large files on the drive. Might help you locate them.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

Rok
Posts: 12
Joined: Sun Jan 11, 2015 12:06 pm

Re: Can't find where it's copied my files to!

Postby Rok » Wed May 30, 2018 5:08 am

I finally got some time to properly investigate this. I'm posting the results here just in case anybody else has a similar problem in the future. Here's what caused the issue -

I used Disk Wizard to mount the new external USB drive (as /var/hda/files/drives/backup)
I backed up up my media drive to the new drive using rsync
I rebooted the server and found that /var/hda/files/drives/backup contained no files
Shortly afterwards I found my main drive was now completely full but was unable to find the new files that were filling it

Here's the outcome -

What had happened was that the files had backed up from my media drive to /var/hda/files/drives/backup, filling up the internal hard drive. When the USB drive was properly mounted to that folder, it basically masked the files that had been copied into it so I could see what was on the new USB drive (which was nothing) but not the files that had copied to /var/hda/files/drives/backup when the drive hadn't been mounted!

And here's the soluition -

The solution was to unmount the new drive, delete the contents of /var/hda/files/drives/backup and then remount the new drive there. And my missing 220GB has finally returned to me :)

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Can't find where it's copied my files to!

Postby bigfoot65 » Wed May 30, 2018 1:13 pm

Ah that makes sense. I think there may be an issue with the disk-wizard App.

I prefer to mount the drives https://wiki.amahi.org/index.php/Adding ... DAmanually myself.

Will mark this topic as resolved.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

Who is online

Users browsing this forum: No registered users and 6 guests