Server hardware queries for New Amahi 7 server-Proxmox

iluciv
Posts: 17
Joined: Fri Apr 23, 2010 3:31 pm

Server hardware queries for New Amahi 7 server-Proxmox

Postby iluciv » Wed Sep 18, 2013 9:38 pm

Hi Bit unsure as where to post this (here or in installation) but I've been reading forum posts for a while and have found some answers but not all yet.

Anyway here goes

Wanting to upgrade to Amahi 7 as my current server has a few problems from playing with the configurations etc, so now, only a rebuild will resolve elegantly. The problem with this is that I stupidly put the fedora OS partition on a drive that also has some of my storage (greyhole across 3 physical discs of varying sizes) so I can't easily just re-install Amahi and point the fresh install at my storage drives. Been loving the server and so with the new release out and all the goodness of it I want to experience, I decided that I will retire the current machine, which is just a bunch of old parts thrown into a case, and build a more robust system that is capable of replacing a few of the other scrappy machine I have in service also.

The other motivation is that I got the opportunity to purchase a really cheap AMD CPU and motherboard.

I'm planning on building a proxmox server on the initial OS drive but I've been reading that it would probably be best to have the Amahi VM on its own dedicated physical drive. If that's the case do you think it would be possible to use a USB 3.0 thumb drive as that dedicated Amahi VM drive? Other wise I'll get another smaller 2.5 notebook drive and use that as the physical drive for the dedicated amahi VM image or can I just put the VM on the proxmox install drive?

I'm hoping I can use openVZ style containers are there any issue that anyone knows with regards to pointing that type of container to separate storage?(see below for storage) Using proxmox appeals to me as I believe I should be easily do a copy of say the Amahi VM after initial successful installation and configuration and if I stuff it up later on fail it over to the fresh copy and be good to go in relatively short amount of time and many other features including eventually using SPICE remote display system.

Storage
I'm think of going a raid this time around for my storage but I'm not sure, I've been happy with greyhole but from what I understand you can't rebuild if a drive fails unless you've selected to have a copy of data on every drive? I've not purchased storage yet as I want to make sure I'm going to get the right type of disks for the server, I want the storage to be reliable (of course) but I even though it'll be on all the time as it does my Lans dns dchp etc, I would like the discs to spin down when idle which will be a fair amount of the time except on weekends when I'm home working and using the LAN. I've listed all the drive brands and types in the server specs at the bottom of the post if I could get peoples experiences with any of the drive types that would be great!

I would like the storage to be as seperate as it can be from the Amahi installation by that I mean I want to be able to rebuild Amahi and fedora OS easily without the storage being in any danger of being unusable. (hope that make sense) I would also like to be able to point other VMs to that storage for their separate use not sure how to do this or if it will be possible at all, this is a bit of a black area in my comprehension and I maybe over complicating the thinking of how to go about this. I thought one way might be sym linking the /var/hda/files section to the folders on the storage? I like the idea of putting in a new drive and rebuilding the array should a drive fail. I will be using bittorrent sync and open cloud (and possibly crash plan) for the backups of important data to machines out side of physical home. I use the storage for streaming my multimedia collection to my various media center front ends as far as accessing the storage goes.

Hardware
Its a bit of a beasty hardware wise, like I said I got the mobo and cpu cheap (ram two for that matter) I don't mind waiting and saving up for the storage as the box I have running at the moment is working fine for what I use it for. I'm leaning towards the WD greens as I think this the spin down in idle will be most helpful. I'm concerned about heat, I live in a hot part of the world in summer and I don't need the storage spinning away if it's not being used (I've no aircon); as it stands at the moment I can only spin down my wd green drive I have in the current server (get some hdtemp locking out my other 2 drives when I try to spin them down) but the one that's spun down is 7C cooler than the others. I do want to run a few other always on appliances on this server in separate VMs, asterix PABX and a mythTV backend (unless there is something better now days) plus a few centOs boxes for my redhat training. I'll post the current specs below and the other bits I'm planning on getting as I save for them. I want this build to last for a long time I believe it will and I also think the outlay will be worthwhile if I get a few years of reliable service out of this box. If you see something that's glaringly obvious that it won't work please let me know.

My amahi currently serves all my dns and dhcp and I have to routers with two internal lan IPs for no real reason other than I can set it up like that lol.


Specs

Mother Board
SABERTOOTH 990FX Revision 1

CPU
AMD FX 8120 +corsair radiator

Ram
Corsair 32GB (4x8GB) Vengeance CMZ32GX3M4X1866C10R DDR3 Red

Lan
lan1841, PCI Express Quad Gigabit ethernet board

OS Drive
Proxmox Base OS /VM hosts drive WD Black 750gig

250Gig notebook drive for dedicated amahi vm

Storage (To be decided) also think I'll need a sata expansion card of some sort

Thinking ORICO 6205SS 5Bay 3.5" SATA Internal Hot Swap Rack Enclosure

Currently thinking

5 x Western Digital Green 3TB Green 64MB SATA 3 WD30EZRX

or

5 x Western Digital RED WD40EFRX RED NAS- 4TB/INTELLIPOWER/DDR2/3.5

or

5 x Seagate SATA3 3TB 7200RPM Barracuda 64mb Cache

or

5 x Seagate ST4000DM000 Desktop HDD 4TB, SATA3, 3.5", 64MB

So if someone can read that ramble and answer a few of the questions that would be great thanks, my main point on concern is how amahi accesses the storage and also how to keep the storage independent if that makes sense. For instance if I decide that proxmox is not for me or too much hassle down the track and I want to switch back to a dedicated amahi drive install it's as simple as reinstall on the OS drive and point it to the storage drives.

trw3937
Posts: 23
Joined: Mon Jul 16, 2012 9:53 am

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby trw3937 » Tue Oct 01, 2013 12:14 pm

I run pmox v3 and vbox. I can't speak to all the questions you have but can confirm running am7 on either v-platform is simple and works great for me. With a beast server like that running an am7 vm will be no problem.

Why you would want to run it on it's own drive externally is beyond me. If you are trying to make it easy to transfer data to a bare metal box then do backups of the data on the vm's hd to another server through smb/nfs. Install am7 and restore the data to it.

Trying to reuse a drive that was attached to a pmox server just seems risky. pmox uses debian lvm with tons of tweaks and am7 uses fedora ext4 by default. I would not use a vz container either. Use full kvm for proper performance.

User avatar
sgtfoo
Posts: 419
Joined: Sun Jul 18, 2010 8:27 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby sgtfoo » Mon Oct 07, 2013 11:10 am

Keep in mind, Proxmox creates a "local" storage logical partition for VM images/"drives".

As it lives within LVM, and it's all optimized for storage for multiple VMs, making it a RAID array for both/either speed/redundancy is good to do if you can afford the drives/storage space.

Avoid having external storage from the proxmox box UNLESS it's iSCSI attached. For a home server, detached storage feels like overkill (IMO)

Having a dedicated drive for amahi OS is definitely overkill. The most commonly accessed data is in the shares, so if anything use a drive exclusively for the shares, or as above, within the local storage in the LVM (RAIDed or not)

Also, the Proxmox OS drive needn't be huge either. The "local" storage, again, carries the brunt of the large files/ storage for VM stuff and containers.

You never know... you might find another virtual server to spawn within thix box that may need more resources than Amahi.

For perspective, my setup just gained a 2TB drive to the Proxmox 'local' storage LVM partition to take on more shares storage and additional storage for larger VM drives. It's easier to resize virtual drives than it is to attached and balance new drives to VMs.
My Proxmox OS lives on a 74GB raptor (got it from a friend so I figured, 'why not'), and the my 'local' storage partition just became my primary storage place for all my VM drives and containers. LVM is a really elegant solution for VM storage.
You can have 2 separate virtual drives for your Amahi to seperate the OS drive and the Amahi shares drive.
for example, a 20GB drive for Fedora root/swap/home/boot; and a 1TB drive (exclusive hardware or as a virtual drive) for /var/hda/files.

As you attack this, check the wiki for overview of operations to go thru. Post in here if you need help too.
SgtFoo
HDA: VM inside oVirt FX-8300 95w (2 cores for HDA), 32GB RAM (2GB for HDA)
My PC: FX-8300, 16GB RAM, 3x 1TB HDDs, Radeon HD6970 2GB video; Win10 Pro x64
Other: PC, Asus 1215n (LXLE), Debian openZFS server (3x(2x2tb) mirrors)
Modem&Network: Thomson DCM475; Asus RT-AC66U; HP 1800-24G switch

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby bigfoot65 » Mon Oct 07, 2013 11:22 am

I have been playing with Proxmox for quite a while. I think I have it working pretty good for my requirements. Simple Ubuntu Amahi 6 with 3 physical drives dedicated to Greyhole.

Maybe what we need is a more in depth tutorial of how to set it up with Amahi and potentially Greyhole. I would be interested to see a step by step account myself, more elaborate than the wiki guidance for newbies.

I also was trying to set up to subnets with two network cards, but that has not went well. Trying to get them to talk to one another is a bit of a pain without second router.

Guess that is why they charge for support. It can be quite complicated depending on what a person wants out of it. I am sure it could take many hours to document the entire build and configuration.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

User avatar
sgtfoo
Posts: 419
Joined: Sun Jul 18, 2010 8:27 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby sgtfoo » Mon Oct 07, 2013 11:44 am

More tutorials can be made, but for each person's wants and needs, there are particular changes to be made to the process.

Using virtualization of any kind with any server system demands that it not be a novice user, because in many cases, you're not following defaults for much of the setup/prep/install stages.
On the other hand, keeping to a bare-simple "VM-with-OS-and-storage" within a virtual system is pretty easy on its own and it easy to take on even if it's a novice at work.

When users toss in RAID, it just means understanding how to setup RAID with Proxmox or VMWare, which is outside the scope of Amahi altogether. Similarly analogous to adding in LVM or unique network structures.

Myself and other users have documented a number of methods for mid-level Amahi users who understand VMs to attack without a problem, while remaining in the Amahi realm for this forum and the wiki. Going into deciding on external or iSCSI setups is WAY beyond what Amahi entails.... better suited for reddit, or a Linux storage subforum in linuxquestions.org.... or something like that.

As for separate sub-nets from multiple nics... I'm 98% sure it can be done.... and it's worth looking at.... lemme work on it a little.
The first thing I can think of is running an instance of a Vyatta router... http://www.vyatta.org/
SgtFoo
HDA: VM inside oVirt FX-8300 95w (2 cores for HDA), 32GB RAM (2GB for HDA)
My PC: FX-8300, 16GB RAM, 3x 1TB HDDs, Radeon HD6970 2GB video; Win10 Pro x64
Other: PC, Asus 1215n (LXLE), Debian openZFS server (3x(2x2tb) mirrors)
Modem&Network: Thomson DCM475; Asus RT-AC66U; HP 1800-24G switch

iluciv
Posts: 17
Joined: Fri Apr 23, 2010 3:31 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby iluciv » Sun Oct 13, 2013 4:30 am

Thanks for your replies I really appreciate your input. I've had a fair amount on so I've not been able to (or had the mind space when free) to sit down and work on the server.

Your comments have made me realise that I'm perhaps approaching the problem incorrectly or just not understanding some of the concepts correctly (most likely both)

bigfoot65 with regards to your 3 drives with greyhole did you set this up post proxmox install? by that I mean did you install proxmox on a drive then install the amahi VM and add the 3 physical drives during that vm install? or where the 3 physical drives for storage also part of the the proxmox lvm? This is where I'm having an issue. as I think getting it to an install like your own to start off with is what I'm after. I can then create other VMs and point them to that storage.

I can't afford any more drives at the moment for storage so I'm just going to keep reading up till I've got the understanding to get this done properly. I'll gladly document it as I go and see if some sort of useful guide comes of it also.

I think I might focus on getting a better understanding of the proxmox layer first as I think that's going to save me a bit of grief.

Again thanks for your input really great :)

User avatar
sgtfoo
Posts: 419
Joined: Sun Jul 18, 2010 8:27 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby sgtfoo » Sun Oct 13, 2013 7:27 am

If you need to understand Proxmox first (which makes sense) then readup in their forums and also try out proxmox within a VirtualBox VM on your own desktop PC beforehand and you can mess around with it.

Proxmox is a hypervisor OS, so in terms of layers.... it's as follows if you want to go with baremetal virtualization:

1. hardware
2. OS - Proxmox or VMWare or Xen or RHEV or open-node, etc
3. VMs within #2

... As such your installation follows the same sequence.
SgtFoo
HDA: VM inside oVirt FX-8300 95w (2 cores for HDA), 32GB RAM (2GB for HDA)
My PC: FX-8300, 16GB RAM, 3x 1TB HDDs, Radeon HD6970 2GB video; Win10 Pro x64
Other: PC, Asus 1215n (LXLE), Debian openZFS server (3x(2x2tb) mirrors)
Modem&Network: Thomson DCM475; Asus RT-AC66U; HP 1800-24G switch

User avatar
bigfoot65
Project Manager
Posts: 11924
Joined: Mon May 25, 2009 4:31 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby bigfoot65 » Sun Oct 13, 2013 9:27 am

bigfoot65 with regards to your 3 drives with greyhole did you set this up post proxmox install? by that I mean did you install proxmox on a drive then install the amahi VM and add the 3 physical drives during that vm install? or where the 3 physical drives for storage also part of the the proxmox lvm? This is where I'm having an issue. as I think getting it to an install like your own to start off with is what I'm after. I can then create other VMs and point them to that storage.
I did install Proxmox before adding any drives. Installing Amahi before or after adding drives did not matter. When you create the Amahi VM, until you add the extra drives in the configuration, it does not see them.

Once I had that all set up, there is a command line that I needed to do on Proxmox to get the drives recognized by Amahi. Using hda-diskmount did not work for me so had to do manually add and mount them. It was a pain, but following guidance from sgtfoo for setup made it easier.

I just plugged in the drives, so don't think they became part of the LVM. They are not mounted on Proxmox, only Amahi if that makes sense.
ßîgƒσστ65
Applications Manager

My HDA: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz on MSI board, 16GB RAM, 1TBx1+2TBx2+4TBx2

olson
Posts: 30
Joined: Mon Dec 03, 2012 7:57 am

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby olson » Wed Dec 04, 2013 2:46 pm

I went with Xenserver instead of proxmox (for lots of reasons, but xenserver is opensource and has better documentation all over the place. Plus there is a lot of bad blood around the web for proxmox).

It is similar to proxmox in that is likes to use LVM for its storage, but with Xenserver, you can decide to NOT use LVM and use normal ext partition with VHD instead (this is the way I went). Here is my setup, I think you can do a similar thing with proxmox (I don't your Drive 1 can do VHD in proxmox though):

Old Dell 530 with intel q6600 and 4 drives, 6gb ram and 5-7 VM's running (these ones are always running: Windows XP, Amahi 6, Mythtv, youtrack VM, development vm):

Drive 1: Xenserver OS + VM Storage
This one is split into 2 partitions, xenserver (linux) partition and VM storage partition. Normal xenserver installs LVM for the VM storage, I have a script that converts it to a normal partiion. This way I can use that partition for greyhole storage (if I want) as well has VM storage.

The best is for this drive to be a small fast drive, possible 2 drives in hardware raid (not in my setup, but it would be better). That way this is only used for hypervisor and VM storage. The rest of your drives would be for data storage.

Drive 2-4 are all normal fully partitioned drives.
These are passed through (NOT using VT-D, my hardware doesn't support VT-D and I don't care about it). I assign these drives to my Amahi VM and it has direct access to it. I can unmount the drive, pull it, and stick it in any linux box and can read it.

Drives 2-4 are in my greyhole pool and snapraided (google snapraid).

Notes:
The issues w/ xenserver and passing drives directly to the VM is that you can't take snapshots (which is what a lot of VM backup software uses for backups. I just do a normal linux type of backup of this VM).

My windows xp VM hosts alike xenserver backup software and does backups of all my VM's (except my amahi one) to my greyhole samba share.

I could switch to proxmox and back w/o much problems since my data is all right on the drives and I'm not messing w/ LVM stuff.

Some of my vm scripts to help speed things up:
https://github.com/that0n3guy/VM-Template-Scripts
https://github.com/that0n3guy/Xenserver-After-Install

Hardware Note: Get something with ECC ram. If your storing lots of data (you have lots of drives), ECC ram is imporant if your data is important. My next setup will have ecc for sure.

jonathankonrad
Posts: 136
Joined: Sat Jul 25, 2009 1:42 pm

Re: Server hardware queries for New Amahi 7 server-Proxmox

Postby jonathankonrad » Sat Dec 14, 2013 12:18 pm

I am also about to embark on a Hypervisor setup and have similar questions, but I'm not really sure of what I am reading. I had imagined this setup;

1 - 3TB 7200 RPM drive for Hypervisor OS and all my VMS
3 or more drives for Data storage. Either RAID or Greyhole (currently using 3 x 2TB in Greyhole with Amahi 6).

My question here is, should I change to;

1 - 128 GB SSD for Hypervisor OS
1 - 3 TB drive for VMs
3 or more drives for data storage.

I happen to have the SSD, but since this runs all the time, and all the data lives off the Hypervisor even the VM drives, would I really gain much by using an SSD here? (Always other projects that could use that SSD!). Thanks for any help.

Who is online

Users browsing this forum: No registered users and 13 guests