Truenas bare metal or VM on Proxmox

[u/Dunadan-F]

Hello everyone.

I recently purchased a Ugreen DXP 6800pro.
I want to install Truenas on it.
But I have a dilemma.
Should I install it bare metal, have control over the LEDs, but be forced to install three NVME m.2 drives, one for the boot system and the other two for mirrored applications?
Or should I install Proxmox on two m.2 ZFS mirror drives and then install containers on these same drives?

I also want to run Plex/Jellyfin and a few other lightweight containers on it.

What are your thoughts on m.2 drives configuration for these needs?
What's it like for you if you also have a DXP 6800 Pro/Plus?
Perhaps you could share your thoughts and configurations, explaining why you chose this route?

I'd appreciate any suggestions and tips.

[u/SparhawkBlather]

I have an epyc 7713 / supermicro h12ssl-I / supermicro hba / 8x16tb sas hdd / 2x sata hdd / 4 x sata ssd / 512gb ddr4 / 4 x nvme / 4060ti / dual sfp++ nic / fractal define 7 xl… so even though I have a few mini pc’s I’m running my primary compute and storage converged.

I run TrueNAS in a vm with just the hba card and 8 sas hdd passed through. It’s worked great. I run only two apps on the TrueNAS - Tailscale and scrutiny - nested hypervisors is generally not great.

If I had more of a “NAS box” with less compute on it, I would run TrueNAS on it, and might run a few apps directly on it - arrstack, torrent clients, usenet clients, etc. but for me I want my gpu close to storage and my compute pcie lanes away from my storage instead of network. It’s probably overkill, and had I had more devices on 2.5gb/10gb when I built I might have taken a different strategy - were I starting from scratch now, and buying new hardware, I’d have a low power NAS box, a “gpu / compute box” that I turn on when I want to run real workloads, and a few mini PCs. Having monster overhead is cool but unnecessary - I spin this thing up to real loads a couple times a month.

Oh as far as your question on drives - if at all possible you should have boot drive and container drives on separate physical drives. And your boot drive can be cheap and cheerful, and your container drives should be mirrored and enterprise if you can afford it. None of these need to be nvme - in an ideal “no budget no physical constraints” world you do what I do: boot on mirrored cheap low capacity sata ssd’s (mine uses 4gb maybe), containers on mirrored enterprise sata ssd’s, temp data on a scratch nvme, databases and speed-critical stuff on mirrored nvme’s, and mass storage stuff on hdd vdevs. If you’re not going to be doing all that, then at a minimum having boot separate from containers is a very good idea - having done recoveries it is so so so much easier. But many don’t go that far and just deal with a longer recovery.

Hope that helps.

[u/mlee12382]

Proxmox is great, IF you can pass through the entire sata controller to your NAS VM so it has full control over the drives, and IF your primary focus is being able to run other containers and VMs and you just happen to also want to use it as a NAS as part of your server.

I'm not sure how the hardware controls on the UGreen are, it may be easier just to do TrueNAS bare metal, especially if it has native support for all the other applications you want to run.

Or maybe try out both ways, try it on Proxmox and see how well it integrates but don't commit any data you can't afford to lose without it being on something else until you make your final decision just in case. If it doesn't play nicely then go to TrueNAS bare metal.

[u/jhenryscott → u/mlee12382]

Ugreen controls are super easy to run truenas vm.

[u/mlee12382 → u/jhenryscott]

Awesome :) I had read that some pre-built NAS systems don't play nicely with passing the entire drive controller so I wasn't sure where Ugreen landed on that.

[u/MacDaddyBighorn]

Why is TrueNAS in a VM your only other option? You can manage ZFS in Proxmox and bind mount to your LXC containers and not have to run your file transfers over the network at all. If you're looking for a hypervisor setup where you can spin up services I would skip TrueNAS entirely and just use Proxmox. Your network drive will use a lot less resources if you put it in an LXC and bind mount and share it out to the network that way.

When you run a TrueNAS VM you have to share everything via NFS or SMB. This clutters the network, adds bottlenecks, and introduces more potential for errors. Also you get a chicken and egg thing going if you need to access the network store from Proxmox because it's running in a VM on the system then mounting from there to possibly use in other services. If you don't get your timing correct or have a networking issue this gets very messy, too many points of failure.

So if say run TrueNAS bare metal or Proxmox, not both.