Although I bought the ML110 G5 a few months ago, I have only been using it as a "play" machine. Earlier this month, Ebuyer got the ML115 G5 back in stock, so I took the opportunity to snap up one of those, along with 2 x 500GB disks, a 4GB RAM upgrade, 8 port Gigabit switch and a 4GB USB key drive.
You see, this is the plan:
The ML110 G5 will be the "production" server, running OpenSolaris 2008.11and playing the role of a storage server along with some core infrastructure services. I took the 250GB disk out of the ML115 and put it in the ML110 and mirrored the existing 250GB disk. I also added the 2 500GB disks in a mirrored pool. All of this is using ZFS for resilience and only takes a handful of commands to setup.
The OpenSolaris server is now running my print server software (CUPS) and works with the Linux machines, the Mac and even T's Vista PC. I'm running a small BIND DNS server for keeping track of the internal machines, and have a number of NFS shares setup for file serving, an ISO store and (in the near future), a VM datastore. The beauty of ZFS is that in addition to serving filesystems using NFS (and CIFS using the new kernel based service), other filesystems can use the ZFS technology thanks to zvols (essentially block devices in a zpool that can be shared using iSCSI and FC).
The 4GB USB key drive has VMware ESXi installed on it and the ML115 boots this. The USB port is internal, on the motherboard, so everything is nicely integrated. The actual VMs will be on the storage server - initially using NFS, but potentially using iSCSI in the next OpenSolaris release as well. Sounds like a SAN at home? Yep! :-) (this is why I've upgraded the switch to gigabit).
One of the things I'm finding with OpenSolaris is that although I've been using Solaris for years, there are a lot of changes (that presumably will make it into Solaris 11). The biggest one I've hit so far is the Image Packaging System (IPS). This appears to use a network repository for installing software (so no local .pkg files) and creates some new differences with Zones (they are now branded as "ipkg" and I think I've lost the ability to create sparse root zones in this release).
Still a lot to do and play with, but OpenSolaris certainly seems very feature-rich, and nothing else currently provides stuff like ZFS.
I might even have a look at getting xVM installed as well, but that might take a bit longer...