The SSDs provide iops for everything that needs them. Obviously 450MB/s write is wasted for stuff like streaming music or books or whatever so that all goes directly on the large slow storage. Most of the SSD storage is there to serve the oVirt (kvm) cluster, which is my desktop and a couple of servers right now. Everything is virtualized including my desktop, because as much as I love Linux I don't love making music on it, so Windows it is. The VMs use shared storage (the SSDs) so no data goes away unless I flush it and the VMs are set to mount and use the networked ZFS store for writing audio.
The 10G connection is what makes it possible. Recording audio over the network in real time is pretty hairy (much like network video editing) and standard ethernet gets a lot of dropouts. I actually rebuilt my storage from a three node Ceph cluster to the current ZFS setup because Ceph, while most robust, isn't really performant the way ZFS is at the scale I was running. I may play around with a Gluster setup at some point as it seems more suited to a home setting than Ceph is right now. I moved my SSD mirror from ZFS to btrfs at the same time. It's all working ok thusfar.
The SSDs and ZFS array all live in the same machine for right now, so I guess that's my NAS. I decommed one of my Ceph cluster to act as an onsite backup and then just stuffed everything else into a single case for the time being. That means there's basically no latency between the fast storage and the ZFS array so I can mirror one to the other without an issue. The SSDs are only 1TB each (so 1TB mirrored) and there's currently ~30TB on the ZFS array so as long as I clean up my messes every now and then space isn't too big an issue. Everything gets incrementally backed up to Amazon Cloud Drive nightly for a little extra security.
The snapshotting is currently just set to every 15 minutes. The way ZFS snapshots work is that they're really just a changelog of blocks that differ from the time of snapshot. So the initial snapshot doesn't actually take any extra space (as actual blocks = snapshot blocks) and only grows as those blocks change (which means that while the computer isn't doing much those snapshots don't grow in size, which is nice). It blows up when I'm writing a lot of data (like recording audio) so I occasionally have to flush the older snapshots. I've got a little script I can run if I want to make a snapshot on the fly. I'll use that when I think I'm doing something really stupid to a song and might want to go back. ZFS will also let you clone data if you want a more permanent snapshot.
If I'm being honest I really just did this as a proof of concept because I had the equipment and I wanted to see if it'd work. It does, but it's a bit janky - I've had the 10G cards stall mid-stream and lose data or crash the network, I've had some weird quorum issues with oVirt and I've had all kinds of things go wrong getting the audio stuff to virtualize. The plan was initially to make it work and then tear it all down and go back to something more reasonable (like a non-virtualized desktop) but it just seemed like a lot of trouble to undo. Now that I've got my storage pretty well set I'm likely going to tear down and tinker with the actual cluster and get something a little more standard.
And that's my setup. Ridiculous, expensive, pretty fun, moderately effective, way more trouble than it's worth. As much as git makes my tear my hair out, it's probably easier than this.