What is FML? I recently spent a little time starting to put together my own USGS topo for the Northeast US, but it's a huge project and I got discouraged. If you just want New Jersey, NJGIN has that. We discussed this a few years ago, and IIRC your objections were that NJGIN's servers were slow and they don't switch to the 100k topo when you zoom out.
I think NJGIN servers work very well now. And there should be a way to combine their 100k and 24k WMS layers to accomplish what you want. In Leaflet it's very simple to create a layer group and configure the zoom levels for each map. Isn't this possible in the Google API? If not, then this open source package should be able to do it:
https://mapproxy.org
If all else fails, I can put a tileset together for South Jersey that you can host locally at NJPB.