I always figured fire deserved the most blame for clearing out remains of historic structures.
KCB, I think the allure of the pines (aside from the solitude and natural beauty) for history buffs lies more in the faded fingerprints of past times that are everywhere in the woods. Cellar holes, old bridges, the ghost of an ancient road bearing a name that has changed almost completely, yet hints at its former spelling, dried bogs whose turf walls can still be traced, slag from colonial-era furnaces, bricks, stones, enigmatic pieces of rusty iron.
One walk I can suggest would be to start at old Harrisville, where you can see some real ruins, and if you learn a bit of the tale tell an impressive story as well. That parking lot across from the glass factory used to be one of the most progressive towns in the pines, with squarely laid-out streets and gas lighting. From there walk the Martha Rd. north along the river to Martha Furnace, where you can pick up chunks of glasseous slag left over from bloom iron making in colonial times. It's only about 3 miles round trip.
Start here:
http://maps.google.com/?ie=UTF8&ll=39.66408,-74.521015&spn=0.005839,0.007532&t=h&z=17
And end here:
http://maps.google.com/?ie=UTF8&ll=39.682156,-74.513054&spn=0.011675,0.015063&t=h&z=16