The reason rifle hunting in NJ would concern me is that a shot, taken and missed at an upward angle, like from behind a blind in a low-lying spot, could send a bullet 1-2 miles downrange in an arc, possibly killing a person where it falls to earth. I understand that a level shot, unless taken extremely irresponsibly, will likely stop in the first decent sized tree.
There's no question that a high-powered rifle is a more effective way to take down game than a shotgun at any sort of range, although a 12 gauge slug has an advantage for very close range bear defense.
Many of the smaller states in the midwest and south have come up with a compromise, where only straight-walled cartridges can be used for deer hunting, like .30 carbine, .45-70, or .450 BM. They are larger caliber but lower velocity, so they dissipate their energy in a shorter distance.
I would guess the woodchuck hunting allowance is there for farmers to mitigate field damage and they figure on private property the danger to the public is less.