I found the following references to caves and dwellings of early settlers.
"The first settlement made in Evesham was made by William and Elizabeth Evans, who emigrated to this country from Whales . . . between 1680 and 1690. . ."
" . . .they were obliged to build a cave to live in, as there were no facilities for house building. They built their cave-dwelling near what is now Mount Laurel, and William died in it in 1728, . . ."
"These passengers having arrived late in the fall [of 1677] but few were able to build themselves log houses before the winter was much spent. During the interim they lived in wigwams built after the manner of the Indians."
". . . they arrived in Burlington in the year 1678, this being the first ship that ever was known to come so high up the Delaware River. Then they landed and made some such dwellings as they could for the present time, some in caves and others in palisade-houses secured."
"Most of the English and many others have their houses made of nothing but clapboards, . . . " c1679-1680
"At first the dwellings were either mere caves hollowed out in banks and protected at their entrances with boards, or the most primitive shanties imaginable." c1677
"On the other side of Craft's Creek a tract of five hundred acres was taken up by Thomas Scattergood, who with his family lived in a cave on the south bank of the stream, traces of which are still discernible. [1883]" c1670's
". . . there was no alternative but to dig a cave in the side of the small hill to live in." c1685
"Like many other settlers, his first home was in a cave, which he furnished in a rude manner for housekeeping. He lived with his wife many years in this cave, and in it was born Henry Jacobse, their son, who was the first white child born within the limits of the township." c1698
All of these reference Burlington County settlers (except the last which was from Little Egg Harbor which was once part of Burlington County).