Zinc chloride toxicity (cf. the link to 1942 operations) is acute; it appears to be the result of hydrolysis to zinc oxychloride and hydrochloric acid, the latter of which will burn mucous membranes, etc. Zinc contamination is bad for aquatic life (high concentrations may affect the embryonic development of various species) but it's fairly harmless to humans when ingested, hence the market for zinc gluconate supplements.
In the mid-Atlantic, zinc contamination is associated with vulcanized fiber manufacturing (which used zinc chloride baths to fuse sheets of paper into a sort of primitive plastic) and smelters like the one at Palmerton, or Monaca in western Pennsylvania. IIRC, zinc ores tend to occur as sulfides, so smelting them also released corrosive sulfuric acid vapors, as well as exciting and nastier co-occurring metals like cadmium and lead. It was impossible to remove all the contamination from the face of Blue Mountain at Palmerton due to the terrain, so they actually wound up airdropping a bunch of soil and turning it into a sort of rocky savannah with relatively metal-resistant flora. One Appalachian species (wild bleeding-heart, S1 or S1.1 in New Jersey) apparently thrives under these conditions and is quite abundant at both smelter sites. See
this excellent report for more than you ever wanted to know about zinc smelting in the mid-Atlantic.
So I wouldn't be too worried about the health effects of zinc exposure at the site. It's possible that there were other contaminants from smaller-scale things (e.g., cleaning equipment with a chlorinated solvent and dumping it on the ground) but Bob's suggestion that any peculiar sensations were due to anxiety is probably the most reasonable explanation.
Chemicals percolating through the soil and contaminating the aquifer make for big and messy cleanups, but walking around a site like that is not likely to cause exposure. That said, I'd probably try to avoid bringing home dust from sites contaminated by the nastier heavy metals (lead, cadmium, nickel come to mind) or by asbestos fibers. (Well, OK, I
did visit the tailings dumps at an asbestos mine once, but those are pretty safe as the asbestos was thoroughly separated by flotation before the tailings were dumped--when the environmental people have pictures of themselves scurrying over the piles without moonsuits, it's probably OK.)