The “Grand Canyon of the Adirondacks,” a deep sandstone gorge near Keeseville cut by the Ausable River, open to visitors since 1870 and walked by galleries, bridges and rafts. Not a cave but real deep rock you climb down into, one of the oldest natural attractions in the country.
New York
See it on the map
Other places on the map within a couple of hours.