The Chazy Reef Formation, extending from lower Quebec to northern Alabama, is known worldwide among paleontologists as the oldest fossil reef in the world. It was formed during the Ordovician Period, some 450-480 million years ago in a shallow tropical sea (this part of North America was below the Equator at the time). Several ages of reef buildup are visible in the various layers exposed by quarrying operations here over the years.
Clearly visible in the quarry walls are fossil remains of stromatoporoids (an extinct ancestor of the sponge), one of the builders the Chazy Reef. The quarry is designated a National Natural Landmark by the US Department of the Interior.
Further description can be found at this Vermont Geological Survey site.