Association of American State Geologists (AASG) have put together a clickable map that allows you to locate the website hosting the geological survey of your state.
Most of these state websites will have links to "Educational Resources" or "Publications" which could be worth exploring further. Some of the educational resources may be aimed at K-12 but others will be appropriate for the educated layperson.
The University of Texas at Austin maintains a good website of virtual field trip guides around the US. Its coverage of the Mid-Atlantic region is unforunately not as strong as its coverage of other regions.
USGS maintains a website called Science in Your Backyard which provides updated regional information on a variety of topics. Depending on your topic, this site may be useful to you. They also have a page on general science topics and one devoted to discussing Geologic Maps and Mapping.
The USGS also has a good website devoted to discussing Geologic Maps and Mapping, one that focuses on minerals, and on water resources.
The United States Department of Agriculture has created a list of publications that pertain to state soils.
The U.S. Energy Information Administration's website is a great place to find energy maps, facts and data for each state in the nation.