This database provides you with all the information you need. Once you type in your zip code, you will be greeted with a page that shows you how many contaminants in total were found in your local drinking water, and whether any of those contaminants exceed the EWG's health guidelines.
You can also learn more about each of the contaminants found in your local drinking water, what they mean, and what concentration was found. What you will likely find is that many of the contaminants found in your water aren't even legally regulated.
If you're feeling freaked out by the idea of contaminants in your water, there are definitely solutions available — and they do not include buying tons and tons of plastic water bottles.