You sure can!  On the Audience list, search for the contact email using the search bar at the top of your page. Hover over the contact and on the right hand side you’ll see a “suppress” link. Click on that and you’re all set.

To suppress many contacts at once, use this guide.