First hand information...We are avid boaters and members of the Miles River Yacht Club, which I highly recommend, but yes, you most definitely can swim in the Miles River and it's tributaries. We do it every year although as Jim said, the sea nettles get to be a problem later in the summer. If you are boaters, it is an easy run up to Kent Narrows with more great spots for socializing and dining.
St. Mikes, as it is know locally, is a great place to live!
Please contact me if I can be of assistance