If this question is crossing your mind, it's going to cross other buyer's minds as well. So yes, it could make the home slightly harder to sell later. Although, a mile away is reasonable, It would not bother me as an owner. Can you see the prison from your house or yard? If so, that would hurt the value more.
I wouldn't say it's much of a safety issue. Prisons are generally pretty secure, and who wants to commit a crime with a large population of policy officers around. Just my opinions.... more