Regarding cap rate, for a short-cut method, get the the sale price of a recently sold comp, find out the net operating income for that comp (the breakdown is usually on the listing sheet. An agent can look up comps for you on the MLS), then divide the net operating income by the sale price to get cap rate. Do this for several comps to get an idea of the average for the area. You can then divide the net operating income of a property you wish to purchase by the capitalization rate you got from sold comps in order to determine a property value. Conversely, you can multiply the asking price by that same cap rate in order to determine what type of income you would need in order to justify the purchase price.
As for whether or not being a landlord is worth the hassle? Well, it depends. Will you be living on the property? If so, keep in mind that your tenants' needs will often come before your own (if you are a good landlord). This means making sure the property is kept well, that you are available to answer needs, that you are mindful of providing a peaceful environment.... If you are not going to be on the property, these things will all still need to be provided - there or not. This may mean running to the building in the middle of the night when a pipe bursts, or hiring someone to do it if you can't.
If you want to be a good landlord and attract quality tenants, you will have to be willing to do all of this and more.
Purchasing an income property can be an amazing investment with huge potential for appreciation and cashflow (I know, I own and live in a multi-unit and have tenants), you obviously want to study the numbers and also reflect on whether or not you are willing to take on the "hassle'. After all, if even before becoming a landlord you already view being one as a hassle, you may want to reconsider.
I encourage my clients to think outside the box and jump into a multiunit property if they can because the income taken from the rents can be used to increase the buyers net income and this gives them more buying power. More power means bigger properties which will mean a better ROI on your downpayment. PLUS you can live in one and deduct the rest on your taxes as expenses.
You've just moved from a buyer to an investor all in one fell swoop