However, if they need to do maintanence through your unit, it may be a design issue. It should not be designed that way when they build out the garden unit.
Longer answer: The association or management certainly can. Individual owners probably can't. You'd have to read your lease or (if it's a condo) your condo documents.
Management or the association may be required to provide reasonable notice . . . except in the case of emergencies, when typically no advance notice needs to be provided.
That information should be in your lease or in your condo documents.
Hope that helps.