Yep, it's about using common sense and courtesy. I won't play my music before 12 noon and not after 10-11pm, unless I see the neighbours go out. I also won't have it on every day, or all day. It's usually every other day for a few hours. Sometimes I end up with it on most days but even then I limit to a few hours, or take a break and come back for more later in the day.
I think it's wrong to be playing music loud at 6am or past 11pm, unless your neighbours are doing the same or for some reason don't mind it.