Am i aloud to clean my duplex at night? is there a rule? we have neighbors below us.?

so when me and my boyfriend both get home at about 9:00 9:30pm we sometimes do some extra cleaning at home. we live in a 2 story duplex, with a married couple below us. and if we walk around at 10:00pm they start beating up here with a broom or pole or something. we have hard wood floors in the living room and it creaks a bit. is there a rule on how late you can clean your house or move stuff around? like if they beat up here more can we just keep cleaning? me and my boyfriend get a little scared, he's kind of a scary guy that yells up here to shut up and slams his front door like he's going to come up stairs and beat our door down.