Get off your lazy ass and care for the children you created!

I swear, my upstairs neighbor annoys the hell out of me.. It's a very odd situation to begin with. From what we have gathered, I believe he recently went through a divorce and he moved in back in October/November. When he first moved in, we introduces ourselves and helped him bring in some furniture. But he only lives in the apartment on Saturdays.. literally only Saturdays. I guess that's when he gets custody of his kids because they come over Saturday morning and are here until Sunday morning.. He has a son who is about 4 and a daughter who is about 8. They are the loudest kids I have ever seen!!!
As I am typing this, they are currently running around upstairs and stomping like a herd of elephants! I am contemplating going up and knocking on his door, since this has been going on since he moved in.. We thought it would just take a while for the kids to adjust and get over the excitement, but nope.. Every Saturday this happens.. And every Sunday, I am woken up at the crack of dawn by the running and jumping and screaming..
Jesus, help me because I am about the lose my mind! It's a hard situation because if I go up there, I risk pissing him off and him just allowing his kids to continue to do what they always do.. But then again, that can't be much worse than what it currently is. Then the other side of me is telling myself to suck it up since they are only here on Saturdays. But seriously, is it fair to listen to a freaking herd of elephants pounding above me and making my walls rattle?!?
Ahhh.. Just when I get up the nerve and begin thinking about going up there to speak with him, the noise stops.. I just got up and glanced out the window and they have just now left. At least I'll have some peace and quiet for a little while.. but it's just a short fix. Maybe I will catch him when he arrives home again and can go outside and mention it to him "in passing". But reality is, by then my courage will be gone because I will not be annoyed because it has been quiet for awhile..