I have a four year old and a four month old, and my girls keep me awake all day (when I try to sleep; I work graveyard shift). My wife doesn't get any sleep at night, alone with the kids, and I don't get any during the day, in a full house with noisy ones. It's got it's pleasant moments, though, and I would much rather lose sleep than not have my lil ones.
Diapers don't bother me, I've been through that. The spit ups don't bother me, either. I never had any free time before the kids (full time work and full time college) so that hasn't really been a change.

Therefore, my vote goes with lack of sleep.