You're How Old?

I recently ran into a couple who used to attend my family's church. All was well until the husband asked for my age. I wanted to say 24 but didn't think that was right. So I stopped and thought for a moment before correctly telling them 23. My drawn-out pause caused the wife to think that I didn't want to reveal my age. (Who cares when you're only in your twenties.) If they knew the real reason, they might wonder about me...