If I died, would they remember me at all?

if I were to die in childbirth, would my not yet three year old son remember me at all, or would it be before real memory set in? would there be lasting psychological consequences for him if the baby survived, would it have any issues? or would they be young enough to not be terribly scathed by it? Its so common now for families to only have one parent, I dont even think it would be weird.