The harmonic content of a sound is a big component of how we hear it, but the articulation over time of the sound also plays a significant role. This is referred to as the ADSR (Attack Decay Sustain Release) envelope. I would also imagine that each harmonic of a sound has its own slightly different ADSR envelope, so that harmonic profile would change over the duration of the sound.