As to correctly transform the Normal (which is a direction not a position) from Object space to World space, we multiply it with inverse of Model matrix/_World2Object matrix (in Unity).
Then why Tangent (which is tangent to normal and is direction as well - its not a position) is multiplied with Model Matrix /_Object2World (in Unity) to transform it to World space?
eg code : Code for reading Tangent space normal in shader :
o.normalWorld = normalize( mul(v.normal, _World2Object) );
o.tangentWorld = normalize( mul(_Object2World, v.tangent) );
o.binormalWorld = normalize( cross(o.normalWorld, o.tangentWorld) * v.tangent.w );
The normal vector is transformed with the transpose of the inverse model matrix from object space to world space (because it is orthogonal to a surface) while the tangent vector specifies a direction between points on a surface and is therefore transformed with the model matrix.
(Source: Wikibooks)
For a better visualization I drawed a little image:
The tangent is a direction on the surface and so if the y axis is squashed it does the same to follow the surface (so the model matrix is used).
The normal is a direction perpendicular to the surface and so if the y axis is squashed it moves the inverse way to stay perpendicular (so the inverse transpose model matrix is used).