I think your question is arising from the fact that linear momentum only depends on the mass of the object and not on the shape of the object.
The intuition to understand why angular momentum (another word for moment of inertia) depends on the shape of the rotating object is by observing a spinning skater. Observe how the skater starts spinning faster as they bring all their extremities close to the axis of rotation. Why does that happen? Is extending the intuition from linear momentum to angular momentum broken? If there is no external force acting upon the body, why is there a change in rotational speed when the same argument tells you that there cannot be any change in linear velocity in the absence of an external force.
The answer is that angular momentum, depends on the shape of the skater. For the angular momentum to stay constant if the shape of the skater changes there has to be a change in the skater's rotational speed.
A more mathematical understanding can be gained by imagining every object to be made up of tiny components where their masses add up to the total mass of the object in question.
Let's first understand linear momentum. Linear momentum of each individual component, i is given by (m_i)*v where v is the velocity of each component. The important thing to note here is that v of all components is exactly the same (this is a solid object). Therefore, total linear momentum = SUM_i (m_i * v) = SUM_i(m_i) * v = m*v. Therefore, linear momentum of the object depends only on the mass of the object and not the shape of the object.
On the other hand, the combined angular momentum of the object is the sum of the individual angular momentum of each component of the object. The angular momentum of each component is "m_i * omega * r_i" where r_i is the distance the component i is from the axis of rotation, and omega denotes how fast the object is rotating about the axis. Therefore, total angular momentum is given by SUM_i (m_i * omega * r_i) which cannot be independent of the shape of the object.