This is an issue that I had to resolve myself after finding no solutions for it.
The issue was that after removing a GameObject element from the ScrollRect's content, it would not clamp its normalized position until the user starts moving it again. This could lead to the ScrollRect showing empty space when it is showing the last elements and has deactivated the last element from the parent.
The best solution I have found is to
Example code:
if (isRemoving) {
Canvas.ForceUpdateCanvases();
scrollRect.horizontalNormalizedPosition = Mathf.Clamp(scrollRect.horizontalNormalizedPosition, 0f, 1f);
}