Hi Engin
I was speaking from personal experience about drawing the sword on the right side of the body

The armour isn't really a hindrance to drawing the sword either on the left or right side, it's not good practice to draw the sword inside the shield arm. And you certainly can't draw the sword over the shield.
The manica (segmented armour) on the sword arm might have been worn to protect against the Dacian
falx in the wars of the late 1st and early 2nd centuries, though IIRC the surviving examples (like nearly all Roman armour) are poorly dated. So it's possible it may have appeared earlier on.
All the best
David