Let X be a complex manifold and L be a holomorphic line bundle on X. Assume that L is semi-positive, namely L admits a smooth Hermitian metric with semipositive Chern curvature. Let Y be a compact Kähler submanifold of X such that the restriction of L to Y is topologically trivial. We investigate the obstruction for L to be unitary flat on a neighborhood of Y in X. As an application, for example, we show the existence of nef, big, and non semi-positive line bundle on a non-singular projective surface.