Representation learning (RL) is a universal technique for deriving low-dimensional disentangled representations from high-dimensional observations, aiding a multitude of downstream tasks. RL has been extensively applied to various data types, including images and natural language. Here, we analyze molecular dynamics (MD) simulation data of biomolecules in terms of RL to obtain disentangled representations related to their conformational transitions. Currently, state-of-the-art RL techniques, which are mainly motivated by the variational principle, try to capture slow motions in the representation (latent) space. Here, we propose two methods based on alternative perspective on the disentanglement in the representation space. The methods introduce a simple prior that imposes temporal constraints in the representation space, serving as a regularization term to facilitate capturing disentangled representations of dynamics. The introduction of this simple prior aids in characterizing the conformational transitions of proteins. Indeed, comparison with other methods via the analysis of MD simulation trajectories for alanine dipeptide and chignolin validates that the proposed methods construct Markov state models (MSMs) whose implied time scales are comparable to state-of-the-art methods. By coarse-graining MSMs, we further show the methods aid to detect physically important interactions for conformational transitions. Overall, our methods provide good representations of complex biomolecular dynamics for downstream tasks, allowing for better interpretations of conformational transitions.