It is difficult to select small targets in freehand distal pointing on large displays due to physical fatigue and jitter. Previous research proposed many solutions but not through the semantic meaning of hand behaviors. The purpose of this study is to investigate the hand behaviors, including hand shapes and motions in the Z-dimension between small and large target selections, by means of two controlled experiments. The generated results indicated that the following: (1) when interacting with a large display from a remote distance, users preferred to complete a low-precision task by the whole hand while using the finger in high-precision tasks. As a result, users moved small targets by the index finger gesture while moving large targets by open hand gesture; (2) selection of small targets led to hand stretching which might reduce the interaction distance; (3) the orientation had significant effects on hand motions in the Z-dimension. The relationship between orientation and hand motions in the Z-dimension is similar to a sinusoidal function; (4) mouse-like pointing had much impact on freehand interaction. Based on these results, some design guidelines for freehand interaction and design suggestions for freehand pointing were discussed.