There has been much recent interest, initiated by work of the physicists Hatano and Nelson, in the eigenvalues of certain random, non-Hermitian, periodic tridiagonal matrices and their bidiagonal limits. These eigenvalues cluster along a "bubble with wings" in the complex plane, and the corresponding eigenvectors are localized in the wings, delocalized in the bubble. Here, in addition to eigenvalues, pseudospectra are analyzed, making it possible to treat the nonperiodic analogues of these random matrix problems. Inside the bubble, the resolvent norm grows exponentially with the dimension. Outside, it grows subexponentially in a bounded region that is the spectrum of the infinite-dimensional operator. Localization and delocalization correspond to resolvent matrices whose entries exponentially decrease or increase, respectively, with distance from the diagonal. This article presents theorems that characterize the spectra, pseudospectra, and numerical range for the four cases of finite bidiagonal matrices, infinite bidiagonal matrices ("stochastic Toeplitz operators"), finite periodic matrices, and doubly infinite bidiagonal matrices ("stochastic Laurent operators").