Although optical imaging of electron beams and X-rays from medical linear accelerators (LINAC) is a possible method for dose distribution measurements, it has been limited to two-dimensional (2D) projection images. For the precise measurement of an optical image of electron beams and X-rays, three-dimensional (3D) imaging is desired. To measure 3D dose distributions, we conducted imaging of electron beams and X-rays using a plastic scintillator plate set in a water phantom. When this plate was immersed in the water phantom, irradiation with electron beams or X-rays was carried out from along the plate's sides. Optical images of the scintillator plate were acquired using a charge-coupled device (CCD) camera from the side during irradiation with electron beams and X-rays. Measurements were conducted at 6 MeV, 9 MeV and 12 MeV for electron beams and at 6 MV and 10 MV for X-rays. The imaging system was set on the bed of the LINAC and moved at 10-mm steps perpendicular to the beam direction to acquire a set of sliced optical images of the beams. A set of these sliced images were stacked and interpolated to form 3D optical images. For the 3D images, after the correction of the Cherenkov-light component in the images, the relative depth and lateral doses were evaluated. From the relative depth doses of electron beams, the half-value depths could be evaluated within an error of 1.3 mm. Lateral widths could be evaluated within an error of less than 2 mm parallel to the plastic scintillator and less than 6.5 mm perpendicular to it. From the relative depth doses of X-rays, the average difference between the measured value and that by a planning system was within an error of 2 %. Lateral widths could be evaluated within an error of less than 0.68 mm parallel to the plastic scintillator and less than 2.6 mm perpendicular to it. We confirmed that 3D imaging of electron beams and X-rays using plastic scintillator plate is feasible and is a promising method for measuring dose images at any position.