Growth of a fully oxidized aluminum oxide layer with low surface roughness on graphene grown by chemical vapor deposition is demonstrated. This is accomplished by the deposition of a 0.2 nm thick titanium seed layer on the graphene prior to the deposition of the aluminum under ultra high vacuum conditions, which was subsequently oxidized. The stoichiometry and surface roughness of the oxide layers were measured for a range of titanium and aluminum depositions utilizing ex situ x-ray photoelectron spectrometry and atomic force microscopy. These fully oxidized films are expected to produce good dielectric layers for use in graphene based electronic devices.