We consider the problem of implementing a Linear Quadratic Gaussian (LQG) controller on a distributed system, while maintaining the privacy of the measurements, state estimates, control inputs and system model. The component subsystems and actuator outsource the LQG computation to a cloud controller and encrypt their signals and matrices. The encryption scheme used is Labeled Homomorphic Encryption, which supports the evaluation of degree-2 polynomials on encrypted data, by attaching a unique label to each piece of data and using the fact that the outsourced computation is known by the actuator. We write the state estimate update and control computation as multivariate polynomials in the encrypted data and propose an extension to the Labeled Homomorphic Encryption scheme that achieves the evaluation of low-degree polynomials on encrypted data, with degree larger than two. We showcase the numerical results of the proposed protocol for a temperature control application that indicates competitive online times. CCS CONCEPTS • Security and privacy → Public key encryption; Privacypreserving protocols; • Information systems → Process control systems; • Computer systems organization → Embedded and cyber-physical systems.