Well in this case don't use Terrains and associated types.
If you are using a mesh, be sure that the mesh is on the right side (if the mesh is reversed it may induce unexpected behavior)
For the behavior difference between Unity and the Oculus Quest a bit more details would be better.
For instance, your GameObject may be attached in some way to your player.
Thus you won't see any changes in Unity as the OVR is not attached to the HMD whereas it is when it runs on the Quest.
Mathias
