Since their introduction, theories with higher derivatives are unavoidable and frequently arise in different contexts. Hence gauge symmetric higher derivative theories got their own importance. Hamiltonian formulation for higher derivative theories become necessary and is discussed in the first order formalism. Being a higher derivative theory, the massive relativistic particle model with curvature shows violation of a very well known rule. The analysis is done using the equivalent first order approach. Also this Hamiltonian formulation is applied to the minisuperspace cosmology following from the geodetic brane gravity model introduced by Regge and Teitelboim. The constraint algebra is shown to realize a truncated Virasoro algebra. The gauge generator containing the exact number of gauge parameters is constructed. Equivalence between the gauge and reparametrization symmetries has been demonstrated. Complete gauge fixed computations have been provided and formal quantization is indicated.