We present temperature-dependent infrared spectroscopic studies of CaMnSb2, a proposed three-dimensional topological material. The low plasma edge in the reflectivity spectrum and small Drude component in the optical conductivity indicate a very low carrier density. The low-frequency optical conductivity is well described by the superposition of narrow and broad Drude terms. Several linear components have been observed in the lowtemperature optical conductivity, but none of them extrapolate to the origin, at odds with the optical response expected for three-dimensional Dirac fermions. A series of absorption peaks have been resolved in the highfrequency optical conductivity. The energy of these peaks agrees well with the interband transitions expected for the band structures from first-principles calculations. Intriguingly, the lowest band gap increases with decreasing temperature, mimicking the temperature evolution of inverted bands. Furthermore, our theoretical calculations demonstrate the existence of weak coupling between two Sb-chain layers results in the topological trivial surface states in CaMnSb2.