In 883 a charter of Conques mentions a " ministerium calvomontense", a religious place with an ecclesiastical service, which concerns the St Michael chapel in the first walls of the castle. The date of building of this castle goes back to IXth or WIIIth century.

The first known lord is Hugues de Calmont who lived in the middle of XIth century. He made donations of Lévinhac church to Aniane abbey and of Perse monastery to Conques abbey.

The lineage of Calmont's lasted until 1299 with the Rodez bishop Raymond de Calmont's death. The castle passed on Castelnau-Bretenoux's family, then in 1530 on Clermont-Lodève's.

This situation does it an exceptional watching point on the whole valley.

Castle had been surrounded in XVth century with an outside wall equipped with 8 towers. It was built in basaltic freestones and was 180m long with 1m50 of thickness walls.

These walls were probably built between the wall of the castle and first walls too much long which surrounded ruins of building still visible, in particular the chapel.

the lower floor

This floor consisted of the kitchen, situated under the dining room, with a freestones chimney and probably an oven. In the western end was a room used like storeroom and lumber room. In eastern end was a room with access to the tank.