The first written mention of the castle of Potštejn is from 1259, 1287 or 1295. The castle was conquered only once in history, in 1339 by Charles IV before he became king. The village below the castle was first mentioned in 1350. The golden times of the castle were after 1495, when it was bought by Vilém II of Pernštejn, who had rebuilt it.

In the 17th century, economic decline came, the castle fell into disrepair, and the entire Potštejn went bankrupt. In 1746, the estate was bought by the nobleman Jan Ludvík Harbuval Chamaré, who had built here a château. After the House of Chamaré died out in the mid-19th century, Potštejn was acquired by marriage by the Dobřenský of Dobřenice family. They owned it until 1945, when their properties were confiscated.

The renewed Unity of the Brethren was founded in Potštejn in 1870.