The earliest written reference to the Old Castle of Celje dates back to 1323. The original design only included a Romanesque palace and the walls.

It was given its today's scope and appearance by the Counts of Celje, the most renowned and notable noble dynasty seated in the area of present-day Slovenia. Their reign and power had a long reach and they had a substantial impact on the political fate of the Central European region. The first Count of Celje to move to the Celje Old Castle with his wife and four children was Friderik I. After moving in, he started to refurbish the castle into a more comfortable dwelling. In the following decades, they extended the curtain wall to include the eastern inner ward, while the four-storey guard tower (the Friderik Tower) took over the main defensive role. Herman II used this tower a dungeon as he locked up his own son Friderik II, after whom the tower is named, due to Friderik's forbidden love for Veronika Deseniška After the demise of the Counts of Celje, the castle was inherited by the Habsburg dynasty who left it to their burgraves.

A modern tourist info centre and café Veronika were added during the most recent renovation, making the castle an appealing tourist sight that offers stunning views of the city and the surroundings. The tourist info centre offers promotional brochures, information about the castle, Celje and its surroundings, and other sights and events in the town.