

In this case the movie version is closer to reality than your “realistic” version. Castles were not just built for defensibility. They have a strong representative aspect. Of course they made them as nice to live in as possible. The walls were covered in tapestries or wood panelling and many rooms were heated. For unheated rooms there were movable heaters (basically small portable fire pits).
Heres an example of a hall in a castle: https://de.wikipedia.org/wiki/Très_Riches_Heures#/media/Datei:Les_Très_Riches_Heures_du_duc_de_Berry_Janvier.jpg








You gave it exactly backwards. The castle is by definition the combination of residence and fortification. The purely military fortress only really developed after the medieval period.