The castle at day.
The Castle of Caspian or House of Caspian was home to the early Telmarine Dynasty (the "Caspians") of the Kingdom of Narnia. It was built by Caspian VI.
The castle was home to various Telmarine kings, of which included Caspian VI, Caspian VII and Caspian VIII, respectively (the first two names are speculation).
The castle at noon.
The last permanent ruler of the palace by the name of "Caspian" was Caspian IX, who was murdered and succeeded by his brother, Miraz.
Because Miraz was the final permanent inhabitant before it was abandoned or destroyed, the palace is sometimes called Miraz's Castle (being referred to as such specifically during the Narnian Revolution).
Afterwards, Caspian X may have lived in it briefly as king, before rebuilding Cair Paravel and making that his new palace.
The castle at night.
After the reconstruction of Cair Paravel was complete, Caspian moved into the rebuilt castle, and the Castle of Caspian was used by both the "Old Narnians" and the remaining "New Narnians" for a variety of reasons, but mostly to support their relationships.
The castle could have possibly been rebuilt from the remains of the White Witch's Castle, and was located at Beaversdam, though on the opposite bank of the Great River.
Trivia
In the Prince Caspian (Disney movie), the Castle was located on a large rock that might have been an Island.