In 1536 Louis X, Duke of Bavaria laid the foundation stone for a new residence in the inner city of Landshut. It was begun in German Renaissance style under the architect Bernhard Zwitzel from Augsburg; this palace is today known as the "German building" (in German: Deutscher Bau). During a journey to Italy the duke got the inspiration for an additional palace.
Behind the German building, close to the river Isar, the so-called "Italian building" (in German: Italienischer Bau) was constructed from 1537 to 1543 in Italian Renaissance style with a spacious courtyard and the palace chapel. It was modeled in particular after the Palazzo Te in Mantua, and was the first Italian style palace erected north of the Alps. Both buildings were connected by two wings. The paintings in the rooms were created by the Germans Hermanus Posthumus, Hans Bocksberger the Elder and Ludwig Refinger, while the stucco was done by Italian artists.
When Count William of Birkenfeld-Gelnhausen, who became later the first Duke in Bavaria, resided in the palace from 1780 onwards the facade of the German building was altered in French Neo-classical style and the so-called "Birkenfeld Rooms" were constructed. These rooms were decorated again with early wallpaper, when Crown Prince Ludwig lived here in the early 19th century, in the course of his studies in Landshut. These rooms are today a part of the Residence Museum, together with the halls of the Italian building.