The Athens Department was founded 17 March 1872 by decision of the German Reichstag as the second branch office (after Rome) of the then Institut für archäologische Korrespondenz. Its inauguration was on 9 December 1874, the birthday of Johann Joachim Winckelmann. The institute thus was the second foreign archaeological research implantation in the capital of the Greek kingdom after the École française d'Athènes which had been founded in 1846.
Ever since 1888 the Athens Department has held its seat in the city's very centre, in a late classicist building in Phidias street (Fidiou 1), so named after the famous sculptor. The edifice was built in 1887/1888 on behalf of Heinrich Schliemann and according to the plans by Wilhelm Dörpfeld and Ernst Ziller. The German state then acquired the estate from Schliemann's heirs in 1899.
Next to recording and publishing ancient monuments according to material category, the Athens Department from its very beginnings was furthermore dedicated to the carrying out of archaeological fieldwork and the study of the history of architecture and topography. Well before the opening of the Istanbul Department in 1929, the Athens institute was active not only in Greece, but also in the western parts of Asia Minor.
The origins of the important excavations, which today still remain under the department's supervision, go back to the first decades of its existence. Among them are the excavations within the sanctuary of Olympia (since 1906, formerly the so-called Reichsgrabung 1875–1881), Tiryns (since 1905, previously excavated by Schliemann 1884–1885), the Kerameikos of Athens (since 1913), and the Heraion of Samos (since 1925, previously excavated by the Berliner Museen 1910–1914). To this number come the excavations in the sanctuary of Kalapodi launched in 1974.

After its WWI shutdown in 1916, the Athens Department was unable to resume normal operations before 1921. Its functioning during the Nazi period between 1933 and 1945 has been the subject of more recent research. After the retreat of the German occupying forces in Greece in 1944, the institute building was handed over to the Greek authorities. The subsequent reopening of the Athens Department in 1951 not least owes to the far-sighted support by Greek colleagues.
Today, the Athens institute is, much like the other foreign archaeological missions to Greece, part of a wider international community. The Athens Department fulfils with its outstanding scientific infrastructure a crucial role, not just for scholars from Germany or Greece, but also for specialists from other countries, regardless of whether they are stationed here on a permanent or temporary basis.