The Academy of Fine Arts was founded on the private initiative of the imperial court painter Peter Strudel (1660–1714) and was officially approved by the imperial household in 1692. After Strudel's death, it was re-established in 1726 as the Public Imperial Court Academy headed by Jacob van Schuppen (1670–1751). The establishment in 1766 of an independent engravers' academy under Jakob Mathias Schmutzer (1733–1811) laid the foundations for the present-day Graphic Collection. Schmutzer's engravers' academy was incorporated in 1772 into the k. k. Akademie der vereinigten bildenden Künste (Imperial Royal Academy of Consolidated Fine Arts), merging the existing art academies in Vienna into a single institution, thereby enabling the collections to be systematically enlarged. A further painting collection, compiled without a specific strategy and intended originally for teaching purposes, was considerably upgraded to become an independent gallery of international standing through the acquisition in 1822 of a legacy from Anton Franz de Paula Graf von Lamberg-Sprinzenstein (1740–1822)—including Hieronymus Bosch's Last Judgment triptych, and works by Rubens, Van Dyck and other seventeenth-century Dutch painters. After moving a number of times within today's 1st district and as a result of the increasing shortage of space, a new building was constructed in 1871 on Schillerplatz, which was officially opened in 1877. After the First World War, the Academy collections became the property of the Republic of Austria.

Already in the years of the Austrofascist regime, the Academy was open to the ideas of National Socialism, and it was rapidly "aligned" after the annexation of Austria in 1938. On 12 March 1938, the Vienna NSDAP appointed a management board consisting of Ferdinand Andri, Wilhelm Dachauer and Alexander Popp, which immediately dismissed thirteen members of staff and remained in place until Popp was appointed rector in summer 1941. The Academy was now answerable to Department IV of the Ministry of Internal and Cultural Affairs headed by Kajetan Mühlmann, and from 1940 the Department for Promotion of the Arts of the Reichsstatthalter in Vienna. Regular courses were suspended from winter semester 1944/45, and the Academy was closed completely at the latest in January 1945 along with all art academies.

In 1938, the Paintings Gallery had around 1,900 works. To enlarge the collection, the provisional board and gallery director Robert Eigenberger, proposed in 1939 to participate in the distribution of art objects from expropriated collections, although no allocations of this type have been identified to date. In August of that year, Eigenberger was instructed to divide the works into categories A, B and C and, in view of the impending war, to arrange for the storage of category A works. In the four storage operations in 1939, 1942, 1944 and 1945, over 600 works, mainly from categories A and B, were stored at Stift Heiligenkreuz, Creditanstalt–Bankverein in Vienna 1, Schottengasse 6-8, Schloss Schönborn in Lower Austria, and Bergwerk Lauffen near Bad Ischl. Eigenberger categorized almost 1,700 works in 1939 as third-class, around 1,300 of which remained in the Academy building. Following an air raid 12 March 1945, 520 works were destroyed or have disappeared. In addition, 120 of the works stored elsewhere were lost through theft or for other war-related reasons. A start was made at the end of 1942 on the storage of works in the library and present-day Graphic Collection. The most valuable objects were stored in a safe in Rockhgasse, Vienna, and less valuable ones in a basement within the Academy, while the least valuable ones were left in place. With few exceptions, surviving source material does not provide any detailed information as to which objects in the present-day Graphic Collection were stored or lost during the war.

On 19 April 1945, when the fighting in Vienna came to an end, Herbert Boeckl took over as temporary rector, and courses started up again a few days later. In the denazification proceedings immediately after the war, thirty-seven members of staff were dismissed, although most were rehabilitated at the latest after the adoption of the 1947 National Socialist Act.

In 1988, the Paintings Gallery was merged with the Glyptothek at the Academy. Under the Art College Organization Act, the Academy was recognized as a university in 1998, retaining the name Academy of Fine Arts Vienna. In 2003, the Graphic Collection became a separate organization unit from the library. Systematic provenance research in the Academy collections began in 2013. Based on its research findings, the Art Restitution Advisory Board recommended on 11 January 2019 the return of three prints from its Graphic Collection originally owned by Moriz Grünebaum and restituted to his heirs in February 2021. In November 2021 the Art Restitution Advisory Board also recommended the restitution of 33 prints from the Sigmund Stiassny collection and 319 drawings from Maria and Rudolf Perlberger, which had been acquired by the Graphic Collection as a result of a donation in the 1980s.