Richard Neumann came from a family of Austrian textile entrepreneurs. After studying philosophy at the University of Heidelberg, he worked in leading positions in the weaving and printing company founded by his grandfather Max Bernhard Neumann in Vienna, then for David Goldmann's wool products sales corporation and his Guntramsdorf cloth printing factory. In 1905 he married his cousin Alice Neumann in the Viennese city temple, and their daughter Marie Dorothee, called Dora, was born in 1906. In 1921, the Federal Monuments Office inspected the paintings, etchings, faience works, majolica works, bronzes, sculptures and old furniture on the ground and first floors of the Neumann villa at Hasenauerstraße 30, in Vienna's 19th district, and determined that the collection was worthy of preservation, which is why it also assessed the rooms as "not suitable for normal residential purposes".

After the "Anschluss" of Austria to the German Reich, the director of the Albertina, Otto Benesch, and the Dorotheum expert Ottokar Weigel valued the 45-item painting collection at 60,750 Reichsmark in the context of the declaration of assets. The remaining art and furnishings in the flat were valued by Dorotheum expert Fritz Polt at approximately 4,000 Reichsmark. While Richard and Alice Neumann fled from Nazi persecution to Switzerland, and later via Paris and Spain to Cuba, their daughter Dora, divorced Selldorf, who had left the Jewish Community in 1934, remained in Vienna for the time being and lived in the Neumann Villa, which was to be "Aryanised" in 1942. In October 1938, the Döbling district authorities secured six objects from the art collection and handed them over to the Kunsthistorisches Museum (KHM) for safekeeping: two wings of an altar with portraits of the altar donors by Marten van Heemskerck, the paintings Washerwomen on the River by Alessandro Magnasco, and Sacrificial Scene (Hannibal's Oath) by Giovanni Battista Pittoni, as well as two statuettes by Alessandro Algardi, depicting Pope Innocent X and St. Philip Neri. The KHM subsequently acquired these works of art from Dora Selldorf, who had been authorised by her father to sell them in order to support herself. Two Kremser Schmidt paintings thus came into the possession of the city of Krems. Other works of art, whose export had been approved by the Central Monument Protection Office, were sold by the Neumann couple in Paris.

After 1945, Richard Neumann, first from Cuba and later from New York, demanded the return of the seized assets, in particular the two Heemskerck panels. After these demands were rejected at first instance, the Higher Restitution Commission at the Oberlandesgericht (Higher Provincial Court) approved his claim according to the Drittes Rückstellungsgesetz (Third Restitution Act) on 29 January 1952. The Supreme Court confirmed the judgement on 15 March 1952, according to which the two altar wings were to be restituted against repayment of the purchase price converted into shillings. However, the Federal Monuments Office did not release the paintings for export. Instead, the KHM entered into exchange and acquisition negotiations with Richard Neumann. In 1953 Neumann received a painting from the KHM in New York, the disposal of which had been approved by the exchange commission, as well as the proceeds of 3,000 US dollars from the disposal of two other paintings. In return, the Heemskerck altar wings became Austrian federal property, as did, as a dedication by Neumann, the two Algardi statuettes and the paintings by Magnasco and Pittoni. The Kremser Schmidt paintings remained in Krems. In 1966, the now widowed Alice Neumann unsuccessfully demanded the restitution of the Heemskerck altar wings from the KHM. Even the claim made in 2004 by Neumann's grandson for all works of art from the Neumann Collection did not initially lead to their restitution. In 2007, the Weinstadtmuseum Krems gave back the two Schmidt paintings from Krems. After the revision of the Art Restitution Act in 2009, the Advisory Board finally recommended the restitution from the KHM in 2010, on the condition that both the valorised purchase amount and the painting transferred from the KHM (or its equivalent) be restituted for the Heemskerck panels. The recommendation was implemented in 2011. In 2013, the Republic of France restituted six works from the Neumann Collection, and a further two paintings were reacquired from the art trade. Since 2021, 14 of the 16 recovered art objects from the Neumann Collection have been on permanent loan to the Worcester Art Museum in Massachusetts.