Heinrich Schwarz was the second child of the industrialist Alois Louis Schwarz and his wife Johanna, née Posamentier. Shortly after his birth, his parentsmoved from Prague to Vienna. Schwarz probably converted to Catholicism in the course of his military service in the First World War, but at the latest in 1918. After studying art history, archaeology and philosophy in Vienna and obtaining a doctoral degree with Max Dvořak on the origins of lithography in Vienna, he worked from October 1922 to March 1923 as an unpaid volunteer in the Albertina. He then became a research assistant at the Österreichische Galerie in the Belvedere under Franz Martin Haberditzl. From December 1927 he was a contract employee in the research department, then a research assistant, and ultimately in 1935 a curator 2nd class. By his own account, he was ultimately promoted to curator 1st class in 1938.

After the annexation of Austria to the Nazi German Reich, Schwarz was dismissed on 4 April 1938 as part of the repression of Jewish civil servants as a "non-Aryan". The same year, the authorities ordered him to pay the Jewish asset levy. In the corresponding asset declaration he listed an art collection with paintings, drawings, engravings and photographs, most of which he had been given during the lifetime of his father, who died in 1930. He did not present a detailed list of the works in his apartment at Schubertring 8 in the 1st district, but he had his own collector's stamp, which can be seen in Frits Lugt's first edition of Les marques de collections (1921) under the number 1372. The entry names Schwarz as a "contemporary collector" and indicates that the collection had only just been started. In 1939 Schwarz felt compelled to leave Austria on account of the mounting persecution. On 17 March 1939 he emigrated first to Copenhagen and later to Stockholm and finally via Norway to the USA, where he arrived in New York on 3 February 1940. To pay for the journey, he sold parts of his art history library and works from his graphics collection both before and during the flight. Among the objects in Schwarz's collection were a series of drawings by the artist Heinrich Reinhold, about whom he had published several articles. Some of these drawings were auctioned by Kunstantiquariat C. G. Boerner in Leipzig in 1939 and 1941 at auction nos. 201 and 204.

After arriving in the USA, Schwarz worked from 1941 to 1942 in the Albright Gallery in Buffalo, New York, and from 1943 to 1953 as a curator for paintings, drawings and graphic art at the Museum of Art of the Rhode Island School of Design in Providence, Rhode Island. Then he taught at  Wesleyan University in Middletown, Connecticut, and Columbia University in New York and was curator of the Davison Art Center Collection (Wesleyan University). His application for US citizenship of 18 April 1940 was approved on 24 September 1945. In 1948 Heinrich Schwarz married Elisabeth Arany, née Goldberger. As compensation for loss of earnings during the Nazi dictatorship he was granted a pension from 1954 under the Civil Servants Compensation Act. In 1956 he applied to the Republic of Austria for compensation for the loss of securities, the furnishings in his last home in Vienna and the library. Research to date has not been able to identify the restitution of these assets. In 2022, four works from the Klassik Stiftung Weimar and the Staatliche Kunstsammlungen Dresden were restituted to the heirs of Heinrich Schwarz. In November 2023, the Art Restitution Advisory Board recommended the return of two drawings from the former Schwarz Collection, which the Albertina had purchased at auction from C. G. Boerner in 1939. For a photograph acquired in 2002, however, no evidence of a Nazi-persecution-related sale could be found.