After leaving school, Josef Siller worked in his parents' restaurant, Meierei Siller, at Kaiser-Ferdinands-Platz (since 1919 Schwedenplatz) 3 in Vienna's 1st district, which he took over in 1898. He later opened a hotel there, which was soon became known as "Kleines Sacher". Apart from his collection activities and patronage of young artists following his marriage in 1903 to Anna, née Gilly, he was involved from 1907 with the Genossenschaft der bildenden Künstler Wiens in the Künstlerhaus. He was one of the most important private collectors in Vienna, and his wife was also a notable collector of glassware and porcelain. The Siller collection, most of which was kept at the hotel, included applied art objects as well as around 1,500 nineteenth- and twentieth-century paintings, with works by Ferdinand Georg Waldmüller, Hans Makart, Emil Jakob Schindler, Rudolf v. Alt, Hans Canon, Moritz Schwind, Peter Fendi, Gustav Klimt, Egon Schiele, Anton Faistauer and the Leibl circle. Siller acquired works from the estates of other collectors, such as Albert Figdor and Gottfried Eissler. He gave works to the Österreichische Galerie between 1914 and 1924. He loaned several works for the thirtieth exhibition in the Neue Galerie in Vienna in 1927 entitled Das unbekannte XIX. Jahrhundert in Werken österreichischer Kunst. Some items lent by him were damaged in the Glaspalast fire in Munich on 6 July 1931. In 1934 Josef and Anna Siller took over Hotel Sacher, which was threatened with bankruptcy. With the aid of her lawyer Hans Gürtler, Anna Siller also acquired the Daringerhof at Sieveringerstraße 36 in the 19th district in 1938, which Margarethe Ticho and Lea Roth had been forced to sell so as to escape to Palestine. That same year, Josef Siller acquired properties in Hetzendorf in the 12th district from Clarisse (Clarence) Endlweber, who was classed as a Jew and murdered in Theresienstadt in 1942. The property was close to the Gallhof, acquired by the Siller family in 1904, where he opened Meierei Café Siller. The Gau authorities in Vienna described Siller in 1938 as rejecting the Nazi regime. In 1940 the Gestapo suspected Siller of illegal political activity but there is no record of any police measures being taken against him. On 23 August 1943 the Institut für Denkmalpflege (Institute for Monument Protection) ordered some of the collection, which was on the Reichsliste national wertvollen Kunstguts (Reich list of national treasures), to be secured. It rescinded the order on 9 September 1943, however, after Siller suggested that the works could be stored in the air raid shelter on Schwedenplatz. The works deposited there were lost when the building was destroyed in April 1945.

Apart from the ruined building at today's Schwedenplatz 3–4 in the 1st district, Siller, who died in 1948, left numerous properties and buildings in Grinzing, Altmannsdorf, Hetzendorf, Liesing, Breitenfurt and on Rosenhügel, as well as the Sacher hotels in central Vienna, but no artworks are mentioned in his estate. After her husband's death, Anna Siller agreed an out-of-court settlement with the former owners of Daringerhof. In 1950 she gave a work by Karl Sterrer to the Österreichische Galerie that Josef Siller had lent to the museum in 1928. In return, the Österreichische Galerie gave back the painting Lady with Hat by Anton Faistauer, which had survived the war in an unknown depot. Anna Siller was unable to trace the collection items stored privately at the villa in Hetzendorf, which were presumably stolen. Josef Siller is named as a former owner of some artworks now in Austrian collections and federal museums that might require systematic provenance research. To date, no Nazi-related expropriations have been identified.