Rudolf Noll studied classical archaeology and philosophy at the University of Vienna and submitted a dissertation in December 1929 on the beginnings of caricature in Greek vase paintings. He was a member of the Akademische Legion and the German nationalist faction of the Viennese student body. After his doctorate in 1930, he worked for some months as a librarian in the Archaeological-Epigraphic Seminar of the University of Vienna and then worked as an unpaid intern in the Collection of Greek and Roman Antiquities at the Kunsthistorisches Museum (KHM) in Vienna. In 1932 he managed the archaeological holdings of the Ferdinandeum in Innsbruck and in 1933 inventoried stone sculptures in the Collection of Greek and Roman Antiquities, before becoming a permanent employee at the KHM. In 1935 he joined the Vaterländische Front (Fatherland Front) as research department head "so as to keep out a genuine 'Vaterländer'", as he wrote in his application for NSDAP membership in 1938. Shortly after the annexation of Austria to the German Reich he was appointed "Betriebszellenleiter" (factory cell head) by the Vienna Gau head office and was responsible for regular roll calls and gymnastic exercises in the museum and the political surveillance of employees. From August 1938 he assumed the role as "Vertrauensmann" (representative) for the Reichsbund der Deutschen Beamten (Reich Association of German Civil Servants). In the run-up to the plebiscite about the annexation, he organized a propaganda event "Zum 10. April 1938" on 26 March 1938 in Hall VII of the Picture Gallery. In May 1938 he attended an air raid warden course in Berlin and became the KHM air raid warden. As Noll stated in June 1938 that he had already been a member of the Kampfopferring from 1932 until the NSDAP was made illegal but had burned his membership card during a house search, he was given the backdated NSDAP membership number 6,127.590 with entry on 1 May 1938. Together with the numismatist and Party comrade Fritz Manns he took part in 1939 in the excavations of the Roman remains at Carnuntum/Petronell in Lower Austria and in the same year undertook study trips to Berlin, Frankfurt am Main and Bad Homburg. Although as air raid warden he was deemed indispensable in autumn 1939, he had this classification revised the following year and joined an anti-aircraft defence unit in the Wehrmacht in December 1940. In 1941 he was promoted to curator with tenure at KHM.

From 1945 until early 1947 he was a prisoner of war in Yugoslavia. As a former NSDAP member he was suspended from the museum in July 1945 and left the KHM in 1947 after his return to Vienna "because of the absence of vacant positions". Although applying a few weeks later to be readmitted to the civil service on the grounds that he had never misused his NSDAP membership, benefitted personally from it or illegally enriched himself, his readmission was initially refused. For a time, Noll lived with his family from his Kriegsversehrtenrente (war invalid pension), and then worked from 1948 to 1952 as an usher in a cinema, before returning at the end of 1952 as a contract employee in the Collection of Greek and Roman Antiquities in the KHM following the retirement of director Fritz Eichler. In 1955 he was appointed an ordinary member of the Österreichisches Archäologisches Institut (Austrian Archaeological Institute) and curator 2nd class. After the suspension of Rudolf Sunkowsky, Eichler's successor as head of the Collection of Greek and Roman Antiquities at the KHM, for making and selling fakes, Noll effectively took over his function in 1956 and was promoted to curator 1st class in 1957, then head in 1958 and finally director of the Collection in 1963. In the same year the Philosophy Faculty of the University of Vienna offered him a lectureship in classical archaeology, specializing in Roman provincial archaeology, and the title of honorary professor. In 1961 Noll became an ordinary member of the Deutsches Archäologisches Institut (German Archaeological Institute), in 1966 a corresponding member and in 1970 regular member of the Austrian Academy of Sciences. He retired at the end of 1971, but continued to lecture at the University of Vienna until 1980. He was awarded the Kulturpreis des Landes Niederösterreich (Culture Prize of the Province of Lower Austria) in 1980 and the Wilhelm Hartel Prize of the Austrian Academy of Sciences in 1982.