Herzog embarked on a long career associated with military affairs. He served in the Haganah during the Arab Revolt of 1936-39. While in London to complete his law studies (1942) he joined the British army and served as an intelligence officer in the Normandy campaign and in occupied Germany. He was discharged with the rank of lieutenant-colonel. In 1948, when the State of Israel was established, he became an officer in the IDF. Herzog rose to the rank of major-general before retiring in 1962.

Over the next two decades, Chaim Herzog combined a business career with public service, first as managing director of an industrial development group, later as a senior partner in a Tel Aviv law firm. He was chief military commentator for Israel Radio during both the Six-Day War and the Yom Kippur War and became renowned for his balanced broadcasts which boosted the morale of the population. Herzog was called back to active duty after the 1967 Six-Day War, to serve as the first military governor of Judea, Samaria and East Jerusalem.

Another milestone of Herzog's multifaceted career were his three years as Israel's ambassador to the UN in particularly trying times (1975-78). Israel's image was eroded by the hostile coalition of the Communist bloc and the Arab countries. The tension reached a height in November 1975 with the passing of the General Assembly Resolution equating Zionism with racism. This act prompted the usually dispassionate ambassador to tear up the draft resolution on the rostrum at the UN at the conclusion of a spirited defense of his country.

Chaim Herzog authored a host of books on military history and was a much sought-after commentator on political and military affairs. In 1981, he was elected a member of the Knesset for the Labor party. He resigned from the Knesset after his election as President, and was sworn in as the sixth President of the State of Israel on May 5, 1983. Chaim Herzog traveled widely. Endowed with impeccable English and a cosmopolitan manner, much of his efforts as President were devoted to enhancing Israel's standing abroad. In official and state visits to over thirty countries - including the first visits by an Israeli head of state to Germany and China, a visit of reconciliation to Spain marking 500 years since the expulsion of the Jews, and an ice-breaking tour of the Pacific - he addressed fifteen parliaments, made countless public appearances, spoke to the media and held private meetings with foreign leaders. Explaining Israel's position, he challenged unfair criticism from the media and foreign governments, encouraged closer diplomatic relations and promoted trade relations. Herzog also emphasized the centrality of Israel and advocated a strong Zionist connection between Israel and Jewish communities around the world, stressing the importance of Jewish education. During his term of office, indecisive election results led to six changes of government and four changes of Prime Minister, and President Herzog often had to play the role of behind-the-scenes arbitrator.

While refraining from criticizing government policies in public, President Herzog was not aloof to politics. In his own urbane but frank manner, he was far more outspoken than his predecessors. Emphasizing that he was the President of "all Israelis," Herzog took pains to visit the Arab and Druze minorities as well as Jewish settlers in the administered areas. He also took a public stand on hotly-debated issues. He called for limitations to be imposed on the freedom of political fringe groups whose ideologies constituted incitement to violence, and he employed the Presidential pardon in a number of controversial cases. President Herzog exercised his authority in the electoral system of the time, and played a key role in the process of naming a government. In 1984 and 1988, he guided the formation of a national unity government.

Chaim Herzog stepped down from the Presidency in May 1993. Returning to private life, he devoted himself primarily to speaking tours, journalistic commentary, board memberships, and to writing his autobiography, Living History: A Memoir, published in 1996. He was actively involved in developing the Center for Middle East Studies and Diplomacy at Ben-Gurion University of the Negev in Be'er Sheva which today carries his name. Chaim Herzog died on April 17th, 1997. Several educational institutions are dedicated to his memory. Yad Chaim Herzog was established by his family to perpetuate his memory and legacy.