JACOB ZIV (1931-2023)

BY NERI MERHAV, SHLOMO SHAMAI (SHITZ),
AND ANDREW VITERBI

JACOB ZIV, a pioneering electrical engineer in the fields of information theory and data compression, was best known as the codeveloper of the Lempel-Ziv data compression algorithms. These foundational algorithms continue to enhance everyday technologies, powering capabilities in devices such as cell phones and tablets.

The Lempel-Ziv algorithms revolutionized data compression by eliminating the need for prior knowledge of a data source’s statistical properties. Ziv also demonstrated that the length of the compressed data sequence approached the theoretical minimum, solidifying the algorithm’s importance both in theory and application.

Jacob was born Nov. 27, 1931, in Tiberias, Israel, along the shore of the Sea of Galilee. He was the younger of two sons of Ben Tzion and Hannah Ziv. His father, an elementary school principal in Tiberias and later Ra’anana, is honored today with a school named after him. Jacob and his wife, Shoshana, had four children, nine grandchildren, and one great-grandchild. He was deeply devoted to both family and scholarship, and his legacy endures not only through his scientific contributions but also through the generations he inspired.

Jacob earned a bachelor’s degree in 1954 and a master’s degree in 1957 from what is now the Department of Electrical and Computer Engineering at the Technion – Israel Institute of Technology. He began his career as a research and development engineer with the Scientific Department of Israel’s Ministry of Defense in 1954. In 1960, he was awarded a fellowship to study at the Massachusetts Institute of Technology, where he earned a Ph.D. in 1962 in communication and information theory. Jacob spent several summers at Bell Laboratories in New Jersey, where he worked with pioneers in information theory. He returned to his post in Israel and remained with the Research Establishment until 1970. That year, he joined the faculty at Technion and spent the next five decades contributing to research, teaching, and administration. He eventually became a Distinguished Professor in the Department of Electrical and Computer Engineering and served in a variety of leadership roles, including dean of the faculty, vice president for academic affairs, chair of the Israeli Planning and Grants Committee, and president of the Israel Academy of Sciences and Humanities. He also maintained research collaboration with Bell Laboratories during sabbaticals.

In the late 1970s, Jacob and his colleague Abraham Lempel introduced a groundbreaking approach to information theory that departed from the traditional probabilistic models.^1,2,3 Instead, they proposed an individual-sequence approach using finite-state encoders and decoders, which laid the groundwork for universal data compression and coded communication. This led to the creation of the Lempel-Ziv (LZ) algorithms: LZ77 in 1977 and LZ78 in 1978. These algorithms became rare examples of elegant theory matched with practical impact. Their influence has been profound – touching nearly every computer, smartphone, or device that stores or transmits digital information. They also represent some of the most widely used techniques for lossless data compression. Among these, DEFLATE stands out as a variant optimized for both decompression speed and compression ratio. In the 1980s, the work of T. Welch led to the development of the Lempel-Ziv-Welch (LZW) algorithm, which quickly became the preferred method for a broad range of compression applications.⁴ Its versatility is reflected in its widespread adoption, from GIF images and compression tools like PKZIP to hardware peripherals such as modems. LZW also supports the compression of file formats including PDF, TIFF, PNG, and ZIP, as well as video formats like MP3 – used daily on millions of cellphones around the world. More broadly, LZ-based compression underlies nearly all devices that store or transmit digital content, operating silently in the background to manage digital storage efficiently. The fact that so many people interact with LZ compression every day without realizing it is a testament to its effectiveness. The LZ algorithms exemplify a rare synthesis of elegant theory and practical utility.

Less widely known – but no less remarkable – are the additional pillars of the individual-sequence approach and the versatility of the LZ algorithms, particularly LZ78. Beyond its central role in universal data compression, the LZ78 algorithm has become a powerful tool for a wide range of information processing tasks. These include universal channel decoding,⁵ prediction,⁶ hypothesis testing,⁷ model order estimation,⁸ guessing,⁹ filtering,¹⁰ and more. Remarkably, the asymptotic optimality of LZ78 as a data compressor induces the asymptotic optimality in all these tasks as well. This broad utility suggests a profound underlying principle in the algorithm’s ability to incrementally parse and statistically characterize data.

Jacob’s contributions to information and communication theory extend far beyond universal lossless data compression. One of his most influential collaborations was with Aaron D. Wyner (NAE 1994) on what became known as the Wyner-Ziv source coding theory.¹¹ This work established the optimal method for lossy data compression when side information is available only at the decoder – an important extension of the well-known Slepian-Wolf coding theory, which applies to lossless compression. Wyner-Ziv coding has had a tremendous impact, particularly in video compression, where it continues to be widely used.¹²

Jacob made additional foundational contributions to information theory and communications. His work on “information combining”¹³ proved central to the theoretical analysis of modern coding schemes, especially low-density parity-check (LDPC) codes that approach the Shannon channel capacity bound.¹⁴ These ideas also relate to information bottleneck problems, which are used to analyze deep learning algorithms.¹⁵ Among his other important contributions is a paper coauthored with Lawrence Ozarow and Aaron D. Wyner that analyzed communications systems under practical constraints such as peak power.¹⁶ He is known for the Ziv-Zakai bound on parameter estimation,¹⁷ one of the tightest and most general accuracy bounds. The bound remains relevant today, including in recent studies on quantum parameter estimation.¹⁸

As his work gained global recognition, Jacob received many honors in engineering and science. These include the Marconi Prize (1995), the BBVA Foundation Frontiers of Knowledge Award (2008), and various Institute of Electrical and Electronics Engineers (IEEE) awards, including: the Richard W. Hamming Medal in 1995 for “contributions to information theory, and the theory and practice of data compression,” the Claude E. Shannon Award in 1997, and – most notably – the IEEE Medal of Honor in 2021. This last award, the IEEE’s highest distinction, cited his “fundamental contributions to information theory and data compression technology, and for distinguished research leadership.” He was the sole recipient that year and joined a select group of just over 100 honorees since 1917. He was also elected to international membership in several distinguished U.S. institutions: the National Academy of Engineering (1988), the American Academy of Arts and Sciences (1998), the American Philosophical Society (2003), and the National Academy of Sciences (2004).

The three authors of this tribute – Neri Merhav, Shlomo Shamai (Shitz) (NAE 2013), and Andrew Viterbi (NAE 1978) – collectively knew and collaborated with Jacob for more than 100 years. He was a great raconteur with a warm sense of humor, respected worldwide for his groundbreaking research, teaching, and wisdom in resolving controversial academic matters with a light touch. Though uninterested in titles or administrative power – often declining leadership roles at his university – he accepted the demanding task of overseeing the allocation of scientific research funds among Israel’s top scientific institutions.

Andrew Viterbi first met Jacob during a visit to Technion in 1967, though their scholarly correspondence began years earlier. Over the next half-century, they became close friends, meeting at conferences across the globe and socializing with their families. “Jacob Ziv was among the most ethical, kind, and generous persons I have ever known,” Viterbi said. “He demonstrated this in the way he shared the credit for research with colleagues. Similarly, he was generous in praise for the work of colleagues and students, both those with whom he was collaborating and those working on related research. In short, Jacob Ziv, in his work and his life, was the model of the ideal academic, producing outstanding research discoveries, teaching, and training excellent students to follow in his footsteps, and together with them raising the reputation of his institution as one of the leaders in the theory and applications of digital communication, data storage, and information processing.”

To fully honor Jacob’s contributions, Viterbi invited his former students and collaborators, Merhav and Shamai (Shitz), to contribute to this tribute. Both recall with great admiration the privilege of having Jacob as their Ph.D. advisor. They fondly remember learning from his courses, receiving his guidance during their research, and continuing to collaborate with him in their academic careers profiting from his influential and original ideas. Though they can no longer seek his excellent guidance and clever advice, they will carry his lessons forward.

___________________________
¹Ziv J. 1978. Coding theorems for individual sequences. IEEE Transactions on Information Theory 24(4):405-12.
²Ziv J, Lempel A. 1977. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23(3):337-43.
³Ziv J, Lempel A. 1978. Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory 24(5):530-6.
⁴Welch T. 1984. A technique for high-performance data compression. Computer 17(6):8-19.
⁵Ziv J. 1985. Universal decoding for finite-state channels. IEEE Transactions on Information Theory 31(4):453-60.
⁶Feder M, Merhav N, Gutman M. 1992. Universal prediction of individual sequences. IEEE Transactions on Information Theory 38(4):1258-70.
⁷Ziv J. 1988. On classification with empirically observed statistics and universal data compression. IEEE Transactions on Information Theory 34(2):278-86.
⁸Merhav N, Gutman M, Ziv J. 1989. On the estimation of the order of a Markov chain and universal data compression. IEEE Transactions on Information Theory 35(5):1014-19.
⁹Merhav N. 2020. Guessing individual sequences: generating randomized guesses using finite-state machines. IEEE Transactions on Information Theory 66(5):2912-20.
¹⁰Ordentlich E, Weissman T, Weinberger MJ, Somekh-Baruch A, Merhav N. 2004. Discrete universal filtering through incremental parsing. Pp. 352-61 in Proceedings of the Data Compression Conference 2004, Snowbird, Utah, March 23-25.
¹¹Wyner A, Ziv J. 1976. The rate-distortion function for source coding with side information at the decoder. IEEE Transactions on Information Theory 22(1):1-10.
¹²Pereira F, Brites C, Ascenso J, Tagliasacchi M. 2008. Wyner-Ziv video coding: A review of the early architectures and further developments. Pp. 625-28 in Proceedings of the 2008 IEEE International Conference on Multimedia and Expo, Hannover, Germany.
¹³Sutskover I, Shamai (Shitz) S, Ziv J. 2005. Extremes of information combining. IEEE Transactions on Information Theory 51(4):1313-25.
¹⁴Sutskover I, Shamai (Shitz) S, Ziv J. 2007. Constrained information combining: Theory and applications for LDPC coded systems. IEEE Transactions on Information Theory 53(5):1617-43.
¹⁵Zaidi A, Estella-Aguerri I, Shamai S. 2020. On the information bottleneck problems: Models, connections, applications and information theoretic views. Entropy 22(2):151.
¹⁶Ozarow LH, Wyner AD, Ziv J. 1988. Achievable rates for a constrained Gaussian channel. IEEE Transactions on Information Theory 34(3):365-70.
¹⁷Ziv J, Zakai M. 1969. Some lower bounds on signal parameter estimation. IEEE Transactions on Information Theory 15(3):386-91.
¹⁸Tsang M. 2012. Ziv-Zakai error bounds for quantum parameter estimation. Physical Review Letters 108:230401.