CMI began as the School of Mathematics, SPIC Science Foundation, in 1989. The SPIC Science Foundation was set up in 1986 by Southern Petrochemical Industries Corporation Ltd (SPIC), one of the major industrial houses in India, to foster the growth of Science and Technology in the country.

In 1996, the School of Mathematics became an independent institution and changed its name to SPIC Mathematical Institute. In 1998, in order to better reflect the emerging role of the institute, it was renamed the Chennai Mathematical Institute (CMI).

From its inception, the Institute has had a vibrant PhD programme in Mathematics and Computer Science. In the initial years, the PhD Programme was affiliated to the Birla Institute of Technology & Science, Pilani and the University of Madras. In December 2006, CMI was recognized as a university under Section 3 of the UGC Act 1956 and will henceforth award PhD degrees directly.

To bridge the gap between teaching and research, in 1998 CMI began an integrated 3-year course in Mathematics and Computer Science leading to a BSc Honours degree. In 2001, the BSc programme was extended to incorporate two courses with research components, leading to an MSc degree in Mathematics, and an MSc degree in Computer Science. In 2003, a new undergraduate course was added, leading to a BSc degree in Physics.

These teaching programmes are run in cooperation with the Institute of Mathematical Sciences (IMSc), Chennai. In the initial years, the degrees were awarded by the Madhya Pradesh Bhoj (Open) University (MPBOU), Bhopal. Since CMI has been recognized as a university, it will now award BSc and MSc degrees directly.