Udaygiri, located 85kms from Bhopal, is renowned for the rock-cut cave sanctuaries known as Ugayagiri Caves. These caves, situated in Sunpura and Udaygiri village of Vidisha district are the finest example of classical Gupta arts. Ugayagiri Caves has around 20 distinctive caves and an inscription in one of the cave states that it were produced during the reign of Chandragupta II (AD 382-401). These caves have been numbered according to the sequence in which they were excavated. Cave 1 which has a frontage adapted out of a natural ledge of rock, thus forming both the roof of the cella and its portico is an impressive one.

In cave 5 one can found the illustration of the boar incarnation of Vishnu rescuing the earth goddesses Prithvi from the engulfing Ocean, holding aloft upon one tusk. There is also another carving of Lord Vishnu in his recline postures. Cave 7 displays various objects used by king Chandragupta-2 for his personal needs. Cave 9 consists of 2.5 m high pillars, pillared halls, large cella and long portico. Cave 20 is decorated with detailed Jain carvings. One can also found the ruins of the 6th-century Gupta temple at the hill top.