Mogalrajapuram Caves
Mogalrajapuram caves are located at a distance of 5km east of city. It has five rock cut sanctuaries dating back to the 5th century AD and of these only one is in good condition. The Mogalarajapuram temple has an 'Ardhanarisvara' statue which is thought to be the earliest in South India and one of its kind. One of the caves has an overhanging cornice with artificial windows. The idols of Lord Nataraja and Vinayaka in one cave are still in good condition compared to the other worn out images of deities. It is also a place of archaeological importance.. 

Undavalli Caves
The Undavalli caves in Guntur district are situated at a distance of only 8km from Vijayawada. These ancient Hindu cave temples  are believed to be of the 4th and 5th century AD's. They are fine specimens of architectural and sculptural models carved out of solid rock. The temples are atop the high hills overlooking the Krishna river. A huge monolith of Lord Vishnu in lying position is worth seeing.

Akkana and Madanna Caves
These caves are situated on the way to Kanaka Durga temple and are dedicated to Akkana and Madanna who were ministers in the court of Abdul Hasan Tanashah in the 17th century. A short distance from these caves is another cave dating back to 2nd century BC which hosts the trinity of Brahma, Vishnu and Maheshwara.