Prior to 1996, the main building of the Shanghai Library was located at 325, Nan Jing Xi Lu (Nanjing Rd. West). The library, with a total floor space of 35,000 square meters, consisted of the main building, the Bibliotheca Zi-Ka-Wei in Xujiahui, stack rooms on Chang Le Lu (Changle Rd.) and Long Wu Lu (Longwu Rd.) and the Audio-Visual Branch on Huai Hai Zhong Lu (Huai Hai Rd. Middle).

In December 1996, the new Shanghai Library moved to its present premises at 1555, Huai Hai Zhong Lu. It covers an area of 3.1 hectares, with a total floor space of 83,000 square meters. It is composed of a podium of 5 stories and two towers 58.8 and 106.9 meters high respectively. The two high-rising towers culminate in pyramids, indicating Shanghai Library being a place for people to gain knowledge and to scale the height of modern science and technology. The two towers also form an eye-catching outline, which has become the symbol of Shanghai Library.

The designers of the building combined the characteristics of the local culture and modern architecture, placing a garden of traditional style inside the plain layout of the building complex. The podium has its outside wall in vertical blocks, which shows the features of some of the famous modern buildings in Shanghai. Above the central catalog hall are arch dome windows that permit natural light in the daytime, and numerous sparkling stars at night. The green area of over 11,000 square meters surrounding the new building highlights the library as a cultural oasis even further.

The interior design of the building meets the needs of library functions, accommodating the divisions of cataloging, reading and book preservation on different levels, and dividing the areas of audio-visual service, exhibitions and conferences according to the horizontal layout of the building block. The public service area is centered on the catalog hall, surrounded by various reading rooms, thus creating a convenient, flexible and efficient environment.

In the new building, there are 32 reading rooms with over 3,000 seats and 20 individual carrels. The two exhibition halls cover floor areas of 1,100 and 650 square meters respectively. There are also various lecture rooms and multi-functional halls for national and international cultural activities, and for general entertainment.

The new library is equipped with a LAN-based computer, which integrates the public and technical services of the library through 700 and more workstations over the whole building. The network server of the library enables users to access the digitized resources on the web, including the library catalog and the selected pages of rare books. There are other high-tech applications in the library, including a satellite-antenna-receiver system, a computerized book-carrier system, a central air-conditioning system, a theft-and-fire detection system, and a hi-fi audio-visual system.