The church of St. Peter and Paul is a temple raised in the 19th century for the purpose of the Russian tsarist army which was stationed in Suwałki then. In 1918 the temple was taken over by the Catholic Church.
The temple was raised along with a complex of barracks surrouning it circa 1900 as a garrison Orthodox chapel. Its unquestionable ornament was a huge tower with an onion-like cupola and similar but smaller towers above the presbytery. After regaining independence the chapel was adopted by the Roman Catholic church, and in 1923 by military church. In 1944 it was severely destroyed by withdrawing German troops.