In 1646 Rethymno was besieged and taken by a large Ottoman expeditionary force; in its train came dervishes, that is to say proponents of Sufi mysticism, whose prayers bolstered the courage of the soldiers. During the siege, a large part of the army, and the dervishes that accompanied it, made their camp at Mastabas, whence they actually entered the city of Rethymno. After the conquest of the city, sheikh Mustafa, who belonged to the Sufi order of the Kantiri, built on the site a tekke, or compound where dervishes could stay and practice spirituality. This, together with the mosque, became known as the Veli Pasha Mosque which still survives in the Mastabas neighborhood. Veli agha, who later became a pasha, was an adjutant of the conqueror of Rethymno, vizier Hussein pasha, and undertook to finance the construction of the tekke, which later took his name. The oldest known mention of the mosque is found in the records of the kadi (judge) of Rethymno of 1651. Through the years, many admired and mentioned this historic monument which was a focus of pilgrimage for the Muslims of Rethymno. One of them was the well-known Ottoman traveler Evliya Celebi (1611-1682), who journeyed extensively across the Ottoman empire. He writes that the tekke of Veli Pasha was one of the three foundations at Rethymno which, night and day, provided soup to rich and poor, old and young. That is to say, the dervishes of the tekke maintained a soup kitchen.

The compound of Veli Pasha consists of the mosque to the south, eight vaulted cells to the west and a two-storey building to the north, established on gently sloping ground. The mosque is almost square in plan, 16.6 by 16.2 meters, and belongs to the multidome type with a portico. It is roofed by six small domes supported by a network of pointed arches and squinches, arranged in two rows of three domes along a NW-SE axis. The portico is attached to its northwest face and consists of an open arcade with three pointed arches covered by three domes (tholos). The form of the arches may be associated with local masons, preserving the architectural idiom of Venice.

Access to the interior of the square building is effected through an imposing entrance of local limestone carved with relief vegetal motifs which is believed to derive from a neighboring church at the Borgo which was abandoned during the Ottoman siege of Rethymno. On the west side opens the door leading to the staircase ascending the minaret, with a landing reached from a stone staircase. Each face of the building except for the west one is furnished with two rectangular windows. In the middle of the SW wall stands the mihrab, the Muslim prayer niche, oriented towards Mecca.

The minaret stands at the NW corner of the building and has only one balcony. According to its inscription, it was built in AH 1204 (AD 1789/90). The inscription is as follows: "Let it be proof to curses! Glory and gratitude to the Lord... We made a donation and restored the minaret so that the ezan (call to prayer) may be heard, and for the charity of God." At the west part of the tekke courtyard, on higher ground, are laid out the eight vaulted identical cells that served as dwellings of the dervishes. The two-storey building forming an extension on the north side is architecturally plain and probably served as residence of the sheikh of the tekke and kitchen. The fountain was situated at the entrance of the compound, as it was essential for the ritual ablutions required before prayer.

The mosque currently houses the Paleontological Museum. Its exhibit comprises finds from the municipal paleontology collection of Rethymno. It introduces the visitor to the geological evolution of Crete with the help of photographs, texts, maps and full-size reconstructions. Finally, the courtyard is used for various cultural functions.