This imposing building occupies a prominent position along the edge of Cathedral Square. It was constructed between 1648 and 1678. Its monumental façade, with its theatrical presence, forms the dramatic backdrop to the Cathedral’s parvis and is visually connected to the church by a small episcopal gallery built in 1627 by Bishop Fabrizio Carafa for solemn blessings.

The remaining sides of the quadrangular palace border the former Hospital of Saint Nicholas and the Palace of the Royal Court.

One of the building's most distinctive features is the rusticated ashlar façade, which lends a sense of grandeur and formality to the structure. The leading portal, designed in a late Renaissance style, is framed by rusticated pilasters and crowned by a decorative frieze. Its off-center placement reflects the constraints of the earlier buildings incorporated into the project. The façade steps back significantly at the corner. Along the wall, the sloping profile of a demolished roof remains visible—these are the traces of the former Church of Saint Mary of Mercy, built before 1586 and demolished in 1905 for urban planning reasons. The church’s portal, leaning against the palace’s tower, features a bas-relief of the Pietà and an inscription calling upon the Virgin Mary’s intercession against drought.

The building’s exterior is divided into two levels by a narrow stringcourse, interrupted by windows that were enlarged during the 18th century. Two rows of windows are adorned with petite, elegantly shaped balconies: the lower-level windows are topped with curved pediments, while the upper-level ones feature pagoda-style gables. The roofline is finished with a decorative cornice crowned by ornamental pinnacles.

A vaulted entrance hall with cross ceilings leads into an inner courtyard, originally featuring an arcaded portico with vestibules and a dramatic staircase ascending to the upper floors. However, various modifications and additional stories have altered its original layout over time. A wide staircase with two flights, covered by a barrel vault, leads to the first floor, while a narrower stairway—likely added in the 18th century under architect Vito Valentino—leads to the second floor.

The de Lerma family, part of the high Spanish nobility, settled in Bitonto at the beginning of the 16th century alongside the Sylos family of Burgos and the Albuquerque of Portugal. As Dukes of Castelmezzano, the de Lermas played an essential role in the city’s civic and religious life, particularly within the Cathedral Chapter. The palace was built in two distinct phases, in tandem with their rural estate, Torre de Lerma. Following the de Lerma family's extinction at the end of the 18th century, the building changed hands several times before becoming a residential property.

Above the large archway leading to the staircase is the family’s coat of arms, upheld by a sculpted angel—one of the last visible signs of the family’s former prestige.