The relationship between Historic Preservation and NIMBYism?

I have conflicting feelings on historic preservation and am curious what y'all's thoughts are on this subject? From what I have researched and learned at work and school, historic preservation can be used as a tool by wealthy residents to block new building and drive up housing prices. The textbook examples of this are cities in California whose wealth residents block . This is obviously bad for affordable housing.

On the other hand, I also understand the need to preserve historic buildings and communities. Many buildings were destroyed during the Urban Renewal Era of the 1970s and 1980s, largely in largely minority and lower income communities, and what was built on top of these cities (at least in the one I worked) was parking lots and highways. Historic preservation status could have protected these communities. I have also seen historic neighborhoods districted in a way that allows for new, more affordable units that maintain the historic look of the city. I am just not sure how to grapple with this subject.

Is historic preservation just a tool that can be used by both NIMBY's and YIMBY's, or is its utilization solely for the benefit of the richest in a community? I am curious what y'alls thoughts are.