SafeRent is a machine learning black box for landlords. It gives landlords a numerical rating of potential tenants and a yes/no result on whether to rent to them.

In May 2022, Massachusetts housing voucher recipients and the Community Action Agency of Somerville sued the company, claiming SafeRent gave Black and Hispanic rental applicants with housing vouchers disproportionately lower scores.

The tenants had no visibility into how the algorithm scored them. Appeals were rejected on the basis that this was what the computer output said.

  • General_Effort@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    28 days ago

    The article is fake news. I suggest looking elsewhere for proper information.

    As for your questions: LLMs were certainly not involved here. I can’t guess what techniques were used.

    Racial discrimination is often hard to nail down. Race is implicit in any number of facts. Place of birth, current address, school, … You could infer race from such data. If you do not look at race at all but the end result still discriminates, then it’s probably still racial discrimination. I say probably because you are free to do what you like and discriminate based on any number of factors, as long as it isn’t race, sex, and the like. You certainly may discriminate based on education or wealth. Things being as they are, that will discriminate against minorities. They have systematically lower credit ratings, for example.

    In the case of generative AI, bias is often not clearly defined. For example, you type “US President” into an image generator. All US presidents so far were male, and all but one white. But half of all people who are eligible for the presidency are female and (I think) a little less than half non-white. So what’s the non-biased output?