California weighs AI bills

A recent audit of algorithmic management tools sold to companies suggests that many use AI to determine pay, a trend that started among gig workers nearly a decade ago that researchers say is now becoming widespread. Tests have found that AI that sifts through resumes can disqualify or downgrade a job applicant for arbitrary reasons like race or sex or because they’re wearing glasses. Inferences based on brain data, which can reveal a person’s health or the words or images they form in their mind, would also be prohibited under the legislation, which will be considered in a hearing scheduled for Friday.

SB 7 is one of a trio of bills supported by the California Federation of Labor Unions, AFL-CIO, this year that attempt to regulate automated systems in the workplace. The federation represents more than 2 million workers statewide and made more than $2.1 million in political donations to Assembly and Senate members last year, according to the CalMatters Digital Democracy database.

The other two bills, Assembly Bill 1331 and AB 1221 both involve regulating workplace surveillance.

Federation president Lorena Gonzalez told CalMatters that employers shouldn’t be allowed to predict if you’re pregnant or what you think about your employer or boss and use that information against you.

“We have to set up guardrails against every kind of surveillance and AI tool to ensure that workers have the privacy and respect and autonomy that they deserve,” Gonzalez said, who continues to work with labor unions in other states to prevent AI from harming front-line workers.

A race to use AI?

Sen. Jerry McNerney, the bill’s author, said there seems to be a race to use AI to displace workers or squeeze more efficiency out of them.

The Stockton Democrat told the audience at a CalMatters event last month in San Francisco that “there’s tremendous opportunity for productivity, for making people more comfortable and safe, but your imagination can run wild with what can go wrong here.”

In response to questions about criticism that SB 7 will drive costs up, McNerney told CalMatters in a statement that he plans to introduce amendments that will substantially reduce the cost of implementing the bill, including the elimination of a process for workers to appeal decisions made by AI, a major point of contention for business groups like the California Chamber of Commerce.

SB 7 is one of a number of bills on what is known as the suspense calendar. Each year, hundreds of bills with a price tag above $50,000 are added to the calendar. Appropriation committees in the Assembly and Senate then have the power to designate some bills as too expensive or otherwise use cost to justify holding or effectively killing the legislation. Roughly two out of three bills make it from the suspense calendar to a final vote on the floor of the California Assembly or Senate, but the fact that this process happens with little debate and out of public view has led some to call it undemocratic and corrupt.

Making sure private companies comply with SB 7 could cost the state $600,000 or more, according to an assembly appropriations committee staff analysis released last week. But the cost for the state’s own agencies to comply is unknown, because the state doesn’t know how many automated employment systems it uses. Earlier this year the California Department of Technology allowed state agencies to self report use of high-risk automated systems like those used in employment. Nearly 200 state agencies reported no use of such technology despite the fact that state agencies are currently using or have used high risk automated systems in the past.

Cost barrier

The cost of compliance or enforcement has stalled AI employment regulation before. Last month, the California Privacy Protection Agency bowed to pressure by lawmakers, business groups and Gov. Newsom and voted to weaken its own automated decision-making technology rules.

In the other chamber of the California Legislature, the California Senate Appropriations Committee is preparing to decide the fate of another bill, Assembly Bill 1018, which would require testing before use of automated systems that make consequential decisions about employment, education, housing, health care, financial services and parts of the criminal justice system. That bill would give people the right to appeal an automated decision within 30 days. An analysis by committee staff found that if the bill passes it could cost local agencies, state agencies, and the state judiciary branch hundreds of millions of dollars.

In a statement shared with CalMatters, Assemblymember Rebecca Bauer-Kahan said she has no plan to make amendments to the draft bill but that she’s committed to bringing down costs wherever it makes sense to do so.

“I will admit to being surprised by the Senate Appropriations estimates, considering CalMatters’ prior reporting that automated decision-making systems use was not widespread at the state level. So I’m working to better understand the cost estimates so I can respond to them appropriately,” she said.

Last year a similar version of the bill was pulled by Bauer-Kahan, a Democrat from San Ramon, after amendments limited its provisions to assessing employment. That bill attracted opposition from Big Tech companies but also a range of nearly 100 companies including Blue Shield of California, dating app company Bumble, biotech company Genentech, and pharmaceutical company Pfizer.

Nearly 80 businesses or groups oppose the bill, including the California Hospital Association and major health care providers like Kaiser Permanente. The advocacy group California Life Sciences argues that passage could lead providers to pass on higher costs to patients, while the California Credit Union League argues that compliance with the bill will slow innovation, an outcome that led Gov. Newsom to veto a major piece of AI legislation last year.

Both bills were authored by California lawmakers with extensive histories of regulating AI. McNerney previously served as co-chair of an AI subcommittee in Congress. As chair of the assembly privacy and consumer protection committee, Bauer-Kahan is one of the most powerful regulators of AI in California, and was one of the first lawmakers nationwide who attempted to enshrine the Biden administration’s AI Bill of Rights into state law.

Partly because of the price tag for businesses that deploy AI, the California Privacy Protection Agency acted against the wishes of unions, digital rights and privacy groups and voted last month to weaken its own draft rules to regulate how businesses use automated decision-making technology. Conversely, the California Civil Rights Department finalized rules to protect workers from automated discrimination during recruitment, hiring and promotion processes. Those rules go into effect in October. The principle that people deserve the right to know when AI makes an important decision about their lives and to appeal if they think AI got it wrong were popularized by Biden’s AI Bill of Rights.

 Survey reveals concerns

A survey of 1,400 California adults released this week by TechEquity, a supporter of AB 1018, found that more people are concerned than excited about AI; that nearly 6 in 10 believe the benefits of AI will accrue to the wealthy instead of the middle class; and that 70% want the government to adopt laws to protect people from harm.

Peter Leroe-Muñoz, general counsel for the Silicon Valley Leadership Group, a business organization with members like Apple, Google and Microsoft, said the group supports responsible AI innovation. Still, he added, the costs of audits, training, staffing, disclosures, data retention and potential lawsuits may ultimately undermine a lot of the services that businesses provide today at a reasonable cost.

“That becomes an additional financial burden in this more uncertain time that really becomes a drag on innovation, a higher cost not only for businesses and consumers but counties and other local municipal governments,” he said.

Passing bills that force employees to notify workers if they use AI to determine things like compensation is a critical first step, said Veena Dubal, a co-author of the study released last week and an outspoken critic of AI that harms workers to enrich tech companies. Because people don’t know this tech is in use today, she hopes that notifying people increases public knowledge, and that leads to a ban of algorithms that determine how much people get paid.

To those who say the cost of implementing these bills is too high, she warns that algorithms are already influencing how much money people make, and that cost gets passed down to taxpayers. Algorithms can also locate union organizers, automate discrimination, or terminate people who managers decide they don’t like.

“These all have extraneous costs on society, extraordinary costs,” she said. “Workers, voters, taxpayers, we should all have a say in the kind of world that these companies are creating with the disproportionate power that they hold.”

If these bills don’t pass, Dubal said, it’s a signal that AI regulation has stalled in California.

“As much as Gov. Newsom wants to juxtapose himself against Donald Trump and California wants to be a sort of savior to the rest of the nation in this moment of extreme right authoritarian actions,” Dubal said, “it’s really important that the state not continue to concede to big tech companies who are very much in bed with the president.”