Technology and Society

Power and prediction: U of T's Avi Goldfarb on the disruptive economics of artificial intelligence

siddiq22 — Fri, 20 Jan 2023 16:38:57 +0000

Power and prediction: U of T's Avi Goldfarb on the disruptive economics of artificial intelligence

siddiq22 Fri, 01/20/2023 - 11:38

Cutline

Avi Goldfarb, a professor at the Rotman School of Management and research lead at the Schwartz Reisman Institute for Technology and Society, says the AI revolution is well underway – but that system-level change takes time (supplied images)

Daniel Browne

Topic

Global Lens

Technology and Society

Schwartz Reisman Institute for Technology and Society

Artificial Intelligence

Business

Health

Rotman School of Management

Startups

Technology

In the new book Power and Prediction: The Disruptive Economics of Artificial Intelligence, co-author Avi Goldfarb argues we live in the “Between Times”: after discovering the potential of AI, but before its widespread adoption.

Delays in implementation are an essential part of any technology with the power to truly reshape society, says Goldfarb, a professor of marketing and the Rotman Chair in Artificial Intelligence and Healthcare at the �ؿ�ζSM's Rotman School of Management and research lead at the Schwartz Reisman Institute for Technology and Society.

He makes the case for how AI innovation will evolve in Power and Prediction, his latest book co-authored with fellow Rotman professors Ajay Agrawal and Joshua Gans. The trio, who also wrote 2018’s Prediction Machines: The Simple Economics of Artificial Intelligence, are the co-founders of the Creative Destruction Lab, a non-profit organization that helps science- and technology-based startups scale.

Goldfarb will give a talk at the Rotman School of Management on Jan. 25 as part of the SRI Seminar Series. He spoke with the Schwartz Reisman Institute to discuss how the evolution of AI innovation will require systems-level changes to the ways that organizations make decisions.

(The interview has been condensed for length and clarity.)

What changed in your understanding of the landscape of AI innovation since your last book?

We wrote Prediction Machines thinking that a revolution was about to happen, and we saw that revolution happening at a handful of companies like Google, Amazon and others. But when it came to most businesses we interacted with, by 2021 we started to feel a sense of disappointment. Yes, there was all this potential, but it hadn’t affected their bottom line yet – the uses that they’d found had been incremental, rather than transformational. And that got us trying to understand what went wrong.

One potential thing that could have gone wrong, of course, was that AI wasn’t as exciting as we thought. Another was that the technology was potentially as big a deal as the major revolutions of the past 200 years – innovations like steam, electricity, computing – and the issue was system-level implementation. For every major technological innovation, it took a long time to figure out how to make that change affect society at scale.

The core idea of Power and Prediction is that AI is an exciting technology – but it’s going to take time to see its effects, because a lot of complementary innovation has to happen as well. Now, some might respond that’s not very helpful, because we don’t want to wait. And part of our agenda in the book is to accelerate the timeline of this innovation from 40 years to 10, or even less. To get there, we then need to think through what this innovation is going to look like. We can’t just say it’s going to take time – that’s not constructive.

What sort of changes are needed for organizations to harness AI’s full potential?

Here, we lean on three key ideas. The first idea is that AI today is not artificial general intelligence (AGI) – it’s prediction technology. The second is that a prediction is useful because it helps you make decisions. A prediction without a decision is useless. So, what AI really does is allow you to unbundle the prediction from the rest of the decision, and that can lead to all sorts of transformation. Finally, the third key idea is that decisions don’t happen in isolation.

What prediction machines do is allow you to change who makes decisions and when those decisions are made. There are all sorts of examples of what seems like an automated decision, but what it actually does is take some human’s decision – typically at headquarters – and scales it. For organizations to succeed, they require a whole bunch of people working in concert. It’s not about one decision – it’s about decisions working together.

One example is health care – at the emergency department, there is somebody on triage, who gives a prediction about the severity of what’s going on. They might send a patient immediately for tests or ask them to wait. Right now, AIs are used in triage at SickKids in Toronto and other hospitals, and they are making it more effective. But to really take advantage of the prediction, they need to coordinate with the next step. If triage is sending people for a particular test more frequently, then there need to be other decisions made about staffing for those tests, and where to offer them. And, if your predictions are good enough, there’s an even different decision to be made – maybe you don’t even need the tests. If your prediction that somebody’s having a heart attack is good enough, you don’t need to send them for that extra test and waste that time or money. Instead, you’ll send them direct to treatment, and that requires coordination between what’s happening upstream on the triage side and what’s happening downstream in terms of the testing or treatment side.

AI is as exciting a technology as electricity and computing, but it will take time to see its effects, Avi Goldfarb says.

Will certain sectors have greater ease in adopting system-level changes than others?

There is a real opportunity here for startups because when building a new system from scratch, it’s often easier to start with nothing. You don’t have to convince people to come along with your changes, so it becomes a less political process – at least within your organization. If you’re trying to change a huge established company or organization, it’s going to be harder.

I’m very excited about the potential for AI and health care, but health care is complicated; there are so many different decision-makers. There are the patients, the payers – sometimes government, sometimes insurance companies, sometimes a combination of the above – and then there are doctors, who have certain interests, medical administrators who might have different interests, and nurses.

AI has potential to supercharge nurses, because a key distinction between a doctor and a nurse in terms of training is diagnosis, which is a prediction problem. If AI is helping with diagnosis, that has potential to make nurses more central to how we structure the system. But that’s going to require all sorts of changes, and we have to get used to that as patients. And so, while I think the 30-year vision for what health care could look like is extraordinary, the five-year timeline is really, really hard.

What are some of the other important barriers to AI adoption?

A lot of the challenges to AI adoption come from ambiguity about what’s allowed or not in terms of regulation. In health care contexts, we are seeing lots of people trying to identify incremental point solutions that don’t require regulatory approval. We may have an AI that can replace a human in some medical process, but to do it is going to be a 10-year, multibillion-dollar process to get approval – so they’ll implement it in an app that people can use at home with a warning that it’s not real medical advice.

The regulatory resistance to change, and the ambiguity of what’s allowed, is a real barrier. As we start thinking about system changes, there is an important role for government through legislation and regulation, as well as through its coordinating function as the country’s biggest buyer of stuff, to help push us toward new AI-based systems.

There are also real concerns about data and bias, especially in the short term. However, in the long run, I’m very optimistic about AI to help with discrimination and bias. While a lot of the resistance to AI implementation right now is coming from people who are worried about [people who will be negatively impacted by] bias [in the data], I think that pretty soon this will flip around.

There’s a story we discuss in the book, where Major League Baseball brought in a machine that could say whether a pitch was a strike or a ball, and the people who resisted it turned out to be the superstars. Why? Well, the best hitters tended to get favored by umpires and face smaller strike zones, and the best pitchers also tended to get favoured and had bigger strike zones. The superstars benefited from this human bias, and when they brought in a fairer system, the superstars got hurt. So, we should expect that people who currently benefit from bias are going to resist machine systems that can overcome it.

What do you look for to indicate where disruptions from AI innovation will occur?

We’re seeing this change already in a handful of industries tech is paying attention to, such as advertising. Advertising had a very Mad Men vibe until recently: there was a lot of seeming magic in terms of whether an ad worked, how to hire an agency and how the industry operated – a lot of charm and fancy dinners. That hasn’t completely gone away, but advertising is largely an algorithm-based industry now. The most powerful players are big tech companies – they’re no longer the historical publishers who worked on Madison Avenue. We’ve seen the disruption – it’s happened.

Think through the mission of any industry or a company. Once you understand the mission, think through all the ways that mission is compromised because of bad prediction. Once you see where the mission doesn’t align with the ways in which an organization is actually operating, those are going to be the cases where either the organization is going to need to disrupt themselves, or someone’s going to come along and do what they do better.

Read the full Q&A at the Schwartz Reisman Institute for Technology and Society

Popularity of 'gamified' apps raises new legal issues, student researchers warn

geoff.vendeville — Tue, 05 Apr 2022 17:26:33 +0000

Popularity of 'gamified' apps raises new legal issues, student researchers warn

geoff.vendeville Tue, 04/05/2022 - 13:26

Cutline

(photo by AsiaVision/Getty Images)

Nina Haikara

Topic

City & Culture

Technology and Society

Faculty of Law

Hal Jackman

With a few pushes of a button or a swipe of your fingers, you can trade stocks, hail a ride, order a pizza or find a date. But what are the implications of these gamified apps on human behaviour and the law?

Law students in the �ؿ�ζSM's Future of Law Lab explore these questions in a new research report that focuses on how current laws should respond to gamification – the introduction of elements of play and gaming across activities and aspects of life.

“My research interest is in securities law, so gamification really captured my attention with the whole meme stock craze, and speculation about how online trading apps might have fed into it,” says Doug Sarro, a doctoral candidate at the Faculty of Law.

“By gamifying investing, did online trading apps lead their users to trade too frequently in assets that were too risky for them? I looked around and saw gamification raises issues in other areas of law, too. For instance, when ridesharing apps use gamification to influence when and where drivers work, does this mean these drivers ought to be considered employees rather than independent contractors?

“I thought the Future of Law Lab would be a fantastic place to explore gamification and gain a broader view of the challenges it poses to law.”

The lab established a co-curricular working group led by Sarro and 16 JD students to research the implications of gamification in four areas: online trading; ridesharing and food delivery; employee productivity; and dating.

First-year law student Nikée Allen, who has a bachelor's degree in psychology with a minor in sociology from Ryerson University, looked at how dating apps can internalize and propagate racial biases among users.

“There are many – especially young – users, including vulnerable members of marginalized communities, who are being ranked by the people who are swiping on them,” Allen says. “If racialized users are being swiped on less, they are ranked lower, and they're being seen by fewer people.”

The Future of Law Lab team that worked on the gamification report (photo by Nina Haikara)

To better understand how the apps worked, Allen downloaded a dating app, made a profile and refused potential matches.

“I kept track of the kind of push notifications I was receiving: ‘You would have more matches if you did this on the app.’ ‘You should buy a boost; more people will be able to see you.’”

Allen says people who don't know how the algorithm works are being induced to engage in a particular behavior, which can have unexpected legal consequences.

“The coercive element comes in when users are being told to use the app more. If no one is seeing them, then they must pay to increase their visibility and it’s only temporary,” Allen says. “So, you keep paying to get equal access to the same service that others can access for free.”

The potential for legal injury will only grow as dating apps become more popular, Allen says.

Fellow first-year law student Samir Reynolds, for his part, studied the design techniques of ride-sharing apps.

“There’s been a lot of press coverage about whether drivers are employees, independent contractors or another classification,” says Reynolds, who has a bachelor's degree in knowledge integration, math and political science from the University of Waterloo. “However, these apps can nudge drivers to work at specific places over specific times. If we think of them as effectively telling their drivers to do that, then the drivers start to look more like employees rather than independent contractors.”

Reynolds adds that, although consumers see prices surge when drivers are in high demand, that doesn't mean the drivers are paid more during peak hours.

“These apps also tend to set artificial goals. Someone will be driving and after a ride is done, they will receive a notification they’re only $6 away from making $80 on their shift. But then after you make that, it will switch to only $7 away from $90.”

Reynolds came to U of T Law after working in the technology sector for two years.

“One of the defining problems of this generation of lawyers is going to be figuring out how law and policy, both from the court and from policy-makers, can evolve and adapt with technology, which is inherently going to advance faster than the law,” he says.

The Future of Law Lab was established in 2020 – thanks to a gift by Hal Jackman, a former U of T chancellor and lieutenant-governor of Ontario – as a co-curricular program to bring together students, academics, lawyers and other professionals to explore the intersection of law, innovation and technology.

"The Future of Law Lab provides students with an opportunity to learn about legal problems from a holistic perspective. Law does not exist in a vacuum and, in our context, legal problems are often business problems,” says lab director Joshua Morrison, a lawyer and graduate of the faculty’s global professional master of laws program with a concentration in innovation, law and technology.

“We ask our students to consider the strategic, operational, marketing, privacy and technology aspects of a particular situation. We’re encouraging them to be innovative and solutions-oriented, while encouraging collaboration among professionals of different disciplines.”

The researchers say a digital choice environment that leads people to situations where there are risks of harm should be treated the same way as a business that creates a physical choice environment that leads to the same consequences.

“Law is always trying to catch up to innovation,” Sarro says. “One of the ways it keeps up is by staying flexible. And we can leverage that flexibility to respond to many of the challenges posed by gamification.”

“If you're using gamification to influence people to work where you want them to work, and for how long, law can say that’s a form of control, and so we factor it in when assessing whether these workers ought to be deemed employees. If you're a business that profits from guiding people to engage in behaviour that creates risks of harm to themselves or others, law can impose a duty to mitigate those risks,” Sarro says.

“Anti-discrimination law has a flexible set of principles that can be used to encourage apps to reflect on the effects design choices have on different groups of people and whether those design choices are helping to mitigate risks of discrimination or are amplifying those risks.”

Allen sees the research project with the Future of Law Lab as good training for a career in technology law following graduation.

“The Future of Law Lab is the best place to set myself up for success,” she says.

AI, tech and social justice: U of T Groundbreakers EP4

Christopher.Sorensen — Thu, 13 Jan 2022 19:05:26 +0000

AI, tech and social justice: U of T Groundbreakers EP4 Christopher.Sorensen Thu, 01/13/2022 - 14:05

Topic

Our Community

Anti-Black Racism

Groundbreakers

Institutional Strategic Initiatives

Technology and Society

Alumni

Artificial Intelligence

Faculty of Applied Science & Engineering

Indigenous

U of T Mississauga

What is the relationship between pollution and colonialism in Canada? How can AI and related technologies avoid perpetuating racism and gender bias?

These are some of the questions explored in episode four of the Groundbreakers video series when host Ainka Jess speaks with researchers from two of the �ؿ�ζSM’s Institutional Strategic Initiatives: the Schwartz Reisman Institute for Technology and Society and the Black Research Network.

U of T Mississauga’s Kristen Bos, an assistant professor of Indigenous science and technology and the co-director of the Technoscience Research Unit, talks about The Land and Refinery project, while U of T Engineering alumna Deborah Raji discusses how bias in AI algorithms can perpetuate racism and gender bias and erode civil rights – and how access to technology can further inclusive excellence.

“As an Indigenous feminist and researcher, I know that the health of our lands is vital to the health of our bodies,” Bos says. “I feel like I have a responsibility. I think we all have a responsibility to hold the industries, the companies and the governments responsible for the creation of pollution and health harms.”

Groundbreakers is a multimedia series that includes articles at U of T News and features research leaders involved with U of T’s Institutional Strategic Initiatives, whose work will transform lives.

U of T sociologist studies women's careers in 'exciting and terrifying' tech sector

Christopher.Sorensen — Mon, 09 Dec 2019 21:41:55 +0000

U of T sociologist studies women's careers in 'exciting and terrifying' tech sector

Christopher.Sorensen Mon, 12/09/2019 - 16:41

Cutline

Sharla Alegria, an assistant professor in the department of sociology who studies racial and gender inequality, has taken a particular interest in the career trajectories of women in the technology sector (photo by Diana Tyszko)

Jovana Jankovic

Topic

Our Community

Technology and Society

Faculty of Arts & Science

Research & Innovation

Sociology

Sharla Alegria is working on work.

“I care an awful lot about work in general,” says the sociologist who joined the �ؿ�ζSM’s Faculty of Arts & Science this fall.

“Work is a huge part of our lives, of how we think about ourselves and compare ourselves to others. It’s also a driver of inequality because your job determines whether you can feed yourself and live a nice life.”

Alegria’s research delves primarily into racial and gender inequality. Her work attempts to evaluate how, why and in what form inequalities persist – and what the implications are for workers’ lives.

In particular, she’s taken an interest in the technology sector, studying the career trajectories of women in tech – a project detailed in a paper published in the October issue of the journal Gender and Society.

Alegria’s research found that women with a technical background – say, in engineering or computer science – may start out in technical jobs, but eventually get promoted into managerial positions often due to the stereotype that women have better interpersonal and communications skills.

While a promotion may seem like a good thing, Alegria says that there are still processes of exclusion at work.

“Having women in leadership positions is important,” says Alegria, who is an assistant professor in the department of sociology. “But they tend to get stuck in mid-level manager positions – never going high enough to make meaningful structural differences in the company culture. Plus, they often accept these promotions to avoid dealing with technical co-workers doubting their competence in a hostile work environment.”

Moreover, the notion that technical jobs like engineering do not require interpersonal skills is itself a grave oversight, says Alegria.

“Engineers have to talk to people to figure out how the product being built will be used,” says Alegria. “That communicative aspect of engineering tends to get ignored or partitioned off into someone else’s job.”

Alegria’s interest in sociology was instant when she took an introductory course early in her undergraduate studies at Vassar College, a small liberal arts school in upstate New York.

“I realized sociology could give me a language to talk about inequality. I had never had that. It was what I wanted most in the world. And it’s still the thing I most want in the world.”

Her work on the tech sector is part of her larger interest in so-called “knowledge work,” the industries and labour processes where knowledge and information are produced and shared – all of which are inextricable from technology.

Researching a sector where there is rapid change is “exciting and terrifying,” says Alegria. “In addition to what technology does, I want to think about structural innovations in how we organize work. Sure, we can get cars to drive themselves, but what does that mean for workers?”

Alegria joins a growing consensus that the world of work is changing – and fast. Companies are outsourcing labour to contract workers, “and that means that there are fewer full-time positions with benefits and things like that,” she says.

While there are some advantages to tech companies creating more flexible, dynamic workplaces, “jobs are definitely more precarious,” adds Alegria.

Asked what advice she’d give students and young workers entering the job market today, Alegria says it’s “important to collect as many skills as you can. Learn to write well, learn to use data. Learn to give a good, thoughtful presentation. These things are important in any job. And organize for labour rights – that’s a big thing for sure.”

But most importantly, she says, “learn how to learn. The world changes so fast that you need to be able to retool and recalibrate your skills.”

In the meantime, Alegria is conveying her enthusiasm for her research inside the classroom.

On the first day of her third-year course on race, class and gender this semester, Alegria showed students a photo of technicians, both male and female, working on the first electronic general-purpose computer, the ENIAC machine.

Originally published in Popular Science magazine, the photo was used by the U.S. Army as a recruitment tool in 1946, at which point the female workers were cropped out of the photo – perpetuating the stereotype that computers are “men’s work.”

“When I showed the students, there was an audible gasp,” says Alegria.

“We then had a great conversation about how women were literally deleted from the picture. And, of course, we still only see white workers on the machine. So, I try to get students to consider how we think about who does what kind of work.”

Next up for Alegria: continuing to get to know U of T and the city she and her partner – also a sociologist – now call home.

“U of T has one of the top sociology programs in the world, so it’s a very exciting place to be,” she says. “I have so many new colleagues whose work I’ve been reading for years and I’m so excited that I get to connect with them. It’s an incredibly vibrant intellectual space.”