Schwartz Reisman Institute for Technology and Society

Schwartz Reisman Institute for Technology and Society

Faculty of Applied Science & Engineering

Subheadline

Roger Grosse and David Duvenaud named Schwartz Reisman Chairs in Technology and Society, while David Lie becomes the institute’s new director

A leading expert in cybersecurity and two renowned AI safety researchers are set to take on leading roles at the ��ɫֱ��’s Schwartz Reisman Institute for Technology and Society.

David Lie, who is known for his seminal work that led to modern trusted execution processor architectures, has been named the new director of the Schwartz Reisman Institute (SRI), which aims to explore and address the ethical and societal implications of artificial intelligence and other emerging technologies.

His four-year appointment, which takes effect July 1, coincides with two renowned experts in AI safety – Roger Grosse and David Duvenaud – being named Schwartz Reisman Chairs in Technology and Society for five-year terms.

“I think one of the top priorities is ensuring that SRI and ��ɫֱ�� are the primary places in Canada – and perhaps in the world – for AI safety discussion and research,” says Lie, a professor in the Faculty of Applied Science & Engineering’s Edward S. Rogers Sr. department of electrical and computer engineering.

“My vision is to make us one of the leaders. Canada has already contributed greatly to machine learning and AI through the contributions of previous scholars like [University Professor Emeritus] Geoffrey Hinton, and I think we have a very strong role to play in this important technology going forward.”

The appointments come as inaugural director and chair Gillian Hadfield prepares to conclude her term as chair this month (she stepped down as director at the end of last year). The institute, created following a historic gift in 2019 from business leaders Gerald Schwartz and Heather Reisman, brings together experts from disciplines across ��ɫֱ��’s three campuses to steer AI development to prioritize safety and human welfare.

“We are thrilled to welcome David Lie, Roger Grosse and David Duvenaud to their new roles at the Schwartz Reisman Institute,” says Melanie Woodin, dean of the Faculty of Arts & Science. “Their expertise and leadership will be instrumental in fostering the interdisciplinary collaboration needed for the ��ɫֱ�� to remain at the forefront of technological innovation that benefits humanity.”

Lie, who has served as a research lead at SRI and holds cross-appointments in the department of computer science and the Faculty of Law, says his decades of research on making computer systems more secure and trustworthy – including contributions to computer architecture, formal verification, techniques using operating systems and networking – have equipped him to tackle the complex issues posed by AI, which will require researchers to anticipate and adapt to the unexpected.

“As AI become more powerful, they may do things – or are already doing things – that we didn’t anticipate or expect,” says Lie. “Bringing cybersecurity skills, thinking and tools into the AI safety discussion will be absolutely critical to solving the problem.”

Lie emphasizes that interdisciplinary collaboration is key to addressing potential AI disruption, noting that it has been pivotal in his own research and other roles.

His current research focuses on securing mobile platforms, cloud computing security and bridging the divide between technology and policy. He is also an associate director at the Data Sciences Institute, a ��ɫֱ�� institutional strategic initiative, a faculty affiliate at the Vector Institute for Artificial Intelligence and a senior fellow at Massey College.

“It’s really one of the things that I love about a place like ��ɫֱ��, because it's big and you have experts in every imaginable field to collaborate with,” he says. “I feel very strongly that we can always accomplish way more together than we can individually. That's true for people, but that's also true for disciplines.”

As incoming Schwartz Reisman Chairs in Technology and Society, Grosse and Duvenaud have vital roles to play in driving SRI’s research agenda and sharing its findings with the world, says Lie.

“One of the main ways universities contribute to society is through research, but we also contribute through discourse; we contribute by translating knowledge and providing that to policymakers, decision-makers and stakeholders,” he says. “I see SRI playing an important part in these roles.”

Both Grosse and Duvenaud are associate professors of computer science in the Faculty of Arts & Science, faculty affiliates at SRI, founding members of the Vector Institute and Canada CIFAR AI chairs – and both are working at San Francisco-based Anthropic, a research company focused on AI safety and alignment.

Grosse, whose research applies our understanding of deep learning to the safety and alignment of AI systems, says academia has an essential role to play in guiding AI development by looking beyond short-term incentives to ask how these technologies can be safely and ethically integrated for the long-term benefit of humanity.

“I'm very excited to be able to understand and mitigate catastrophic risks from AI, to be part of an interdisciplinary community that's especially well positioned to make progress in these issues, and I really appreciate the leadership that donors are showing and supporting this work,” he says.

“I think academia is great for being able to ask the more fundamental questions, to carry out maybe more forward-looking research that might not be directly on a company's critical path, but will contribute to safety efforts at many different organizations.”

Duvenaud’s research, meanwhile, focuses on probabilistic deep learning, artificial general intelligence governance and dangerous capabilities evaluation.

He envisions SRI as a “centre of gravity” where academics, industry members, government leaders and other stakeholders can engage with each other and shape the future of AI technologies.

“The idea is that by having this institute dedicated to this direction, we’ll be able to do things like host visitors and engage with academics from all sorts of disciplines –such as law, economics, and other parts of civil society – so that, ultimately, when policy discussions come up, we’ll be equipped and credible as people who can help governments navigate these decisions,” says Duvenaud, who is cross appointed to the department of statistical sciences.

Sheila McIlraith, an associate director and research lead at SRI, professor of computer science in the Faculty of Arts & Science, and a Canada CIFAR AI Chair at the Vector Institute, underlines the importance of rallying diverse disciplinary experts from across ��ɫֱ�� to address the opportunities and challenges that AI will wield in the coming years.

“AI is no longer the sole purview of computer scientists. It is reshaping the way we live, work, and interact with each other, and it will take experts from a broad range of disciplines to help ensure that AI is developed and deployed for the benefit of humanity, and that Canada adapts swiftly to protect our institutions," says McIlraith, who is an expert in AI safety research herself.

“Threats are already upon us; now is the time to act.”

What Now? AI, Episode 5: This Is Not Real

mattimar — Thu, 23 May 2024 19:33:08 +0000

What Now? AI, Episode 5: This Is Not Real mattimar Thu, 05/23/2024 - 15:33

Topic

Schwartz Reisman Institute for Technology and Society

Munk School of Global Affairs & Public Policy

Political Science

Artificial intelligence presents new opportunities to strengthen democracy even as it threatens to cast a shadow over election integrity and further the spread of misinformation.

In the fifth episode of What Now? AI, ��ɫֱ�� hosts Beth Coleman and Rahul Krishnan are joined by experts Harper Reed and Peter Loewen, who is also from ��ɫֱ��, to explore the impact of AI on the political realm.

Listen to episode five on Apple, Spotify, SoundCloud, iHeartRadio and Amazon. Watch episode five on YouTube.

Loewen, director of ��ɫֱ��’s Munk School of Global Affairs & Public Policy and a professor in the department of political science in the Faculty of Arts & Science, explains how AI removes the human touch from politics, potentially making the public uneasy.

“We still don't like the fact that it might be a machine that we're talking to,” said Loewen, who is also the associate director of the Schwartz Reisman Institute for Technology and Society.

“But then if you layer on this dimension of not knowing if this is actually the campaign that’s doing it, I think that’s probably orders of magnitude worse because what it does is it takes us from the realm of kind of feeling uneasy about something into feeling like this thing is corrupted.”

Reed, meanwhile, spoke about his experience as the chief technology officer on former U.S. president Barack Obama’s re-election campaign in 2012.

“The technology we built was not about convincing someone at the time that Mitt Romney was a bad person or a good person,” said Reed during a conversation with Coleman about AI and democracy that was filmed live at the Schwartz Reisman Institute for Technology and Society’s annual conference Absolutely Interdisciplinary, a portion of which was used in the podcast episode.

“The tech was more about making sure you got to vote.”

When asked about the biggest threats to AI and democracy, Reed emphasized that he is less worried about the technology itself and more with ensuring it’s beneficial to societal use.

“I’m worried about who has access to it and how they are using it.”

About the hosts:

Beth Coleman is an associate professor at ��ɫֱ�� Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information. She is also a research lead on AI policy and praxis at the Schwartz Reisman Institute for Technology and Society. Coleman authored Reality Was Whatever Happened: Octavia Butler AI and Other Possible Worlds using art and generative AI.

Rahul Krishnan is an assistant professor in ��ɫֱ��’s department of computer science in the Faculty of Arts & Science and department of laboratory medicine and pathobiology in the Temerty Faculty of Medicine. He is a Canada CIFAR Chair at the Vector Institute, a faculty affiliate at the Schwartz Reisman Institute for Technology and Society and a faculty member at the Temerty Centre for AI Research and Education in Medicine (T-CAIREM).

Note: The artwork in the background of Peter Loewen’s interview belong to the Mirvish Family’s private collection. The large image, titled Floating Free, is by K.M. Graham. The smaller image is untitled and by the same artist.

��ɫֱ�� initiative encourages computer science students to incorporate ethics into their work

Christopher.Sorensen — Fri, 26 Apr 2024 19:22:07 +0000

��ɫֱ�� initiative encourages computer science students to incorporate ethics into their work

Christopher.Sorensen Fri, 04/26/2024 - 15:22

Cutline

(photo by urbazon/Getty Images)

Krystle Hewitt

Topic

Schwartz Reisman Institute for Technology and Society

Subheadline

Total enrolment in courses featuring Embedded Ethics Education Initiative modules exceeded 8,000 students this year

Computer science students at the ��ɫֱ�� are learning how to incorporate ethical considerations into the design and development of new technologies such as artificial intelligence with the help of a unique undergraduate initiative.

The Embedded Ethics Education Initiative (E3I) aims to provide students with the ability to critically assess the societal impacts of the technologies they will be designing and developing throughout their careers. That includes grappling with issues such as AI safety, data privacy and misinformation.

Program co-creator Sheila McIlraith, a professor in the department of computer science in the Faculty of Arts & Science and an associate director at the Schwartz Reisman Institute for Technology and Society (SRI), says E3I aims to help students “recognize the broader ramifications of the technology they’re developing on diverse stakeholders, and to avoid or mitigate any negative impact.”

First launched in 2020 as a two-year pilot program, the initiative is a collaborative venture between the department of computer science and SRI in association with the department of philosophy. It integrates ethics modules into select undergraduate computer science courses – and has reached thousands of ��ɫֱ�� students in this academic year alone.

Malaikah Hussain is one of the many ��ɫֱ�� students who has benefited from the initiative. As a first-year student enrolled in CSC111: Foundations of Computer Science II, she participated in an E3I module that explored how a data structure she learned about in class laid the foundation of a contact tracing system and raised ethical issues concerning data collection.

“The modules underlined how the software design choices we make extend beyond computing efficiency concerns to grave ethical concerns such as privacy,” says Hussain, who is now a third-year computer science specialist.

Hussain adds that the modules propelled her interest in ethics and computing, leading her to pursue upper year courses on the topic. During a subsequent internship, she organized an event about the ethics surrounding e-waste disposal and the company’s technology life cycle.

“The E3I modules have been crucial in shaping my approach to my studies and work, emphasizing the importance of ethics in every aspect of computing,” she says.

The program, which initially reached 400 students, has seen significant growth over the last four years. This academic year alone, total enrolment in computer science courses with E3I programming has exceeded 8,000 students. Another 1,500 students participated in E3I programming in courses outside computer science.

Clockwise from top left: Steven Coyne, Diane Horton, David Liu and Sheila McIlraith (supplied images)

In recognition of the program’s impact on the undergraduate student learning experience, McIlraith and her colleagues – Diane Horton and David Liu, a professor and associate professor, teaching stream, respectively, in the department of computer science, and Steven Coyne, an assistant professor who is jointly appointed to the departments of computer science and philosophy – were recently recognized with the 2024 Northrop Frye Award (Team), one of the prestigious ��ɫֱ�� Alumni Association Awards of Excellence.

Horton, who leads the initiative’s assessment efforts, points to the team’s recently published paper showing that after participating in modules in only one or two courses, students are inspired to learn more about ethics and are benefiting in the workplace.

“We have evidence that they are better able to identify ethical issues arising in their work, and that the modules help them navigate those issues,” she says.

Horton adds that the findings build on earlier assessment work showing that after experiencing modules in only one course, students became more interested in ethics and tech, and more confident in their ability to deal with ethical issues they might encounter.

The team says the initiative’s interdisciplinary nature is key to delivering both a curriculum and experience with an authentic voice, giving instructors and students the vocabulary and depth of knowledge to engage on issues such as privacy, well-being and harm.

“As a philosopher and ethicist, I love teaching in a computer science department,” says Coyne. “My colleagues teach me about interesting ethical problems that they’ve found in their class material, and I get to reciprocate by finding distinctions and ideas that illuminate those problems. And we learn a lot from each other – intellectually and pedagogically – when we design a module for that class together.”

E3I is founded upon three key principles: teach students how – not what – to think; encourage ethics-informed design choices as a design principle; and make discussions safe, not personal.

“Engaging with students and making them feel safe, not proselytizing, inviting the students to participate is especially important,” says Liu.

The modules support this type of learning environment by using stakeholders with fictional character profiles that include names, pictures and a backstory.

“Fictional stakeholders help add a layer of distance so students can think through the issues without having to say, ‘This is what I think,’” Horton says. “Stakeholders also increase their awareness of the different kinds of people who might be impacted.”

McIlraith adds that having students advocate for an opinion that is not necessarily their own encourages empathy, while Liu notes that many have a “real hunger” to learn about the ethical considerations of their work.

“An increasing number of students are thinking, ‘I want to be trained as a computer scientist and I want to use my skills after graduation,’ but also ‘I want to do something that I think will make a positive impact on the world,’” he says.

Together, the E3I team works with course instructors to develop educational modules that tightly pair ethical concepts with course-specific technical material. In an applied software design course, for example, students learn about accessible software and disability theory; in a theoretical algorithms course, they learn about algorithmic fairness and distributive justice; and in a game design course, they learn about addiction and consent.

Steve Engels, a computer science professor, teaching stream, says integrating an ethics module about addiction into his fourth-year capstone course on video game design felt like a natural extension of his lecture topic on ludology – in particular, the psychological techniques used to make games compelling – instead of something that felt artificially inserted into the course.

“Project-based courses can sometimes compel students to focus primarily on the final product of the course, but this module provided an opportunity to pause and reflect on what they were doing and why,” Engels says. “It forced them to confront their role in the important and current issue of gaming addiction, so they would be more aware of the ethical implications of their future work and thus be better equipped to handle it.”

By next year, each undergraduate computer science student will encounter E3I modules in at least one or two courses every year throughout their program. The team is also exploring the adoption of the E3I model in other STEM disciplines, from ecology to statistics. Beyond ��ɫֱ��, the team plans to share their expertise with other Canadian universities that are interested in developing a similar program.

“This initiative is having a huge impact,” McIlraith says. “You see it in the number of students we’re reaching and in our assessment results. But it’s more than that – we’re instigating a culture change.”

What Now? AI, Episode 4: AI and Creativity

mattimar — Thu, 25 Apr 2024 21:22:39 +0000

What Now? AI, Episode 4: AI and Creativity mattimar Thu, 04/25/2024 - 17:22

Topic

Schwartz Reisman Institute for Technology and Society

Alumni

Startups

The creative industry is poised to be forever changed by artificial intelligence.

Writing tools like ChatGPT and image generators like Midjourney and DALL-E have exploded into the mainstream. Adobe built its own version of generative AI technology for its creative suites and OpenAI announced its text-to-video model, Sora.

What impact will these tools and models have on the creative process? How will they change the role of an artist?

In the fourth episode of What Now? AI, hosts Beth Coleman and Rahul Krishnan dive into these questions with AI researchers Sanja Fidler of the ��ɫֱ�� and Nick Frosst, who co-founded the startup Cohere.

Listen to episode four on  Apple, Spotify, SoundCloud, iHeartRadio and Amazon. Watch episode four on YouTube. 

Fidler, the vice president of AI research at NVIDIA and an associate professor of mathematical and computational sciences at ��ɫֱ�� Mississauga, says that while AI technology is still in its early stage, it has the potential to provide artists with more adaptability and creative control.

“When artists see these methods like text-to-x, text-to-image or text-to-video, I feel that they have pushback because now there is only text that allows you to control the content,” says Fidler, an affiliate faculty member at the Vector Institute, which she co-founded.

“I think artists do want to have this iterative creative control. They have some idea in their head, and they have all these tools that allowed them to go from that idea into the final product. We want to do the same thing with AI as well.”

Frosst, who sings in the band Good Kid, says he doesn’t use large language models to help him write songs – only to help analyze lyrics and themes.

“I’m not really looking to optimize my artistic expression,” says Frosst, who completed his undergraduate degree in computer science and cognitive science at ��ɫֱ��.

“I don’t really want to write a new Good Kid song and be less involved. I want to be more involved.”

Frosst believes AI will change the way art is created, but not to the point where people aren’t interested in the artists who are making it.

“We want to know who made it, and that’s mostly what’s enjoyable about it.”

About the hosts:

What Now? AI, Episode 3: Innovation for Good

Christopher.Sorensen — Thu, 18 Apr 2024 20:47:11 +0000

What Now? AI, Episode 3: Innovation for Good Christopher.Sorensen Thu, 04/18/2024 - 16:47

Topic

Unity Health

Schwartz Reisman Institute for Technology and Society

Leslie Dan Faculty of Pharmacy

Startups

While the news headlines about AI often focus on dangers and risks, the potential for life-saving innovation in fields such as health care is huge.

In the third episode of What Now? AI, co-hosts Beth Coleman and Rahul Krishnan of the ��ɫֱ�� are joined by experts Christine Allen and Andrew Pinto to discuss AI’s potential to advance drug development and dramatically improve primary care.

Listen to episode three on Apple, Spotify, SoundCloud, iHeartRadio and Amazon. Watch episode three on YouTube.

Allen, a professor in ��ɫֱ��’s Leslie Dan Faculty of Pharmacy and an expert in drug formulation and development, co-founded Intrepid Labs Inc. with Alán Aspuru-Guzik, a professor in the departments of chemistry and computer science in ��ɫֱ��’s Faculty of Arts & Science.

One of the first startups to emerge from the Acceleration Consortium at ��ɫֱ��, Intrepid Labs is accelerating pharmaceutical drug development through the integration of AI, automation and advanced computing.

“It’s this concept of using AI to explore the unexplored,” Allen says.

“What if that formulation could really transform the properties and performance of your drug is one of those unexplored formulations. That will then take that drug through clinical development smoothly and get it to patients faster, which is really the goal.”

Meanwhile, Pinto, a family physician at St. Michael’s Hospital and director of the Upstream Lab, Unity Health Toronto, focuses his research on addressing the social determinants of health by running clinical trials of interventions and using AI tools for surveillance of respiratory illness.

“When we started to do this work around AI in primary care, we wanted to be directed by primary care providers and patients,” says Pinto, an associate professor in the department of family and community medicine in ��ɫֱ��’s Temerty Faculty of Medicine and at the Dalla Lana School of Public Health.

Reducing health-care inequities is top of mind for Pinto. His lab focuses on implementing AI to prioritize community engagement and bridge socioeconomic gaps to mitigate biases.

“We’re using these tools to look at all of the patients in a population and then focus our attention on the people who need it most.”

About the hosts:

What Now? AI, Episode 2: Safe and Accountable

Christopher.Sorensen — Thu, 11 Apr 2024 12:57:17 +0000

What Now? AI, Episode 2: Safe and Accountable Christopher.Sorensen Thu, 04/11/2024 - 08:57

Mariam Matti

Topic

Schwartz Reisman Institute for Technology and Society

As AI becomes more integrated into our lives, how do we make sure it aligns with our values while minimizing risks?

In the second episode of What Now? AI, hosts Beth Coleman and Rahul Krishnan are joined by ��ɫֱ�� experts Gillian Hadfield and Roger Grosse as they tackle critical questions surrounding AI safety, regulation and alignment.

Listen to episode two on Apple, Spotify, SoundCloud, iHeartRadio and Amazon. Watch episode two on YouTube.

Grosse, an associate professor of computer science in the Faculty of Arts & Science and a founding member of the Vector Institute, joined the technical staff on the alignment team at Anthropic, an AI safety and research company based in San Francisco, during a sabbatical last year.

He calls working on AI research and systems while investigating safety a “difficult needle to thread.”

“As you move up the ladder of different AI capabilities, new requirements start kicking in – in terms of keeping the models secure from bad actors and being able to make sure they won’t intentionally carry out harmful plans,” says Grosse, a faculty affiliate at the Schwartz Reisman Institute for Technology and Society.

Hadfield, a professor of law and strategic management in the Faculty of Law and the inaugural Schwartz Reisman Chair in Technology and Society, has proposed a national registry for large AI models. She thinks companies should disclose to governments what they’re building, the data being used and the AI model’s capabilities.

“This is a unique moment in human history,” says Hadfield, who holds a CIFAR AI Chair at the Vector Institute for AI and served as a senior policy adviser to OpenAI from 2018 to 2023. “I think this is the first time that you have such a powerful technology that is being developed almost exclusively within private technology companies, so the public and the academic sector don’t have full visibility into how the technology is working.”

About the hosts:

Beth Coleman is an associate professor at ��ɫֱ�� Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information. She is also a research lead on AI policy and praxis at the Schwartz Reisman Institute for Technology and Society. Coleman authored Reality Was Whatever Happened: Octavia Butler AI and Other Possible Worlds using art and generative AI.

Rahul Krishnan is an assistant professor in ��ɫֱ��’s department of computer science in the Faculty of Arts & Science and department of laboratory medicine and pathobiology in the Temerty Faculty of Medicine. He is a Canada CIFAR Chair at the Vector Institute, a faculty affiliate at the Schwartz Reisman Institute for Technology and Society and a faculty member at the Temerty Centre for AI Research and Education in Medicine (T-CAIREM).

How will AI change our world? ��ɫֱ�� podcast explores technology’s impact on society

Christopher.Sorensen — Thu, 04 Apr 2024 15:17:45 +0000

How will AI change our world? ��ɫֱ�� podcast explores technology’s impact on society

Christopher.Sorensen Thu, 04/04/2024 - 11:17

Mariam Matti

Topic

Unity Health

Schwartz Reisman Institute for Technology and Society

Alumni

CIFAR

Dalla Lana School of Public Health

Leslie Dan Faculty of Pharmacy

Geoffrey Hinton

Startups

Subheadline

In What Now? AI, hosts Beth Coleman and Rahul Krishnan explore – and demystify – artificial intelligence and its impact on society with the help of leading experts

Rapid advances in artificial intelligence are posing profound questions about the future – and about us.

Can we ensure safety and alignment within AI systems? How might AI forever transform fields like health care? What ripple effects could AI have on jobs and livelihoods, including in creative industries?

��ɫֱ�� researchers Beth Coleman and Rahul Krishnan explore – and demystify – these and other topics by tapping into the knowledge of leading AI experts in What Now? AI, a new ��ɫֱ�� podcast that launches this week.

It can be found on Apple, Spotify, Soundcloud, iHeartRadio and Amazon.

An associate professor at ��ɫֱ�� Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information, Coleman says she hopes the episodes help audiences make sense of new AI tools and systems by cutting through “all the noisiness and controversy that has taken over the headlines.”

“It can be complex and technical, but it’s also social,” says Coleman, a research lead on AI policy and praxis at the Schwartz Reisman Institute for Technology & Society.

“What we do with AI makes a difference and more people need to be able to share that knowledge.”

Coleman’s own research centres around technology and society with a focus on data and cities, AI and policy, and generative arts. Inspired by Octavia Butler’s 1980 Xenogenesis trilogy, Coleman authored Reality Was Whatever Happened: Octavia Butler AI and Other Possible Worlds using art and generative AI.

Krishnan, meanwhile, is an assistant professor in ��ɫֱ��’s department of computer science in the Faculty of Arts & Science and department of laboratory medicine and pathobiology in the Temerty Faculty of Medicine. A Canada CIFAR AI Chair at the Vector Institute and Canada Research Chair in computational medicine, Krishnan and his team focus on teaching neural networks about causality, building deep learning models that analyze cause and effect from data.

“I’m excited to co-host this podcast to explore and demystify for a broader audience AI through the lens of an accomplished and diverse set of experts,” says Krishnan, who is also a faculty affiliate at the Schwartz Reisman Institute for Technology and Society and a faculty member at the Temerty Centre for AI Research and Education in Medicine (T-CAIREM).

What Now? AI picks up where the conversation started last year by Geoffrey Hinton, the cognitive psychologist and University Professor emeritus of computer science who is known as the “Godfather of AI.” After a lifetime spent developing a type of AI known as deep learning, Hinton stepped back from his role at Google to warn about the existential threats of unchecked AI development.

Since then, there have been ongoing advancements in AI research, technological applications and policy development.

Coleman and Krishnan will tackle these and other topics with guests:

Gillian Hadfield, professor of law and strategic management at the Faculty of Law and the Schwartz Reisman Chair in Technology and Society.
Roger Grosse, associate professor of computer science in the Faculty of Arts & Science and founding member of the Vector Institute.
Christine Allen, professor at the Leslie Dan Faculty of Pharmacy and co-founder and CEO of Intrepid Labs Inc.
Andrew Pinto, a family physician at St. Michael’s Hospital, Unity Health Toronto, and associate professor in the Temerty Faculty of Medicine and the Dalla Lana School of Public Health.
Nick Frosst, co-founder of Cohere, singer in Good Kid band and a ��ɫֱ�� computer science and cognitive science alumnus.

“The What Now? AI podcast highlights the incredible researchers at the ��ɫֱ�� who are exploring the profound implications of this transformative technology,” says Leah Cowen, ��ɫֱ��’s vice-president, research and innovation, and strategic initiatives. “These discussions tackle critical questions surrounding AI safety and alignment and its myriad implications across various domains.

“The university is committed to fostering informed discussions that will shape our collective understanding of AI’s role in our society and in our future.”

Coleman says she hopes listeners come away from the podcast feeling more grounded.

Krishnan, for his part, wants the audience to understand “that there is no one group that has ownership” over the technology” and that “the free exchange of ideas and open-source tools encourage people from all disciplines to come see how accessible AI can be, what AI can do for them and how they can advance the discourse in the field.”

Four AI trends to watch in 2024

Christopher.Sorensen — Fri, 19 Jan 2024 17:02:40 +0000

Four AI trends to watch in 2024

Christopher.Sorensen Fri, 01/19/2024 - 12:02

Cutline

AI was a hot topic at this week’s annual meeting of the World Economic Forum in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket via Getty Images)

Jovana Jankovic

Daniel Browne

Topic

Schwartz Reisman Institute for Technology and Society

Munk School of Global Affairs & Public Policy

Faculty of Applied Science & Engineering

Global

Graduate Students

Rotman School of Management

Subheadline

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions”

As artificial intelligence continues to develop rapidly, the world is watching with excitement and apprehension – as evidenced by the AI buzz in Davos this week at the World Economic Forum’s annual meeting.

��ɫֱ�� researchers are using AI to advance scientific discovery and improve health-care delivery, exploring how to mitigate potential harms and finding new ways to ensure the technology aligns with human values.

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions,” says Monique Crichlow, executive director of the Schwartz Reisman Institute for Technology and Society (SRI). “Researchers at SRI and across the university are tackling how to build and regulate AI systems for safer outcomes, as well as the social impacts of these powerful technologies.”

“From health-care delivery to accessible financial and legal services, AI has the potential to benefit society in many ways and tackle inequality around the world. But we have real work to do in 2024 to ensure that happens safely.”

As AI continues to reshape industries and challenge many aspects of society, here are four emerging themes ��ɫֱ�� researchers are keeping their eyes on in 2024:

1. AI regulation is on its way

U.S. Vice President Kamala Harris applauds as U.S. President Joe Biden signs an executive order on the safe, secure, and trustworthy development and use of artificial intelligence on Oct. 30, 2023 (photo by Brendan Simialowski/AFP/Getty Images)

As a technology with a wide range of potential applications, AI has the potential to impact all aspects of society – and regulators around the world are scrambling to catch up.

Set to pass later this year, the Artificial Intelligence and Data Act (AIDA) is the Canadian government’s first attempt to comprehensively regulate AI. Similar attempts by other governments include the European Union’s AI Act and the Algorithmic Accountability Act in the United States.

But there is still much to be done.

In the coming year, legislators and policymakers in Canada will tackle many questions, including what counts as fair use when it comes to training data and what privacy means in the 21st century. Is it illegal for companies to train AI systems on copyrighted data, as a recent lawsuit from the New York Times alleges? Who owns the rights to AI-generated artworks? Will Canada’s new privacy bill sufficiently protect citizens’ biometric data?

On top of this, AI’s entry into other sectors and industries will increasingly affect and transform how we regulate other products and services. As Gillian Hadfield, a professor in the Faculty of Law and the Schwartz Reisman Chair in Technology and Society, Policy Researcher Jamie Sandhu and Faculty of Law doctorial candidate Noam Kolt explore in a recent policy brief for CIFAR (formerly the Canadian Institute for Advanced Research), a focus on regulating AI through its harms and risks alone “obscures the bigger picture” of how these systems will transform other industries and society as a whole. For example: are current car safety regulations adequate to account for self-driving vehicles powered by AI?

2. The use of generative AI will continue to stir up controversy

Microsoft Bing Image Creator is displayed on a smartphone (photo by Jonathan Raa/NurPhoto/Getty Images)

From AI-generated text and pictures to videos and music, use of generative AI has exploded over the past year – and so have questions surrounding issues such as academic integrity, misinformation and the displacement of creative workers.

In the classroom, teachers are seeking to understand how education is evolving in the age of machine learning. Instructors will need to find new ways to embrace these tools – or perhaps opt to reject them altogether – and students will continue to discover new ways to learn alongside these systems.

At the same time, AI systems created more than 15 billion images last year by some counts – more than the entire 150-year history of photography. Online content will increasingly lack human authorship, and some researchers have proposed that by 2026 as much as 90 per cent of internet text could be generated by AI. Risks around disinformation will increase, and new methods to label content as trustworthy will be essential.

Many workers – including writers, translators, illustrators and designers – are worried about job losses. But a tidal wave of machine-generated text could also have negative impacts on AI development. In a recent study, Nicolas Papernot, an assistant professor in the Edward S. Rogers Sr. department of electrical and computer engineering in Faculty of Applied Science & Engineering and an SRI faculty affiliate, and his co-authors found training AI on machine-generated text led to the system becoming less reliable and subject to “model collapse.”

3. Public perception and trust of AI is shifting

A person walks past a temporary AI stall in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket/Getty Images)

Can we trust AI? Is our data secure?

Emerging research on public trust of AI is shedding light on changing preferences, desires and viewpoints. Peter Loewen – the director of the Munk School of Global Affairs & Public Policy, SRI’s associate director and the director of the Munk School’s Policy, Elections & Representation Lab (PEARL) – is developing an index measuring public perceptions of and attitudes towards AI technologies.

Loewen’s team conducted a representative survey of more than 23,000 people across 21 countries, examining attitudes towards regulation, AI development, perceived personal and societal economic impacts, specific emerging technologies such as ChatGPT and the use of AI by government. They plan to release their results soon.

Meanwhile, 2024 is being called “the biggest election year in history,” with more than 50 countries headed to the polls, and experts expect interference and misinformation to hit an all-time high thanks to AI. How will citizens know which information, candidates, and policies to trust?

In response, some researchers are investigating the foundations of trust itself. Beth Coleman, an associate professor at ��ɫֱ�� Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information who is an SRI research lead, is leading an interdisciplinary working group on the role of trust in interactions between humans and AI systems, examining how trust is conceptualized, earned and maintained in our interactions with the pivotal technology of our time.

4. AI will increasingly transform labour, markets and industries

A protester in London holds a placard during a rally in Leicester Square (photo by Vuk Valcic/SOPA Images/LightRocket via Getty Images)

Kristina McElheran, an assistant professor in the Rotman School of Management and an SRI researcher, and her collaborators may have recently found a gap between AI buzz in the workplace and businesses who are actually using it – but there remains a real possibility that labour, markets and industries will undergo massive transformation.

��ɫֱ�� researchers who have published books on how AI will transform industry include: Rotman faculty members Ajay Agrawal, Joshua Gans and Avi Goldfarb, whose Power and Prediction: The Disruptive Economics of Artificial Intelligence argues that “old ways of doing things will be upended” as AI predictions improve; and the Faculty of Law’s Benjamin Alarie and Abdi Aidid, who propose in The Legal Singularity: How Artificial Intelligence Can Make Law Radically Better that AI will improve legal services by increasing ease of access and fairness for individuals.

In 2024, institutions – public and private – will be creating more guidelines and rules around how AI systems can or cannot be used in their operations. Disruptors will be challenging the hierarchy of the current marketplace.

The coming year promises to be transformative for AI as it continues to find new applications across society. Experts and citizens must stay alert to the changes AI will bring and continue to advocate that ethical and responsible practices guide the development of this powerful technology.

Achieving alignment: How ��ɫֱ�� researchers are working to keep AI on track

bresgead — Fri, 08 Dec 2023 21:18:24 +0000

Achieving alignment: How ��ɫֱ�� researchers are working to keep AI on track

bresgead Fri, 12/08/2023 - 16:18

Cutline

Michael Zhang, a PhD student in computer science, says there are myriad reasons why AI models may not respect the intentions of their human creators (supplied image)

Adina Bresge

Topic

Institutional Strategic Initiatives

Schwartz Reisman Institute for Technology and Society

Graduate Students

Subheadline

Michael Zhang focuses on AI safety as a graduate researcher at ��ɫֱ��'s Schwartz Reisman Institute for Technology and Society

In the year since OpenAI released ChatGPT, what once seemed like an esoteric question among researchers has pushed its way to the forefront of public discourse: As artificial intelligence becomes more capable, how do we ensure AI systems act in the best interests of humans and – crucially – not turn against us?

This dilemma could determine the fate of humanity in the eyes of some researchers, including ��ɫֱ�� University Professor emeritus Geoffrey Hinton, known as the “godfather of AI,” who is warning that the technology he helped create could evolve into an existential threat. Others have raised alarm about nearer-term risks such as job losses, disinformation and AI-powered warfare.

Michael Zhang, a PhD student in computer science in ��ɫֱ��’s Faculty of Arts & Science, is focused on AI safety and interdisciplinary thinking about the technology as a graduate fellow at the Schwartz Reisman Institute for Technology and Society – and co-authored an article on the subject earlier this year.

He recently spoke with ��ɫֱ�� News about the alignment problem and what is being done to try and solve it.

What, exactly, is meant by AI alignment?

In the research sense, it means trying to make sure that AI does what we intended it to do – so it follows the objectives that we try to give it. But there are lots of problems that can arise, some of which we’re already seeing in today’s models.

One is called reward misspecification. It’s tricky to specify what reward function, or objective, you want in the form of a number that an AI model can understand. For example, if you’re a company, you might try to maximize profits – that’s a relatively simple objective. But in pursuing it, there can be unintended consequences in the real world. The model might make or recommend decisions that are harmful to employees or the environment. This example of rewards being underspecified can occur in even more simple settings. If we ask a robot to bring us coffee, we are also implicitly asking it to do so without breaking anything in the kitchen.

Another problem is bias. The AI model doesn’t have a mind of its own – it’s given a very strict mathematical objective. But we’re biased, and we generate data that’s biased, and that’s what we give our models. If there exists some underlying bias in the training dataset, the model will “learn” the bias because it best accomplishes that mathematical objective. We’ve already seen how this can lead to issues when we ask AI systems to make decisions such as whether someone should receive bail, or to do the first round of resume screening.

If we built the AI models, how is it they learn to do things we didn’t foresee?

When we talk about emergent behaviours – abilities that are present in larger models but not in smaller ones – it’s useful to think about large language models (LLMs) such as ChatGPT. If given an incomplete sentence, ChatGPT’s objective is to predict what the next word is going to be. But if you’re giving it a bunch of different training data – from the works of Shakespeare to mathematical textbooks – the model is going to gain some level of understanding in order to get better at predicting what word comes next.

We don’t specify hard-coded rules for what these models are supposed to learn, so we don’t have that much control over what the model generates. One example of this is hallucinations, where models such as ChatGPT create plausible but false claims.

What is artificial general intelligence (AGI) and what are some of the existential concerns about it?

There are many definitions, but in a general sense, AGI refers to the potential that we develop an AI system that performs most tasks that require intelligence better than or at the same level as humans.

People who believe this might happen are concerned about whether these models are going to be aligned with human values. In other words, if they’re more intelligent than the average human, it’s not clear that they’ll actually help us.

Some sci-fi ideas about AIs taking over the world or hurting a lot of humans are getting a lot of media attention. One reason people think this might happen is an AI can often act better on its objectives if it has more resources. Hypothetically, an AI system might decide that manipulating humans, or hurting them in some way, might make it easier to acquire resources. This scenario is not going to happen today, but the potential risk is why luminaries such as Geoffrey Hinton emphasize the importance of studying and better understanding the models we are training.

How are ��ɫֱ�� researchers working to tackle the short- and long-term risks of AI?

There are five key areas of AI alignment research: specification, interpretability, monitoring, robustness and governance. The Schwartz Reisman Institute is at the forefront of bringing together people from different disciplines to try to steer this technology in a positive direction.

In the case of specification, a common approach to fix the problem of reward misspecification is a technique that allows models to learn from human feedback. This is already being put into practice in training LLMs like ChatGPT. Going forward, some researchers are looking for ways to encode a set of human principles for future advanced models to follow. An important question that we can all think about is alignment to whom? What sort of guidelines do we want these models to follow?

Then there’s interpretability. A lot of these giant models, like ChatGPT, might have millions or even billions of parameters. These parameters take in an input and then compute a complicated mathematical function to give us the output – but we’re not always sure what happens in this “black box” in the middle. The goal of interpretability is to try to better understand how a model arrives at a given decision. For example, Roger Grosse, an associate professor in the department of computer science in the Faculty of Arts & Science and faculty affiliate at SRI, and his students are researching influence functions, which aims to understand which training examples are most responsible producing a certain output.

Another area is monitoring. Due to the presence of emergent behaviours, sometimes we don’t actually know what a new model is capable of until a bunch of different researchers and practitioners poke around and figure it out. This area of research aims to create systematic ways to understand how capable a model actually is. For example, PhD students Yangjun Ruan and Honghua Dong are among the ��ɫֱ�� researchers who co-authored a paper that used simulation testing to evaluate the safety risks that could arise from giving current LLMs access to tools such as email and bank accounts.

Robustness is a term that broadly refers to making sure that AI models are resistant to unusual events or manipulations by bad actors. That means models shouldn’t be sensitive to small changes and should behave consistently in various circumstances. SRI Faculty Affiliate Nicolas Papernot, an assistant professor in the [Edward S. Rogers Sr.] department of computer engineering in the Faculty of Applied Science & Engineering and the department of computer science, has been working on trustworthy machine learning, which seeks to address some of these challenges.

Finally, there’s governance. Many different countries are trying to develop rules on how we should regulate AI. For example, SRI Chair Gillian Hadfield has been influential in pushing for policies to curb the dangers of highly capable frontier AI models. There’s also research on the technical side about tools to hold AI developers accountable. PhD student Dami Choi and Associate Professor David Duvenaud recently co-authored a paper developing a method to check that a model is actually trained on the data that the organization claims.

Geoffrey Hinton fields questions from scholars, students during academic talk on responsible AI

Christopher.Sorensen — Thu, 02 Nov 2023 14:48:22 +0000

Geoffrey Hinton fields questions from scholars, students during academic talk on responsible AI

Christopher.Sorensen Thu, 11/02/2023 - 10:48

Cutline

Geoffrey Hinton, a University Professor Emeritus of computer science who has been dubbed the “Godfather of AI,” delivers an academic talk about artificial intelligence in ��ɫֱ��’s Convocation Hall (photo by Johnny Guatto)

Chris Sorensen

Topic

Institute of Biomedical Engineering

Schwartz Reisman Institute for Technology and Society

Faculty of Applied Science & Engineering

Deep Learning