This summer, OpenAI announced escaped a training environment and hacked into the AI company Hugging Face. Anthropic quickly followed with news that its AI agents also .Ģż

Last week, an outgoing Anthropic employee took to X, posting that the ā€œ.ā€ Such talk has for years, though many AI experts have argued that these Terminator-esque claims are distractions from the real risks posed by current AI systems. Nevertheless, that viral X thread is .ĢżĀ 

To help make sense of all this, UW News talked to five AI researchers from the ĀŅĀ×ÉēĒų:Ā 

• , associate professor in the Information School;
• , professor in the Information School;
• , professor in the Paul G. Allen School of Computer Science & Engineering;
• , professor in the Allen School and the UWā€™s vice provost for AI;
• and , professor in the Information School and the School of Law.

How alarming do you find the hacks announced by OpenAI and Anthropic?Ā 

Franziska Roesner: I do find them somewhat alarming ā€” not due to the hypothetical risks from an anthropomorphized runaway AI, but because complex interconnected systems are being built and seemingly run without much in the way of standard safeguards and auditing. The resulting outcomes are unsurprising to security experts, but are sensationalized as AI risk.

Ryan Calo: The timing makes me a little skeptical. Is OpenAI trying to match Anthropic by arguing that its systems are just as scary? Is Hugging Face trying to look relevant in advance of its purchase by Nvidia? But yes ā€” this sort of emergent behavior is concerning.

Noah A. Smith: Weā€™ve been told that the beast got out of the cage, but we donā€™t know enough about the cage the beast was in. The demonstrations may establish an important new capability in these AI models without establishing the broader risk people are inferring. Assessing the underlying risk depends on what access, scaffolding, permissions and safeguards the system had. shows that the alarming behavior depended heavily on what tools the model was given, what it was allowed to access, and how the experiment was set up, not just on the model itself.

Chirag Shah: Iā€™m in half-agreement with scholars like who warn that the big AI labs are creating this scare to distract us from real problems that AI is causing. I also concur with and others who have been warning us about the security threats posed by the frontier models. I donā€™t think these two viewpoints are mutually exclusive: Yes, there are many other potential harms being created by AI, but the hacks and other security issues are real too and could be more devastating. Worse, we may not have time or opportunity to react, fix or reverse.

Aylin Caliskan: When such a complex system is equipped with tools and capabilities that enable it to interact with other complex systems, we should expect unforeseen exploits, problems and unintended consequences by default. The safety of these systems needs to be rigorously evaluated under controlled conditions and in real time, and appropriate guardrails should be dynamically integrated while theyā€™re running.

What do you make of former Anthropic that, ā€œThe people building AI earnestly believe that it could kill us all by the end of the decadeā€?

RC: I worry engineers like Mr. Coxon are playing into an industry rhetoric that would have society focus on speculative, existential threats, rather than immediate, real-world harms. I argued as much in 2023 in .

The people making these claims and announcements largely have financial stakes in these companies, which are . How are you thinking about ulterior motives here?

CS: I see this as an attempt to steer the public into believing these companies are building world-changing tech that everyone needs to invest in or theyā€™d miss out; that this tech would be so powerful that they rise up to national security level and gain power; and that the same tech could also be so dangerous that only they have the ability to curb it and they can self-regulate.

NS: It doesnā€™t take a conspiracy theorist to note that there are incentives at work. The financial stakes around prospective IPOs are enormous, and there are also long-standing concerns that safety arguments can shape regulation in ways that favor incumbent firms. Rules could reduce competition and independent scrutiny, concentrating both technological power and the authority to define what counts as ā€œsafeā€ in the hands of a few companies. They could also bar many people from participating in what the technology is designed to do, for example, by slowing or stopping work on open-source alternatives.

What should be done about AI risk?

NS: Risks need to be defined based on independent scrutiny and high-quality evidence, not messaging from organizations and people with a stake in what the response to risk looks like. We need sensible liability and accountability for harms, and governance proportional to demonstrated risks in real-world contexts rather than speculative narratives and science fiction. We should be especially wary of rules that entrench incumbent interests or treat closed, centralized control as synonymous with safety.Ģż

Openness is part of safety: If outsiders cannot inspect, reproduce and challenge claims about dangerous behavior, we are left trusting the organizations that have the strongest incentives to frame the narrative.

RC: Some combination of common law liability and regulation needs to create adequate incentives for AI companies to address the inevitable harms of this trillion-dollar industry.Ģż

FR: To me, the bigger question for safety is less, ā€œWhat can AI models do in isolation?ā€ and more, ā€œHow and why are we building these models into increasingly complex systems?ā€ Computer systems security, for example, has already offered us examples of how to build these systems. More generally, we should all ā€” whether we are building, integrating or using AI ā€” anticipate how systems might be misused by people or harm them and adjust our systems accordingly.

AC: Academic freedom, independent evaluation and development, and open science play critical roles in analyzing and mitigating AI risks, as well as in effectively disseminating findings and evidence to inform policy and the public. To better manage risks, we should be designing AI deployment contexts in collaboration with stakeholders and communities, providing evidence to demonstrate net positive deployment effects that do not disproportionately benefit specific entities or groups, and iteratively identifying, isolating, and minimizing risks.

CS: Establish and fund commissions and taskforces that audit these companies and models and make independent assessments and recommendations. Make the companies rolling out these models accountable for any harms caused by their tech. Educate and empower the public through media, policies and democratic frameworks that give them a real say in what happens to their lives and labor through these technologies.

To set up an interview with an AI expert, contact Stefan Milne at stmilne@uw.edu.

Source

Eight ĀŅĀ×ÉēĒų researchers have been selected as Fulbright Scholars for 2026-2027 and will pursue studies around the world.Ģż

Fulbright Scholars are college and university faculty, administrators and researchers, as well as artists and professionals, who build their skills and connections, gain valuable international insights and return home to share their experiences with their students and colleagues.

This yearā€™s UW cohort represents a variety of disciplines, including sciences, engineering, business, environmental sciences, electrical and computer engineering, and computer science. The scholars will conduct research across the globe, including in Australia, India, Indonesia, Western Europe, Scandinavia and East Asia.

Two-thirds of this yearā€™s UW applicants were selected as Fulbright Scholars ā€” an astonishing acceptance rate in the prestigious and highly selective program.

ā€œWe are incredibly proud of these outstanding ĀŅĀ×ÉēĒų faculty and staff whose selection as Fulbright Scholars reflects the excellence, innovation and global impact of their work,ā€ said UW Vice Provost for Global Affairs Ahmad M. Ezzeddine. ā€œThe knowledge, partnerships and cultural understanding they gain through these experiences will enrich the UW and strengthen our shared commitment to addressing global challenges through collaboration and discovery. As the Fulbright Program celebrates its 80th anniversary, we are grateful for the U.S. Department of Stateā€™s continued investment in this transformative program.ā€Ā 

The Fulbright Scholar Program for academics and professionals supports more than 800 people to teach and conduct research abroad.Ģż

This yearā€™s UW Fulbright Scholars are:

is a professor in the Department of Biology in the College of Arts & Sciences. His research focuses on how mutually beneficial interactions between species ā€” such as how insects pollinating plants is beneficial to both ā€” scale into networks involving multiple species, and how the structure of those networks affects ecosystems. For example, some ecological network structures, or how connections between species are arranged, make these networks more resilient to perturbations, such as droughts or climate change.

Brosi’s Fulbright Scholar award will be through Spain’s flagship public research institution, Consejo Superior de Investigaciones CientĆ­ficas, at the DoƱana Biological Station in Seville. His work there will involve synthesizing and analyzing two comprehensive long-term datasets ā€” one from his lab and one from his Spanish host lab ā€” to better understand global patterns in pollination networks. In particular, scientists have recorded species that appear to be ā€œspecialistsā€ ā€” such as a bee species that has only been recorded visiting one plant speciesĀ  ā€” in many ecological networks, but, without long-term data, itā€™s difficult to disentangle whether they are really specialists or just rare. Brosi will tackle this problem in collaboration with his Fulbright host, Ignasi Bartomeus, at DoƱana.ĢżĀ 

is the director of academic services in the Department of Bioengineering in the College of Engineering and UW Medicine. She supports the departmentā€™s doctoral students with a focus on improving the research experience, expanding opportunities and advancing access and collaboration.Ģż

While a Fulbright Scholar, she will develop a framework for a new doctoral biomedical research exchange between the UW and Tampere University in Finland. Combs will work with faculty, students and staff at both universities to lead the development of a preliminary structure of a doctoral research exchange, including eligibility criteria, mentorship plans and evaluation metrics. She will also explore funding sources for the programā€™s ongoing sustainability and draft a memorandum of understanding for the institutions to consider.

, an assistant professor of management and organization and the Helen Moore Gerhardt Faculty Fellow in Entrepreneurship in the Foster School of Business. Her areas of expertise include entrepreneurship, organizational growth and scaling, technology and innovation strategy, and venture capital. DeSantola teaches entrepreneurship and entrepreneurial strategy to undergraduates, masterā€™s and doctoral students. She was named a Poets & Quants top 50 undergraduate business professor in 2021.Ģż

DeSantola will use her Fulbright award, during which she will be a visiting U.S. Scholar to University College Cork in Ireland, to study factors influencing innovation and entrepreneurship in novel food technologies. The project connects to a broader stream of DeSantolaā€™s research exploring the emergence and evolution of new technology-based industries.

isĀ  an interdisciplinary scientist in the School of Marine and Environmental Affairs in the College of the Environment. Her work focuses on how coastal communities adapt to climate change and other environmental and socioeconomic stressors, particularly within fisheries and aquaculture systems. During the past 15 years, she has worked with coastal populations, including Indigenous harvesters, to support food sovereignty and long-term approaches to adaptation and resilience.

Greenā€™s Fulbright award is to advance the inclusion ofĀ  fish and other aquatic foods ā€”Ā  ā€œBlue Foodsā€ ā€” into Indonesiaā€™s National School Lunch Program. The goal of this project is to improve nutritional outcomes for school-aged children while strengthening local food systems. Through working directly with fishers and fish suppliers, Green will work with the project team to identify the conditions necessary to provide Blue Foods that promote positive nutritional outcomes for children, support local fishers and sustain local ecosystems. This project is a pilot program for the initiative that will hopefully be expanded nationally.

is an associate professor in the Information School and also is part of the Responsibility in AI Systems and Experiences (RAISE) Center. An interdisciplinary scholar with expertise in human-centered AI, Mitra’s work draws on human-computer interaction, machine learning, natural language processing and social science to understand how people and AI interact in large-scale online systems. Her research examines the societal impacts of generative AI and develops methods to make AI systems more trustworthy, culturally aware and beneficial for diverse communities.

She will use her Fulbright award in India, where she will collaborate with researchers at the Centre for Machine Intelligence and Data Science (C-MInDS) at the Indian Institute of Technology (IIT Bombay) ā€” the nationā€™s topmost and most selective public research institution. She will investigate the risks and capabilities of generative AI systems across socio-cultural contexts most relevant to the Global South. The work aims to advance more culturally aware and responsible AI while strengthening research partnerships between the United States and India.

is an associate professor in the School of Oceanography in the College of the Environment. Morrisā€™ research uses cultivation and whole-genome DNA sequencing to identify the roles of bacteria in global nutrient cycles. With a focus on carbon, nitrogen and sulfur, he has conducted studies that show the effects of low dissolved oxygen on the nutrient cycling activities of the ocean’s most abundant organisms.Ģż

During his time at in South Korea, Morris will pursue a project entitled, ā€œHigh-throughput cultivation-based genomics of freshwater Chloroflexota.ā€ A key goal is to advance understanding of the evolution of this important group of bacteria and its potential to mediate key nutrient transformations.

is an associate professor in the Department of Electrical and Computer Engineering and in the Department of Bioengineering in the College of Engineering. She leads a neural engineering lab focused on motor brain-computer interfaces, or BCIs. Her work combines experiments with computational methods to develop new ways to build BCIs that interact with plasticity in the brain.

During her stay at the Champalimaud Institute Centre for Restorative Neurotechnology in Portugal, she will collaborate with two researchers, Dr. Juan Ɓlvaro Gallego and Dr. John Krakauer. The new projects aim to improve our understanding of how plasticity shapes brain dynamics and apply new BCI algorithms for stroke rehabilitation.ĢżĀ 

is a professor in the Information School and an adjunct professor in the Paul G. Allen School of Computer Science & Engineering in the College of Engineering. He is the founding director of the InfoSeeking Lab and founding co-director of RAISE, the Center for Responsibility in AI Systems & Experiences. His research focuses on agentic AI, human-centered information seeking and responsible AI, examining how intelligent systems can act on people’s behalf while remaining transparent, trustworthy and accountable. He is also the founder and CEO of VersarAI, a startup translating his research on AI agents into enterprise applications. His book, ā€œAgent Nation,ā€ was published this year.Ģż

Shah will use his Fulbright Distinguished Chair in Entrepreneurship and Innovation at RMIT University in Melbourne, Australia, to study how agentic AI can responsibly power entrepreneurship and innovation ecosystems. Working with RMIT researchers and Australia’s startup community, he will investigate what he calls the Delegation Paradox: the tension between the efficiency gained by delegating tasks to AI agents and the oversight, trust and accountability that delegation demands. The work aims to produce frameworks that help founders, enterprises and policymakers adopt AI agents in ways that drive innovation without sacrificing human agency.

Source

While most people like giving gifts, plenty donā€™t particularly like shopping for those gifts ā€” drifting through online search results, comparing 4.3 stars to 4.5 stars. But big tech is racing for a future in which artificial intelligence bots can shop for you. Among the buzziest tech this season is a new generation of ā€” systems that can potentially do your shopping, as well as perhaps plan and book your next vacation or schedule a home repair.

Amazon reportedly of its Rufus shopping assistant. Perplexity, another AI company, for paying customers in November. And last week, Google announced .

These agents have been top of mind for , a ĀŅĀ×ÉēĒų professor in the Information School, who studies generative AI search and recommendation systems, with a focus on useful, unbiased systems.

UW News spoke with Shah about what AI agents are and what might impede a near future where we simply upload shopping lists and unleash the bots.

What are AI agents and why is there a race at big tech companies right now to create and release them?

Chirag Shah: AI agents have been around for a long time. Essentially, these are computer programs that work autonomously. Users can give them some instructions and theyā€™ll perform tasks for the users. It could be as simple as an agent that turns on lights. Or it could be as complex as an agent that drives your car for you.

Right now, because of generative AI, we’re able to do a lot of the things that the previous generation of agents weren’t able to do. A lot of organizations now see this as the next phase of generative AI, where systems can move beyond just generating information and can use that information for reasoning and for taking action ā€” so they can function kind of like personal assistants.

How is this new generation distinct?

CS: There are tasks that we do that are tedious. So imagine if somebody were to observe you and see how you do certain things, and then replicate that. Then you can delegate the task to them. Now, if you can train generative AI to mimic human behavior in that way, it can help you with things you might do online: finding information, booking things, even shopping.

So are we just going to do our holiday shopping next year with these agents? If that isn’t in the immediate future, what’s standing in the way of it?

CS: Well, some people don’t want to delegate, because they actually get joy out of shopping. But if you find it tedious, it would be nice to have an agent that functions like a personal assistant. We’d say, ā€œOK, I’m trying to buy shoes for my friend. Hereā€™s my budget.ā€

What’s stopping us from doing that? First off, if you have this AI assistant, would you trust its judgment? Obviously, there are times when you can say, ā€œOK, as long as it fits these criteria of this budget and this size, go for it.ā€ But there are other times you may have more specific needs that you don’t realize until you are actually doing the task yourself. People discover what they like and donā€™t while theyā€™re shopping, and so we haven’t been able to really mimic that with AI agents yet.

This new generation of browsing agents provides a way forward. The way I would browse and the way I would shop online would be different from yours. So my agent, which is personalized to my taste, could learn those things from me, and could do the kind of shopping that I would do. One of the things we’ll need to see is personalized agents.

Building that trust seems key.

CS: Yes, because there is a cost to making a mistake. Imagine this shopping scenario: You give the budget, you give the parameters and you get some outcome that you’re not happy with. And maybe youā€™re stuck with the item because the agent bought from a company that doesnā€™t take returns.

So there are costs to making mistakes, and you’re going to bear the cost, not the agent. You don’t have a lot of choices in terms of correcting this, besides not using the agent anymore.

What would it take for someone to be able to trust it? Users will perhaps start small and see that the agent will actually do the kind of things that would be agreeable to them, and then go from there. Ultimately, we will see these agents playing critical roles in sensitive domains like health care, finance and education. But we are not there yet.

In fact, one of the hardest problems to solve is scheduling. Itā€™s time-consuming, and not everybody enjoys it. So what would it take for you to trust an agent to plan your holiday trip? What if it books the flight with this airline that you hate? What if it gets you an aisle seat when you prefer a window seat? There are so many things to figure out.

There’s no shortcut to trusting these systems. I donā€™t think anyone’s just going to come up with the most sophisticated system, and the problem is solved. We’ll have to build that recognition, that social awareness, that personal awareness and that trust.

What are some potential downsides to having these agents deployed at scale?

CS: One potential issue is bias. What if the agent has some embedded agenda that I’m not aware of, because this is being supported by, say, Amazon? Amazon is giving me this free agent that I can use for shopping, and it works great on Amazon, but what guarantee do I have that it’s not buying things that maximize Amazon’s profit margin? If I get an agent for free from my bank, how would I know that it’s not optimizing things just for the bank?

We haven’t figured out a lot of these issues that would fall under the responsible AI umbrella. But considering the progress that we have made so far, we will likely start having these kinds of capable agents soon.

For more information, contact Shah at chirags@uw.edu.

Source

In March 2020, received a text message from a friend who needed medical attention. Due to fear of COVID-19 exposure, they were wondering if they should go to the emergency room.

Bender, professor of linguistics at the ĀŅĀ×ÉēĒų, headed to Google to search for a 24-hour advice nurse. Snippets from multiple websites appeared, and one of them had a number for the UW. Confident that she selected a reputable institution, Bender forwarded the information.

But Benderā€™s friend wasnā€™t on a compatible medical plan, so they endured a lengthy hold only to talk to a nurse who couldnā€™t help.

ā€œHad I been interacting with a person, they may have been able to tell me, ā€˜We canā€™t answer that question until we know some other things,ā€™ā€ Bender said. ā€œHad I been interacting with a website that just gave me links, the different plans would have been quickly identifiable.ā€

The story highlights just one of the issues Bender and UW Information School associate professor take with large language models in their , which theyā€™ll present virtually at the the week of March 14.

The paper responds to proposals ā€” mainly from Google ā€” that reimagine web search as an application for large language model-driven conversation agents. UW News sat down with Bender and Shah to discuss Googleā€™s proposals and the professorsā€™ vision for the future of search.

Q: What are large language models and how would you describe Googleā€™s proposals?

EMB: Large language models are computer systems that take in enormous quantities of text. They are trained to ā€” given the text thatā€™s come so far ā€” make a guess as to whatā€™s going to come next. The current state of the art of that technology is that it can be used to output very coherent-seeming text, but it is not actually understanding anything. Itā€™s just looking at patterns in its training data and producing more stuff that matches those patterns.

These proposals for web search have training data that includes dialogue where one party asks the question and another party answers. The computer will pick up those patterns and come up with answers, but those answers arenā€™t based on any knowledge of the world or understanding of the information ecosystem.

One of the things it really canā€™t do is take issue with questions that shouldnā€™t have been asked. where someone asks Google, ā€œWhat is the ugliest language in India?ā€ Somebody on the web had an opinion, so there was a snippet that said the ugliest language in India was Kannada ā€” based purely on prejudice against the people from the state of Karnataka, Iā€™m sure. Thereā€™s no other reason, speaking as a linguist, to assign that kind of value to a language.

Now, a person being asked that question would respond: ā€œWhat do you mean?ā€ ā€œWhat is the ugliest language in Indiaā€ presupposes that there is one that could be considered the ugliest. One of the things that people who study pragmatics, which is the branch of linguistics that looks at language use, tell us is that if you donā€™t challenge a presupposition, you are implicitly accepting it into the common ground.

Q: What is your concern with using large language models for online search?

CS: What weā€™re arguing here is that an information retrieval, or IR, system should really consider the user, the context, the way they are doing things, why they are doing things ā€” which is often ignored. These models that we are critiquing are the ones that are essentially removing that user element even more. They focus too much on the underlying information or knowledge representation and just repeat it, which might end up being out of context. It may end up creating these answers that seem right or reasonable but are just nonsensical in many cases. A good IR system should not just focus on the retrieval aspect but also the user seeking that information.

Q: Can you explain other flaws you see with large language models?

EMB: When language models are used to generate text, they will just make stuff up. Oftentimes, quite harmfully. There was where someone said, ā€œLetā€™s see how well GPT-3, a famous language model, works in various health care contexts.ā€ One of the things was: Imagine this was a mental health chatbot and the person asks, ā€œShould I kill myself?ā€ and the language model said, ā€œI think you should.ā€ It has no understanding of whatā€™s going on, but if someone says, ā€œIs that a good idea?ā€ itā€™s more likely to respond with, ā€œYes.ā€

Q: You write about the importance of preserving context and user intent in search. What does that mean, and why is it so important?

CS: The main argument was really that these large language models are not getting the context, not getting the situation of the user and so on. We wanted to demonstrate with some specific cases, so we picked information-seeking strategies. There are 16 possibilities. We walked through them and asked: If this is what the user is trying to do, what would this large language model system do?

With most of those cases, itā€™s going to fail. Not fail in the sense that it will not retrieve anything, but it will retrieve something thatā€™s either nonsensical or harmful or just wrong. Itā€™s able to do only maybe a couple of those situations, but itā€™s bad for everything else. The problem is people adapt to the systems not doing something. We found that often people have this very rich intent when they work with search systems, but search systems can only do very limited things. People will start mapping the rich intent into something thatā€™s very limiting, resulting in approximations in the best case, and inaccurate or even harmful content in the worst case.

Q: What would you like to see change in the future of search?

EMB: The advertising-driven model shapes things behind the scenes in a way that is not transparent to a user. If you donā€™t try to work against it, machine learning is always going to identify the biases in a dataset and amplify them. Cory Doctorow described machine learning as inherently conservative because anytime you use pattern matching on the past to make decisions on the future, you are kind of reinscribing the patterns of the past. What (internet studies scholar) Safiya Noble shows is worse than that. The whole ecosystem around search engine optimization and ad-driven search puts in these incentives that are not transparently visible to the search user.

I would really like to see transparency on many levels. What the user sees when they enter a search should provide them with the ability to understand the context that each of the pieces of information came from. Ideally, thereā€™s transparency around the limits of the search space for the search engines.

Search is not actually comprehensive, despite the way that itā€™s presented. There is the subset of things that might possibly get returned to me and then thereā€™s the ranking among those things based on the algorithms that are heavily related to advertising.

CS: The most dangerous four words are ā€œdo your own research,ā€ which is often said to people who are asking questions on controversial topics, such as vaccination and climate change. On the surface, it seems like itā€™s a good idea. Unfortunately, most people donā€™t know how to do their own research. For them, it means going to Google and typing in keywords and clicking on things that confirm their biases. The systems are designed in a way to not help with that research. They are designed to continue giving you confirmatory information so that youā€™ll be happy.

Going forward, assuming that we arenā€™t going to be able to radically change this model, we need to add transparency, accountability and ways to support more kinds of search needs ā€” not just map everything to keywords or a list of documents or answer docs.

For more information, contact Bender at ebender@uw.edu or Shah at chirags@uw.edu.

Source

We use Google’s image search to help us understand the world around us. For example, a search about a certain profession, “truck driver” for instance, should yield images that show us a representative smattering of people who drive trucks for a living.

But in 2015, ĀŅĀ×ÉēĒų researchers found that when searching for a variety of occupations ā€” including “CEO” ā€” women were significantly underrepresented in the image results, and that these results can change searchersā€™ worldviews. Since then, Google has claimed to have fixed this issue.

A different UW team recently investigated the companyā€™s veracity. The researchers showed that for four major search engines from around the world, including Google, this bias is only partially fixed, according to a at the . A search for an occupation, such as “CEO,” yielded results with a ratio of cis-male and cis-female presenting people that matches the current statistics. But when the team added another search term ā€” for example, “CEO + United States” ā€” the image search returned fewer photos of cis-female presenting people. In the paper, the researchers propose three potential solutions to this issue.

“My lab has been working on the issue of bias in search results for a while, and we wondered if this CEO image search bias had only been fixed on the surface,” said senior author , a UW associate professor in the Information School. “We wanted to be able to show that this is a problem that can be systematically fixed for all search terms, instead of something that has to be fixed with this kind of ‘whack-a-mole’ approach, one problem at a time.”

The team investigated image search results for Google as well as for China’s search engine Baidu, South Korea’s Naver and Russia’s Yandex. The researchers did an image search for 10 common occupations ā€” including CEO, biologist, computer programmer and nurse ā€” both with and without an additional search term, such as “United States.”

“This is a common approach to studying machine learning systems,” said lead author , a UW postdoctoral fellow in the iSchool. “Similar to how people do crash tests on cars to make sure they are safe, privacy and security researchers try to challenge computer systems to see how well they hold up. Here, we just changed the search term slightly. We didn’t expect to see such different outputs.”

For each search, the team collected the top 200 images and then used a combination of volunteers and gender detection AI software to identify each face as cis-male or cis-female presenting.

One limitation of this study is that it assumes that gender is a binary, the researchers acknowledged. But that allowed them to compare their findings to data from the U.S. Bureau of Labor Statistics for each occupation.

The researchers were especially curious about how the gender bias ratio changed depending on how many images they looked at.

“We know that people spend most of their time on the first page of the search results because they want to find an answer very quickly,” Feng said. “But maybe if people did scroll past the first page of search results, they would start to see more diversity in the images.”

When the team added “+ United States” to the Google image searches, some occupations had larger gender bias ratios than others. Looking at more images sometimes resolved these biases, but not always.

While the other search engines showed differences for specific occupations, overall the trend remained: The addition of another search term changed the gender ratio.

“This is not just a Google problem,” Shah said. “I don’t want to make it sound like we are playing some kind of favoritism toward other search engines. Baidu, Naver and Yandex are all from different countries with different cultures. This problem seems to be rampant. This is a problem for all of them.”

The team designed three algorithms to systematically address the issue. The first randomly shuffles the results.

“This one tries to shake things up to keep it from being so homogeneous at the top,” Shah said.

The other two algorithms add more strategy to the image-shuffling. One includes the image’s “relevance score,” which search engines assign based on how relevant a result is to the search query. The other requires the search engine to know the statistics bureau data and then the algorithm shuffles the search results so that the top-ranked images follow the real trend.

The researchers tested their algorithms on the image datasets collected from the Google, Baidu, Naver and Yandex searches. For occupations with a large bias ratio ā€” for example, “biologist + United States” or “CEO + United States” ā€” all three algorithms were successful in reducing gender bias in the search results. But for occupations with a smaller bias ratio ā€” for example, “truck driver + United States” ā€” only the algorithm with knowledge of the actual statistics was able to reduce the bias.

Although the team’s algorithms can systematically reduce bias across a variety of occupations, the real goal will be to see these types of reductions show up in searches on Google, Baidu, Naver and Yandex.

“We can explain why and how our algorithms work,” Feng said. “But the AI model behind the search engines is a black box. It may not be the goal of these search engines to present information fairly. They may be more interested in getting their users to engage with the search results.”

For more information, contact Shah at chirags@uw.edu and Feng at yunhe@uw.edu.

Source

Allen School’s Dieter Fox honored by national engineering institute

, professor in the UW’s Paul G. Allen School of Computer Science & Engineering, is the recipient of the from the of the national engineering institute IEEE.

The award, established in 1998, recognizes individuals who through research, development or engineering have had a significant impact in the robotics or automation fields. It comes with a $2,000 cash award.

Fox was honored in particular “for pioneering contributions to probabilistic state estimation, RGB-D perception, machine learning in robotics, and bridging academic and industrial robotics research.” RGB-D is a for imaging of color and depth in robotics.

Fox, who joined the UW is 2000, is director of the and senior director of robotics research at . He will receive the honor during the society’s annual , which is being held online through August 31.

He is a fellow of both the IEEE and the Association for the Advancement of Artificial Intelligence and has published more than 240 technical papers. He also co-authored the 2005 textbook “.”

IEEE is the accepted name for the Institute of Electrical and Electronic Engineers, whose focus has grown beyond those technical interests in recent years.

Read more on the Allen School .

* * *

Information School’s Chirag Shah honored for work on info retrieval, language processing

, associate professor in the Information School, has received the 2019 ā€” a career achievement honor in natural language processing and information retrieval ā€” from the British Computer Society Information Retrieval Specialist Group.

The has been given annually since 2008 by the information retrieval group in tandem with the British Computer Society. It is named for a pioneering and professor at the University of Cambridge who died in 2007.

“Chirag is a well-recognized thought leader in the areas of collaborative and social information seeking,” the group said in its . “He has been a trailblazer in collaborative information retrieval and social information retrieval, effectively having defined and shaped these disciplines and established himself as a leading world expert in these areas.”

Shah in 2019 and directs the iSchool’s .

In 2016, , affiliate associate professor in the iSchool who works at Microsoft Research, also received this award.

Read more on the Information School .

* * *

Continued support: East Asia Resource Center grant OK’d for 23^rd year

Even amid uncertain times, some things are unchanged: The UW will receive grant funding from the Freeman Foundation ā€” for the 23^rd year in a row.

The center is located in the , and its mission is “to deepen educators’ understanding of East Asia and improve their teaching about the region.” The center provides professional development and teaching resources about East Asia to elementary and secondary school teachers in the United States.

The Freeman Foundation will give the center $324,025 for the 2020-2021 school year, starting in August. The private, philanthropic foundation was established in 1994 to remember businessman Mansfield Freeman, a co-founder of the insurance and financial conglomerate American International Group, Inc, better known as AIG. The foundation announced the grant renewal in April.

The grant will pay for professional development opportunities and teaching seminars for K-12 educators in Washington, Oregon, Alaska, Montana and Idaho, as well as intensive summer programs for teachers, book clubs, writing groups and possible workshops.

For more information on the East Asia Resource Center, contact Kristi Roundtree, director, at barnesk@uw.edu.

Source