Research Horizons

Georgia Tech Team Takes Second Place at ICRA Robot Teleoperation Contest

dgivens8 — Tue, 29 Jul 2025 12:58:25 +0000

Georgia Tech Team Takes Second Place at ICRA Robot Teleoperation Contest dgivens8 Tue, 07/29/2025 - 08:58

An algorithmic breakthrough from School of Interactive Computing researchers that earned a Meta partnershipdrew more attention at the IEEE International Conference on Robotics and Automation (ICRA).

Meta announced in February its partnership with the labs of professors Danfei Xu and Judy Hoffman on a novel computer vision-based algorithm called EgoMimic. It enables robots to learn new skills by imitating human tasks from first-person video footage captured by Meta’s Aria smart glasses.

Xu’s Robot Learning and Reasoning Lab (RL2) displayed EgoMimic in action at ICRA May 19-23 at the World Congress Center in Atlanta.

Lawrence Zhu, Pranav Kuppili, and Patcharapong “Elmo” Aphiwetsa — students from Xu’s lab — used Egomimic to compete in a robot teleoperation contest at ICRA. The team finished second in the event titled What Bimanual Teleoperation and Learning from Demonstration Can Do Today, earning a $10,000 cash prize.

Teams were challenged to perform tasks by remotely controlling a robot gripper. The robot had to fold a tablecloth, open a vacuum-sealed container, place an object into the container, and then reseal it in succession without any errors.

Teams completed the tasks as many times as possible in 30 minutes, earning points for each successful attempt.

The competition also offered different challenge levels that increased the points awarded. Teams could directly operate the robot with a full workstation view and receive one point for each task completion. Or, as the RL2 team chose, teams could opt for the second challenge level.

The second level required an operator to control the task with no view of the workstation except for what was provided to through a video feed. The RL2 team completed the task seven times and received double points for the challenge level.

The third challenge level required teams to operate remotely from another location. At this level, teams could earn four times the number of points for each successful task completed. The fourth level challenged teams to deploy an algorithm for task performance and awarded eight points for each completion.

Using two of Meta’s Quest wireless controllers, Zhu controlled the robot under the direction of Aphiwetsa, while Kuppili monitored the coding from his laptop.

“It’s physically difficult to teleoperate for half an hour,” Zhu said. “My hands were shaking from holding the controllers in the air for that long.”

Being in constant communication with Aphiwetsa helped him stay focused throughout the contest.

“I helped him strategize the teleoperation and noticed he could skip some of the steps in the folding,” Aphiwetsa said. “There were many ways to do it, so I just told him what he could fix and how to do it faster.”

Zhu said he and his team had intended to tackle the fourth challenge level with the EgoMimic algorithm. However, due to unexpected time constraints, they decided to switch to the second level the day before the competition due to unexpected time constraints.

“I think we realized the day before the competition training the robot on our model would take a huge amount of time,” Zhu said. “We decided to go for the teleoperation and started practicing.”

He said the team wants to tackle the highest challenge level and use a training model for next year’s ICRA competition in Vienna, Austria.

ICRA is the world’s largest robotics conference, and Atlanta hosted the event for the third time in its history, drawing a record-breaking attendance of over 7,000.

Subtitle

Summary sentence

A Georgia Tech team earned second place in the ICRA Robot Teleoperation Contest for their EgoMimic algorithm, which allows robots to learn skills by mimicking human tasks from first-person video.

Summary

Students from Georgia Tech's Robot Learning and Reasoning Lab earned second place and a $10,000 cash prize in a robot teleoperation contest at the 2025 International Conference on Robotics and Automation in Atlanta. The RL2 lab announced a partnership with Meta in February on a novel computer vision-based algorithm called EgoMimic. It enables robots to learn new skills by imitating human tasks from first-person video footage captured by Meta’s Aria smart glasses.

Dateline

Wed, 06/11/2025 - 12:00

Associated importer

Keywords

cc-research; ic-ai-ml; ic-robotics

artificial intelligence (AI)

go-ai

go-researchnews

Research Horizons

student competition

Categories

Computer Science/Information Technology and Security

Robotics

Student Competition WInners (academic, innovation, and research)

Mercury ID

682761

Source updated

Thu, 06/12/2025 - 07:52

Middle Schoolers’ Feedback Informs New Approach to AI-based Museum Exhibits

dgivens8 — Fri, 30 May 2025 18:14:41 +0000

Middle Schoolers’ Feedback Informs New Approach to AI-based Museum Exhibits dgivens8 Fri, 05/30/2025 - 14:14

Researchers at Georgia Tech are creating accessible museum exhibits that explain artificial intelligence (AI) to middle school students, including the LuminAI interactive AI-based dance partner developed by Regents' Professor Brian Magerko.

Ph.D. students Yasmine Belghith and Atefeh Mahdavi co-led a study in a museum setting that observed how middle schoolers interact with the popular AI chatbot ChatGPT.

“It’s important for museums, especially science museums, to start incorporating these kinds of exhibits about AI and about using AI so the general population can have that avenue to interact with it and transfer that knowledge to everyday tools,” Belghith said.

Belghith and Mahdavi conducted their study with nine focus groups of 24 students at Chicago’s Museum of Science and Industry. The team used the findings to inform their design of AI exhibits that the museum could display as early as 2025.

Belghith is a Ph.D. student in human-centered computing. Her advisor is Assistant Professor Jessica Roberts in the School of Interactive Computing. Magerko advises Mahdavi, a Ph.D. student in digital media in the School of Literature, Media, and Communication.

Belghith and Mahdavi presented a paper about their study in May at the Association for Computing Machinery (ACM) 2024 Conference on Human Factors in Computing Systems (CHI) in Honolulu, Hawaii.

Their work is part of a National Science Foundation (NSF) grant dedicated to fostering AI literacy among middle schoolers in informal environments.

Expanding Accessibility

While there are existing efforts to reach students in the classroom, the researchers believe AI education is most accessible in informal learning environments like museums.

“There’s a need today for everybody to have some sort of AI literacy,” Belghith said. “Many middle schoolers will not be taking computer science courses or pursuing computer science careers, so there needs to be interventions to teach them what they should know about AI.”

The researchers found that most of the middle schoolers interacted with ChatGPT to either test its knowledge by prompting it to answer questions or socialize with it by having human-like conversations.

Others fit the mold of “content explorers.” They did not engage with the AI aspect of ChatGPT and focused more on the content it produced.

Mahdavi said regardless of their approach, students would get “tunnel vision” in their interactions instead of exploring more of the AI’s capabilities.

“If they go in a certain direction, they will continue to explore that,” Mahdavi said. “One thing we can learn from this is to nudge kids and show them there are other things you can do with AI tools or get them to think about it another way.”

The researchers also paid attention to what was missing in the students’ responses, which Mahdavi said was just as important as what they did talk about.

“None of them mentioned anything about ethics or what could be problematic about AI,” she said. “That told us there’s something they aren’t thinking about but should be. We take that into account as we think about future exhibits.”

Making an Impact

The researchers visited the Museum of Science and Industry June 1-2 to conduct the first trial run of three AI-based exhibits they’ve created. One of them is LuminAI, which was developed in Magerko’s Expressive Machinery Lab.

LuminAI is an interactive art installation that allows people to engage in collaborative movement with an AI dance partner. Georgia Tech and Kennesaw State recently held the first performance of AI avatars dancing with human partners in front of a live audience.

Duri Long, a former Georgia Tech Ph.D. student who is now an assistant professor at Northwestern University, designed the second exhibit. KnowledgeNet is an interactive tabletop exhibit in which visitors build semantic networks by adding different characteristics to characters that interact together.

The third exhibit, Data Bites, prompts users to build datasets of pizzas and sandwiches. Their selections train a machine-learning classifier in real time.

Belghith said the exhibits fostered conversations about AI between parents and children.

“The exhibit prototypes successfully engaged children in creative activities,” she said. “Many parents had to pull their kids away to continue their museum tour because the kids wanted more time to try different creations or dance moves.”

Summary sentence

Partnering with Chicago's Museum of Science and Industry, Researchers at Georgia Tech are creating accessible museum exhibits that explain artificial intelligence (AI) to middle school students.

Summary

Ph.D. students Yasmine Belghith and Atefeh Mahdavi co-led a study in a museum setting that observed how middle schoolers interact with the popular AI chatbot ChatGPT.

Dateline

Fri, 06/21/2024 - 12:00

Contact

Nathan Deen

Communications Officer I

School of Interactive Computing

Associated importer

Keywords

go-ai

artificial intelligence

middle school

AI education

go-researchnews

Research Horizons

Categories

Community

Education

Exhibitions

Georgia Tech Arts

Music and Music Technology

Mercury ID

675196

Source updated

Wed, 07/17/2024 - 10:05

Georgia Tech to Strengthen Nation’s Faculty Development in Geospace Science

admin — Tue, 20 May 2025 16:30:33 +0000

Georgia Tech to Strengthen Nation’s Faculty Development in Geospace Science admin Tue, 05/20/2025 - 12:30

Georgia Tech’s Colleges of Engineering and Sciences have been chosen by the National Science Foundation (NSF) to hire a new faculty member focused on solar-terrestrial science and space weather research. The NSF is prioritizing a national need in geospace physics and selected Georgia Tech from a pool of national universities.

“Space weather has many societal implications, including dangers to the power grid, the aviation sector, satellite lifetimes, communications, and navigation,” said Morris Cohen, professor in the School of Electrical and Computer Engineering (ECE) and the grant’s co-principal investigator. “However, the number of qualified graduating students interested in this area is not sufficient to meet the future demand. This is especially true as the generation of professionals trained during the space race of the 1960s and ‘70s continues to retire.”

NSF will fund the position for five years and $1.5 million. The grant is led by Susan Lozier, dean of the College of Sciences and Betsy Middleton and John Clark Sutherland Chair. She and Cohen are joined by Raheem Beyah, dean of the College of Engineering and Southern Company Chair, and Glenn Lightsey, the John W. Young Chair in the Guggenheim School of Aerospace Engineering (AE).

The two Colleges relied heavily on their strength in space research and Georgia Tech’s culture of multidisciplinary collaborations in the NSF application. These traits will allow Georgia Tech to conduct a unique search process. Instead of one unit making the hire as is typical in higher education, leaders from four schools will team up with the Georgia Tech Research Institute (GTRI) for the search process. It’s an approach that addresses a nationwide problem in the field.

“Decades ago, space physics largely fell within electrical engineering,” Beyah said. “These days, it’s highly interdisciplinary and typically has no true home — faculty are often scattered across aerospace engineering, applied physics, and earth sciences.”

Beyah said that a few universities have a large cluster of space physics faculty as a result. Many others have none. He said this limits the pipeline of future space science professionals because a substantial fraction of students has little or no exposure to the field.

Georgia Tech is right in the middle, with a presence in solar-terrestrial science and space weather research but not a large cluster of faculty members. The new hire will allow Tech to reach more students interested in the field. Georgia Tech also pointed to its Vertically Integrated Projects program as a mechanism to get many new students involved in the new hire’s research.

According to Lozier, solar-terrestrial science and space weather encompass at least four buckets: advanced theory and simulations that span the extremes of physics; big data and machine learning; innovative tools to collect new types of measurements; and operational needs in industry and defense, which motivate translation of research into real-world practice.

“This breadth has hampered faculty growth in this area, as it has other interdisciplinary research fields like quantum computing and neuroscience,” Lozier said. “These areas straddle pure science and engineering, which often are separate in university hierarchy. We believe these interdisciplinary aspects of geospace science should be celebrated. More importantly, we believe they can be turned into a strength.”

Representatives from AE, ECE, GTRI, the School of Earth and Atmospheric Sciences, and the School of Physics will form the hiring committee. The hire will complement Georgia Tech’s February announcement of a new Space Research Initiative. Once the NSF-funded position is filled, the Colleges will collectively fund and search for a second faculty member in the field.

Subtitle

With NSF support, Colleges of Sciences and Engineering will collaborate to hire a researcher focused on solar-terrestrial science and space weather.

Summary sentence

With NSF support, Colleges of Sciences and Engineering will collaborate to hire a researcher focused on solar-terrestrial science and space weather.

Summary

With NSF support, Colleges of Sciences and Engineering will collaborate to hire a researcher focused on solar-terrestrial science and space weather.

Dateline

Wed, 05/22/2024 - 12:00

Contact

Jason Maderer
Director of Communications
College of Engineering

Associated importer

Keywords

College of Sciences

college of engineering

Geospace Science

space weather

go-researchnews

Research Horizons

Categories

Engineering

Physics and Physical Sciences

Research

Student and Faculty

Mercury ID

674860

Source updated

Fri, 05/24/2024 - 15:24

Computing Framework Could Reveal Signs of Neuro Disorders Hidden within Brain Data

admin — Mon, 05 May 2025 13:58:41 +0000

Computing Framework Could Reveal Signs of Neuro Disorders Hidden within Brain Data admin Mon, 05/05/2025 - 09:58

A Georgia Tech doctoral student’s dissertation could help physicians diagnose neuropsychiatric disorders, including schizophrenia, autism, and Alzheimer’s disease. The new approach leverages data science and algorithms instead of relying on traditional methods like cognitive tests and image scans.

Ph.D. candidate Md Abdur Rahaman’s dissertation studies brain data to understand how changes in brain activity shape behavior.

Computational tools Rahaman developed for his dissertation look for informative patterns between the brain and behavior. Successful tests of his algorithms show promise to help doctors diagnose mental health disorders and design individualized treatment plans for patients.

“I've always been fascinated by the human brain and how it defines who we are,” Rahaman said.

“The fact that so many people silently suffer from neuropsychiatric disorders, while our understanding of the brain remains limited, inspired me to develop tools that bring greater clarity to this complexity and offer hope through more compassionate, data-driven care.”

Rahaman’s dissertation introduces a framework focusing on granular factoring. This computing technique stratifies brain data into smaller, localized subgroups, making it easier for computers and researchers to study data and find meaningful patterns.

Granular factoring overcomes the challenges of size and heterogeneity in neurological data science. Brain data is obtained from neuroimaging, genomics, behavioral datasets, and other sources. The large size of each source makes it a challenge to study them individually, let alone analyze them simultaneously, to find hidden inferences.

Rahaman’s research allows researchers and physicians to move past one-size-fits-all approaches. Instead of manually reviewing tests and scans, algorithms look for patterns and biomarkers in the subgroups that otherwise go undetected, especially ones that indicate neuropsychiatric disorders.

“My dissertation advances the frontiers of computational neuroscience by introducing scalable and interpretable models that navigate brain heterogeneity to reveal how neural dynamics shape behavior,” Rahaman said.

“By uncovering subgroup-specific patterns, this work opens new directions for understanding brain function and enables more precise, personalized approaches to mental health care.”

Rahaman defended his dissertation on April 14, the final step in completing his Ph.D. in computational science and engineering. He will graduate on May 1 at Georgia Tech’s Ph.D. Commencement.

After walking across the stage at McCamish Pavilion, Rahaman’s next step in his career is to go to Amazon, where he will work in the generative artificial intelligence (AI) field.

Graduating from Georgia Tech is the summit of an educational trek spanning over a decade. Rahaman hails from Bangladesh where he graduated from Chittagong University of Engineering and Technology in 2013. He attained his master’s from the University of New Mexico in 2019 before starting at Georgia Tech.

“Munna is an amazingly creative researcher,” said Vince Calhoun, Rahman’s advisor. Calhoun is the founding director of the Translational Research in Neuroimaging and Data Science Center (TReNDS).

TReNDS is a tri-institutional center spanning Georgia Tech, Georgia State University, and Emory University that develops analytic approaches and neuroinformatic tools. The center aims to translate the approaches into biomarkers that address areas of brain health and disease.

“His work is moving the needle in our ability to leverage multiple sources of complex biological data to improve understanding of neuropsychiatric disorders that have a huge impact on an individual’s livelihood,” said Calhoun.

Summary sentence

A Georgia Tech doctoral student’s dissertation could help physicians diagnose neuropsychiatric disorders, including schizophrenia, autism, and Alzheimer’s disease.

Summary

Ph.D. candidate Md Abdur Rahaman’s dissertation studies brain data to understand how changes in brain activity shape behavior.

Dateline

Thu, 04/24/2025 - 12:00

Contact

Bryant Wine, Communications Officer
bryant.wine@cc.gatech.edu

Keywords

College of Computing

School of Computational Science and Engineering

Research Horizons

go-researchnews

artificial intelligence (AI)

go-ai

Daily Digest

Georgia Tech News Center

News room topics

Health and Medicine

Categories

Biotechnology, Health, Bioengineering, Genetics

Computer Science/Information Technology and Security

Life Sciences and Biology

Research

Student Research

Mercury ID

682026

Source updated

Mon, 05/05/2025 - 09:58

Faculty Earn Fellowships for Heart Modeling and Data Optimization Research

admin — Fri, 25 Apr 2025 14:42:41 +0000

Faculty Earn Fellowships for Heart Modeling and Data Optimization Research admin Fri, 04/25/2025 - 10:42

Two faculty members represented Georgia Tech as new fellows to the world’s leading organization dedicated to applied mathematics, computational science, and data science.

The Society for Industrial and Applied Mathematics (SIAM) selected Elizabeth Cherry and Katya Scheinberg as Class of 2025 fellows. The two Georgia Tech faculty join an illustrious class of 23 other researchers from around the globe in this year’s class.

SIAM selected Cherry to recognize her contributions to mathematical and computational modeling and extensive service to the SIAM community. She studies the electrical behavior of cardiac cells and tissue.

Cherry’s computer models and simulations improve understanding of cardiac dynamics in normal and diseased states. Using these tools, she designs advanced strategies for preventing and treating arrhythmias.

“SIAM has played a huge role in my professional development—the first conference I attended as a graduate student was a SIAM conference, and I’ve attended at least one SIAM conference almost every year since then,” Cherry said.

“Given this long history, it means a lot to me for SIAM to acknowledge my contributions in this way.”

Scheinberg, from Georgia Tech’s College of Engineering, was selected for her foundational contributions to derivative-free optimization and optimization applications in data science and her dedicated service to the optimization community.

Cherry is the fifth faculty member from the School of Computational Science and Engineering (CSE) to be selected as a SIAM Fellow.

Cherry’s announcement as a SIAM Fellow comes weeks after serving in a leadership role at a SIAM conference. She co-chaired the organizing committee of the SIAM Conference on Computational Science and Engineering (CSE25).

In 2023, SIAM members reelected Cherry to a second term as a council member-at-large. She began her three-year term in January 2024.

"SIAM Fellows are selected for deep mathematical contributions. Receiving Fellow status is a high honor for any applied mathematician," said Regents’ Professor Srinivas Aluru, senior associate dean of the College of Computing and Class of 2020 SIAM Fellow.

"Not only are Elizabeth's contributions technically outstanding, but her work also provides deep insights into the functioning of the heart and its abnormalities."

Cherry’s leadership and service extends outside of SIAM, influencing students and faculty across Georgia Tech.

In December, the College of Computing appointed Cherry as associate dean for graduate education. Before this appointment, she served as associate chair for academic affairs of the School of CSE.

With her new role as associate dean, Cherry continues serving as director of CSE programs at Georgia Tech.

In March 2024, Cherry was among five Georgia Tech faculty members selected for the ACC Academic Leaders Network (ACC ALN) Fellows program. The ALN program fosters cross-institutional networking and collaboration between ACC schools, increasing each institution’s academic leadership capacity.

Cherry was part of a team of Georgia Tech and Emory University researchers who won a Georgia Clinical and Translational Science Alliance award in 2023. The group earned the Team Science Award of Distinction for Early Stage Research Teams award for work that captures high-resolution visualizations of spiral waves that create heart arrhythmias.

SIAM will recognize Cherry, Scheinberg, and Class of 2025 fellows during a reception at the SIAM/CAIMS Annual Meetings this July in Montréal.

“It is such an honor to be recognized as a SIAM Fellow,” Cherry said. “I’m thrilled to join my CSE colleagues who have also received this recognition.”

Subtitle

Summary sentence

The Society for Industrial and Applied Mathematics (SIAM) selected Elizabeth Cherry and Katya Scheinberg as Class of 2025 fellows.

Summary

Two faculty members represented Georgia Tech as new fellows to the world’s leading organization dedicated to applied mathematics, computational science, and data science.

Dateline

Fri, 04/11/2025 - 12:00

Contact

Bryant Wine, Communications Officer
bryant.wine@cc.gatech.edu

Associated importer

Keywords

College of Computing

School of Computational Science and Engineering

go-researchnews

Research Horizons

Daily Digest

Georgia Tech News Center

SIAM

News room topics

Science and Technology

Categories

Biotechnology, Health, Bioengineering, Genetics

Computer Science/Information Technology and Security

Life Sciences and Biology

Student and Faculty

Mercury ID

681734

Source updated

Fri, 04/25/2025 - 10:41

Thesis on Human-Centered AI Earns Honors from International Computing Organization

dgivens8 — Tue, 22 Apr 2025 14:30:41 +0000

Thesis on Human-Centered AI Earns Honors from International Computing Organization dgivens8 Tue, 04/22/2025 - 10:30

A Georgia Tech alum’s dissertation introduced ways to make artificial intelligence (AI) more accessible, interpretable, and accountable. Although it’s been a year since his doctoral defense, Zijie (Jay) Wang’s (Ph.D. ML-CSE 2024) work continues to resonate with researchers.

Wang is a recipient of the 2025 Outstanding Dissertation Award from the Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI). The award recognizes Wang for his lifelong work on democratizing human-centered AI.

“Throughout my Ph.D. and industry internships, I observed a gap in existing research: there is a strong need for practical tools for applying human-centered approaches when designing AI systems,” said Wang, now a safety researcher at OpenAI.

“My work not only helps people understand AI and guide its behavior but also provides user-friendly tools that fit into existing workflows.”

Wang’s dissertation presented techniques in visual explanation and interactive guidance to align AI models with user knowledge and values. The work culminated from years of research, fellowship support, and internships.

Wang’s most influential projects formed the core of his dissertation. These included:

CNN Explainer: an open-source tool developed for deep-learning beginners. Since its release in July 2020, more than 436,000 global visitors have used the tool.
DiffusionDB: a first-of-its-kind large-scale dataset that lays a foundation to help people better understand generative AI. This work could lead to new research in detecting deepfakes and designing human-AI interaction tools to help people more easily use these models.
GAM Changer: an interface that empowers users in healthcare, finance, or other domains to edit ML models to include knowledge and values specific to their domain, which improves reliability.
GAM Coach: an interactive ML tool that could help people who have been rejected for a loan by automatically letting an applicant know what is needed for them to receive loan approval.
Farsight: a tool that alerts developers when they write prompts in large language models that could be harmful and misused.

“I feel extremely honored and lucky to receive this award, and I am deeply grateful to many who have supported me along the way, including Polo, mentors, collaborators, and friends,” said Wang, who was advised by School of Computational Science and Engineering (CSE) Professor Polo Chau.

“This recognition also inspired me to continue striving to design and develop easy-to-use tools that help everyone to easily interact with AI systems.”

Like Wang, Chau advised Georgia Tech alumnus Fred Hohman (Ph.D. CSE 2020). Hohman won the ACM SIGCHI Outstanding Dissertation Award in 2022.

Chau’s group synthesizes machine learning (ML) and visualization techniques into scalable, interactive, and trustworthy tools. These tools increase understanding and interaction with large-scale data and ML models.

Chau is the associate director of corporate relations for the Machine Learning Center at Georgia Tech. Wang called the School of CSE his home unit while a student in the ML program under Chau.

Wang is one of five recipients of this year’s award to be presented at the 2025 Conference on Human Factors in Computing Systems (CHI 2025). The conference occurs April 25-May 1 in Yokohama, Japan.

SIGCHI is the world’s largest association of human-computer interaction professionals and practitioners. The group sponsors or co-sponsors 26 conferences, including CHI.

Wang’s outstanding dissertation award is the latest recognition of a career decorated with achievement.

Months after graduating from Georgia Tech, Forbes named Wang to its 30 Under 30 in Science for 2025 for his dissertation. Wang was one of 15 Yellow Jackets included in nine different 30 Under 30 lists and the only Georgia Tech-affiliated individual on the 30 Under 30 in Science list.

While a Georgia Tech student, Wang earned recognition from big names in business and technology. He received the Apple Scholars in AI/ML Ph.D. Fellowship in 2023 and was in the 2022 cohort of the J.P. Morgan AI Ph.D. Fellowships Program.

Along with the CHI award, Wang’s dissertation earned him awards this year at banquets across campus. The Georgia Tech chapter of Sigma Xi presented Wang with the Best Ph.D. Thesis Award. He also received the College of Computing’s Outstanding Dissertation Award.

“Georgia Tech attracts many great minds, and I’m glad that some, like Jay, chose to join our group,” Chau said. “It has been a joy to work alongside them and witness the many wonderful things they have accomplished, and with many more to come in their careers.”

Summary sentence

Zijie (Jay) Wang (Ph.D. ML-CSE 2024) is a recipient of the 2025 Outstanding Dissertation Award from the Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI).

Summary

Dateline

Thu, 04/17/2025 - 12:00

Contact

Bryant Wine, Communications Officer
bryant.wine@cc.gatech.edu

Keywords

College of Computing

School of Computational Science and Engineering

artificial intelligence (AI)

Georgia Tech News Center

Daily Digest

Research Horizons

go-researchnews

go-ai

Categories

Computer Science/Information Technology and Security

Congressional Testimony

Digital Media and Entertainment

Economic Development and Policy

Education

Energy

Engineering

Environment

Exhibitions

Georgia Tech Arts

Industrial Design

Institute and Campus

Institute Leadership

International Education

Life Sciences and Biology

Military Technology

Music and Music Technology

Nanotechnology and Nanoscience

Performances

Physics and Physical Sciences

Policy, Social Sciences, and Liberal Arts

Research

Robotics

Special Events and Guest Speakers

Student Honors and Achievements

Student Research

Mercury ID

681961

Source updated

Tue, 04/22/2025 - 10:29

Machine Learning Encoder Improves Weather Forecasting and Tsunami Prediction

admin — Wed, 26 Mar 2025 01:20:41 +0000

Machine Learning Encoder Improves Weather Forecasting and Tsunami Prediction admin Tue, 03/25/2025 - 21:20

Successful test results of a new machine learning (ML) technique developed at Georgia Tech could help communities prepare for extreme weather and coastal flooding. The approach could also be applied to other models that predict how natural systems impact society.

Ph.D. student Phillip Si and Assistant Professor Peng Chen developed Latent-EnSF, a technique that improves how ML models assimilate data to make predictions.

In experiments predicting medium-range weather forecasting and shallow water wave propagation, Latent-EnSF demonstrated higher accuracy, faster convergence, and greater efficiency than existing methods for sparse data assimilation.

“We are currently involved in an NSF-funded project aimed at providing real-time information on extreme flooding events in Pinellas County, Florida,” said Si, who studies computational science and engineering (CSE).

“We're actively working on integrating Latent-EnSF into the system, which will facilitate accurate and synchronized modeling of natural disasters. This initiative aims to enhance community preparedness and safety measures in response to flooding risks.”

Latent-EnSF outperformed three comparable models in assimilation speed, accuracy, and efficiency in shallow water wave propagation experiments. These tests show models can make better and faster predictions of coastal flood waves, tides, and tsunamis.

In experiments on medium-range weather forecasting, Latent-EnSF surpassed the same three control models in accuracy, convergence, and time. Additionally, this test demonstrated Latent-EnSF's scalability compared to other methods.

These promising results support using ML models to simulate climate, weather, and other complex systems.

Traditionally, such studies require employment of large, energy-intensive supercomputers. However, advances like Latent-EnSF are making smaller, more efficient ML models feasible for these purposes.

The Georgia Tech team mentioned this comparison in its paper. It takes hours for the European Center for Medium-Range Weather Forecasts computer to run its simulations. Conversely, the ML model FourCastNet calculated the same forecast in seconds.

“Resolution, complexity, and data-diversity will continue to increase into the future,” said Chen, an assistant professor in the School of CSE.

“To keep pace with this trend, we believe that ML models and ML-based data assimilation methods will become indispensable for studying large-scale complex systems.”

Data assimilation is the process by which models continuously ingest new, real-world data to update predictions. This data is often sparse, meaning it is limited, incomplete, or unevenly distributed over time.

Latent-EnSF builds on the Ensemble Filter Scores (EnSF) model developed by Florida State University and Oak Ridge National Laboratory researchers.

EnSF’s strength is that it assimilates data with many features and unpredictable relationships between data points. However, integrating sparse data leads to lost information and knowledge gaps in the model. Also, such large models may stop learning entirely from small amounts of sparse data.

The Georgia Tech researchers employ two variational autoencoders (VAEs) in Latent-EnSF to help ML models integrate and use real-world data. The VAEs encode sparse data and predictive models together in the same space to assimilate data more accurately and efficiently.

Integrating models with new methods, like Latent-EnSF, accelerates data assimilation. Producing accurate predictions more quickly during real-world crises could save lives and property for communities.

To share Latent-EnSF to the broader research community, Chen and Si presented their paper at the SIAM Conference on Computational Science and Engineering (CSE25). The Society of Industrial and Applied Mathematics (SIAM) organized CSE25, held March 3-7 in Fort Worth, Texas.

Chen was one of ten School of CSE faculty members who presented research at CSE25, representing one-third of the School’s faculty body. Latent-EnSF was one of 15 papers by School of CSE authors and one of 23 Georgia Tech papers presented at the conference.

The pair will also present Latent-EnSF at the upcoming International Conference on Learning Representations (ICLR 2025). Occurring April 24-28 in Singapore, ICLR is one of the world’s most prestigious conferences dedicated to artificial intelligence research.

“We hope to bring attention to experts and domain scientists the exciting area of ML-based data assimilation by presenting our paper,” Chen said. “Our work offers a new solution to address some of the key shortcomings in the area for broader applications.”

Summary sentence

Ph.D. student Phillip Si and Assistant Professor Peng Chen developed Latent-EnSF, a technique that improves how ML models assimilate data to make predictions.

Summary

Ph.D. student Phillip Si and Assistant Professor Peng Chen developed Latent-EnSF, a technique that improves how ML models assimilate data to make predictions.

Dateline

Fri, 03/14/2025 - 12:00

Contact

Bryant Wine, Communications Officer
bryant.wine@cc.gatech.edu

Keywords

College of Computing

School of Computational Science and Engineering

go-researchnews

Research Horizons

Daily Digest

Georgia Tech News Center

machine learning

artificial intelligence

News room topics

Earth and Environment

Science and Technology

Categories

City Planning, Transportation, and Urban Growth

Community

Computer Science/Information Technology and Security

Engineering

Environment

Research

Student and Faculty

Student Research

Mercury ID

681164

Source updated

Tue, 03/25/2025 - 21:19

Minority English Dialects Vulnerable to Automatic Speech Recognition Inaccuracy

admin — Mon, 02 Dec 2024 16:41:41 +0000

Minority English Dialects Vulnerable to Automatic Speech Recognition Inaccuracy admin Mon, 12/02/2024 - 11:41

The Automatic Speech Recognition (ASR) models that power voice assistants like Amazon Alexa may have difficulty transcribing English speakers with minority dialects.

A study by Georgia Tech and Stanford researchers compared the transcribing performance of leading ASR models for people using Standard American English (SAE) and three minority dialects — African American Vernacular English (AAVE), Spanglish, and Chicano English.

Interactive Computing Ph.D. student Camille Harris is the lead author of a paper accepted into the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) this week in Miami.

Harris recruited people who spoke each dialect and had them read from a Spotify podcast dataset, which includes podcast audio and metadata. Harris then used three ASR models — wav2vec 2.0, HUBERT, and Whisper — to transcribe the audio and compare their performances.

For each model, Harris found SAE transcription significantly outperformed each minority dialect. The models more accurately transcribed men who spoke SAE than women who spoke SAE. Members who spoke Spanglish and Chicano English had the least accurate transcriptions out of the test groups.

While the models transcribed SAE-speaking women less accurately than their male counterparts, that did not hold true across minority dialects. Minority men had the most inaccurate transcriptions of all demographics in the study.

“I think people would expect if women generally perform worse and minority dialects perform worse, then the combination of the two must also perform worse,” Harris said. “That’s not what we observed.

“Sometimes minority dialect women performed better than Standard American English. We found a consistent pattern that men of color, particularly Black and Latino men, could be at the highest risk for these performance errors.”

Addressing underrepresentation

Harris said the cause of that outcome starts with the training data used to build these models. Model performance reflected the underrepresentation of minority dialects in the data sets.

AAVE performed best under the Whisper model, which Harris said had the most inclusive training data of minority dialects.

Harris also looked at whether her findings mirrored existing systems of oppression. Black men have high incarceration rates and are one of the people groups most targeted by police. Harris said there could be a correlation between that and the low rate of Black men enrolled in universities, which leads to less representation in technology spaces.

“Minority men performing worse than minority women doesn’t necessarily mean minority men are more oppressed,” she said. “They may be less represented than minority women in computing and the professional sector that develops these AI systems.”

Harris also had to be cautious of a few variables among AAVE, including code-switching and various regional subdialects.

Harris noted in her study there were cases of code-switching to SAE. Speakers who code-switched performed better than speakers who did not.

Harris also tried to include different regional speakers.

“It’s interesting from a linguistic and history perspective if you look at migration patterns of Black folks — perhaps people moving from a southern state to a northern state over time creates different linguistic variations,” she said. “There are also generational variations in that older Black Americans may speak differently from younger folks. I think the variation was well represented in our data. We wanted to be sure to include that for robustness.”

TikTok barriers

Harris said she built her study on a paper she authored that examined user-design barriers and biases faced by Black content creators on TikTok. She presented that paper at the Association of Computing Machinery’s (ACM) 2023 Conference on Computer Supported Cooperative Works.

Those content creators depended on TikTok for a significant portion of their income. When providing captions for videos grew in popularity, those creators noticed the ASR tool built into the app inaccurately transcribed them. That forced the creators to manually input their captions, while SAE speakers could use the ASR feature to their benefit.

“Minority users of these technologies will have to be more aware and keep in mind that they’ll probably have to do a lot more customization because things won’t be tailored to them,” Harris said.

Harris said there are ways that designers of ASR tools could work toward being more inclusive of minority dialects, but cultural challenges could arise.

“It could be difficult to collect more minority speech data, and you have to consider consent with that,” she said. “Developers need to be more community-engaged to think about the implications of their models and whether it’s something the community would find helpful.”

Subtitle

Summary sentence

The Automatic Speech Recognition (ASR) models that power voice assistants like Amazon Alexa may have difficulty transcribing English speakers with minority dialects.

Summary

Interactive Computing Ph.D. student Camille Harris is the lead author of a paper accepted into the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) this week in Miami.

Dateline

Fri, 11/15/2024 - 12:00

ndeen6@gatech.edu

Contact

Nathan Deen

Communications Officer

School of Interactive Computing

Associated importer

Keywords

speech recognition

bias

Research Horizons

go-research

go-researchnews

go-ai

Artifical Intelligence

inequality

News room topics

Society and Culture

Categories

Computer Science/Information Technology and Security

Mercury ID

678471

Source updated

Mon, 12/02/2024 - 11:39

Study Highlights Challenges in Detecting Violent Speech Aimed at Asian Communities

admin — Mon, 12 Aug 2024 14:36:41 +0000

Study Highlights Challenges in Detecting Violent Speech Aimed at Asian Communities admin Mon, 08/12/2024 - 10:36

A research group is calling for internet and social media moderators to strengthen their detection and intervention protocols for violent speech.

Their study of language detection software found that algorithms struggle to differentiate anti-Asian violence-provoking speech from general hate speech. Left unchecked, threats of violence online can go unnoticed and turn into real-world attacks.

Researchers from Georgia Tech and the Anti-Defamation League (ADL) teamed together in the study. They made their discovery while testing natural language processing (NLP) models trained on data they crowdsourced from Asian communities.

“The Covid-19 pandemic brought attention to how dangerous violence-provoking speech can be. There was a clear increase in reports of anti-Asian violence and hate crimes,” said Gaurav Verma, a Georgia Tech Ph.D. candidate who led the study.

“Such speech is often amplified on social platforms, which in turn fuels anti-Asian sentiments and attacks.”

Violence-provoking speech differs from more commonly studied forms of harmful speech, like hate speech. While hate speech denigrates or insults a group, violence-provoking speech implicitly or explicitly encourages violence against targeted communities.

Humans can define and characterize violent speech as a subset of hateful speech. However, computer models struggle to tell the difference due to subtle cues and implications in language.

The researchers tested five different NLP classifiers and analyzed their F1 score, which measures a model's performance. The classifiers reported a 0.89 score for detecting hate speech, while detecting violence-provoking speech was only 0.69. This contrast highlights the notable gap between these tools and their accuracy and reliability.

The study stresses the importance of developing more refined methods for detecting violence-provoking speech. Internet misinformation and inflammatory rhetoric escalate tensions that lead to real-world violence.

The Covid-19 pandemic exemplified how public health crises intensify this behavior, helping inspire the study. The group cited that anti-Asian crime across the U.S. increased by 339% in 2021 due to malicious content blaming Asians for the virus.

The researchers believe their findings show the effectiveness of community-centric approaches to problems dealing with harmful speech. These approaches would enable informed decision-making between policymakers, targeted communities, and developers of online platforms.

Along with stronger models for detecting violence-provoking speech, the group discusses a direct solution: a tiered penalty system on online platforms. Tiered systems align penalties with severity of offenses, acting as both deterrent and intervention to different levels of harmful speech.

“We believe that we cannot tackle a problem that affects a community without involving people who are directly impacted,” said Jiawei Zhou, a Ph.D. student who studies human-centered computing at Georgia Tech.

“By collaborating with experts and community members, we ensure our research builds on front-line efforts to combat violence-provoking speech while remaining rooted in real experiences and needs of the targeted community.”

The researchers trained their tested NLP classifiers on a dataset crowdsourced from a survey of 120 participants who self-identified as Asian community members. In the survey, the participants labeled 1,000 posts from X (formerly Twitter) as containing either violence-provoking speech, hateful speech, or neither.

Since characterizing violence-provoking speech is not universal, the researchers created a specialized codebook for survey participants. The participants studied the codebook before their survey and used an abridged version while labeling.

To create the codebook, the group used an initial set of anti-Asian keywords to scan posts on X from January 2020 to February 2023. This tactic yielded 420,000 posts containing harmful, anti-Asian language.

The researchers then filtered the batch through new keywords and phrases. This refined the sample to 4,000 posts that potentially contained violence-provoking content. Keywords and phrases were added to the codebook while the filtered posts were used in the labeling survey.

The team used discussion and pilot testing to validate its codebook. During trial testing, pilots labeled 100 Twitter posts to ensure the sound design of the Asian community survey. The group also sent the codebook to the ADL for review and incorporated the organization’s feedback.

“One of the major challenges in studying violence-provoking content online is effective data collection and funneling down because most platforms actively moderate and remove overtly hateful and violent material,” said Tech alumnus Rynaa Grover (M.S. CS 2024).

“To address the complexities of this data, we developed an innovative pipeline that deals with the scale of this data in a community-aware manner.”

Emphasis on community input extended into collaboration within Georgia Tech’s College of Computing. Faculty members Srijan Kumar and Munmun De Choudhury oversaw the research that their students spearheaded.

Kumar, an assistant professor in the School of Computational Science and Engineering, advises Verma and Grover. His expertise is in artificial intelligence, data mining, and online safety.

De Choudhury is an associate professor in the School of Interactive Computing and advises Zhou. Their research connects societal mental health and social media interactions.

The Georgia Tech researchers partnered with the ADL, a leading non-governmental organization that combats real-world hate and extremism. ADL researchers Binny Mathew and Jordan Kraemer co-authored the paper.

The group will present its paper at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), which takes place in Bangkok, Thailand, Aug. 11-16

ACL 2024 accepted 40 papers written by Georgia Tech researchers. Of the 12 Georgia Tech faculty who authored papers accepted at the conference, nine are from the College of Computing, including Kumar and De Choudhury.

“It is great to see that the peers and research community recognize the importance of community-centric work that provides grounded insights about the capabilities of leading language models,” Verma said.

“We hope the platform encourages more work that presents community-centered perspectives on important societal problems.”

Visit https://sites.gatech.edu/research/acl-2024/ for news and coverage of Georgia Tech research presented at ACL 2024.

Summary sentence

A study of language detection software found that algorithms struggle to differentiate anti-Asian violence-provoking speech from general hate speech. Left unchecked, threats of violence online can go unnoticed and turn into real-world attacks.

Summary

A research group is calling for internet and social media moderators to strengthen their detection and intervention protocols for violent speech.

Dateline

Wed, 08/07/2024 - 12:00

Contact

Bryant Wine, Communications Officer
bryant.wine@cc.gatech.edu

Keywords

go-ai

Daily Digest

Research Horizons

go-researchnews

School of Computational Science and Engineering

artificial intelligence (AI)

News room topics

Science and Technology

Categories

Computer Science/Information Technology and Security

Research

Student and Faculty

Student Research

Mercury ID

675889

Source updated

Mon, 08/12/2024 - 10:35

Meet VAL, an AI Teammate That Can Adapt to Your Tendencies

admin — Fri, 12 Jul 2024 13:51:41 +0000

Meet VAL, an AI Teammate That Can Adapt to Your Tendencies admin Fri, 07/12/2024 - 09:51

A team’s success in any competitive environment often hinges on how well each member can anticipate the actions of their teammates.

Assistant Professor Christopher MacLellan thinks teachable artificial intelligence (AI) agents are uniquely suited for this role and make ideal teammates for video gamers.

With the help of funding from the U.S. Department of Defense, MacLellan hopes to prove his theory with a conversational, task-performing agent he co-engineered called the Verbal Apprentice Learner (VAL).

“You need the ability to adapt to what your teammates are doing to be an effective teammate,” MacLellan said. “We’re exploring this capability for AI agents in the context of video games.”

Unlike generative AI chatbots like ChatGPT, VAL uses an interactive task-learning approach.

“VAL learns how you do things in the way you want them done,” MacLellan said. “When you tell it to do something, it will do it the way you taught it instead of some generic random way from the internet.”

A key difference between VAL and a chatbot is that VAL can perceive and act within the gaming world. A chatbot, like ChatGPT, only perceives and acts within the chat dialog.

MacLellan immersed VAL into an open-sourced, simplified version of the popular Nintendo cooperative video game Overcooked to discover how well the agent can function as a teammate. In Overcooked, up to four players work together to prepare dishes in a kitchen while earning points for every completed order.

How Fast Can Val Learn?

In a study with 12 participants, MacLellan found that users could often correctly teach VAL new tasks with only a few examples.

First, the user must teach VAL how to play the game. Knowing that a single human error could compromise results, MacLellan designed three precautionary features:

When VAL receives a command such as "cook an onion," it asks clarifying questions to understand and confirm its task. As VAL continues to learn, clarification prompts decrease.
An “undo” button to ensure users can reverse an errant command.
VAL contains GPT subcomponents to interpret user input, allowing it to adapt to ambiguous commands and typos. The GPT subcomponents drive changes in VAL’s task knowledge, which it uses to perform tasks without additional guidance.

The participants in MacLellan’s study used these features to ensure VAL learned the tasks correctly.

The high volume of prompts creates a more tedious experience. Still, MacLellan said it provides detailed data on system performance and user experience. That insight should make designing a more seamless experience in future versions of VAL possible.

The prompts also require the AI to be explainable.

“When VAL learns something, it uses the language model to label each node in the task knowledge graph that the system constructs,” MacLellan said. “You can see what it learned and how it breaks tasks down into actions.”

Beyond Gaming

MacLellan’s Teachable AI Lab is devoted to developing AI that inexperienced users can train.

“We are trying to come up with a more usable system where anyone, including people with limited expertise, could come in and interact with the agent and be able to teach it within just five minutes of interacting with it for the first time,” he said.

His work caught the attention of the Department of Defense, which awarded MacLellan multiple grants to fund several of his projects, including VAL. The possibilities of how the DoD could use VAL, on and off the battlefield, are innumerable.

“(The DoD) envisions a future in which people and AI agents jointly work together to solve problems,” MacLellan said. “You need the ability to adapt to what your teammates are doing to be an effective teammate.

“We look at the dynamics of different teaming circumstances and consider what are the right ways to team AI agents with people. The key hypothesis for our project is agents that can learn on the fly and adapt to their users will make better teammates than those that are pre-trained like GPT.”

Design Your Own Agent

MacLellan is co-organizing a gaming agent design competition sponsored by the Institute of Electrical and Electronic Engineers (IEEE) 2024 Conference on Games in Milan, Italy.

The Dice Adventure Competition invites participants to design their own AI agent to play a multi-player, turn-based dungeon crawling game or to play the game as a human teammate. The competition this month and in July offers $1,000 in prizes for players and agent developers in the top three teams.

Summary sentence

A new AI teammate developed by Assistant Professor Christopher MacLellan could be the ideal co-opt video game partner.

Summary

A team’s success in any competitive environment often hinges on how well each member can anticipate the actions of their teammates.

Assistant Professor Christopher MacLellan thinks teachable artificial intelligence (AI) agents are uniquely suited for this role and make ideal teammates for video gamers.

Dateline

Thu, 06/27/2024 - 12:00

Contact

Nathan Deen

Communications Officer

School of Interactive Computing

Associated importer

Keywords

go-ai

artificial intelligence (AI)

Video gaming

gaming

go-researchnews

Research Horizons

News room topics

Science and Technology

Categories

Computer Science/Information Technology and Security

Mercury ID

675255

Source updated

Fri, 07/12/2024 - 09:49

Research Horizons

Georgia Tech Team Takes Second Place at ICRA Robot Teleoperation Contest

Keywords

Categories

Middle Schoolers’ Feedback Informs New Approach to AI-based Museum Exhibits

Expanding Accessibility

Making an Impact

Keywords

Categories

Georgia Tech to Strengthen Nation’s Faculty Development in Geospace Science

Keywords

Categories

Computing Framework Could Reveal Signs of Neuro Disorders Hidden within Brain Data

Related links

Keywords

News room topics

Categories

Faculty Earn Fellowships for Heart Modeling and Data Optimization Research

Keywords

News room topics

Categories

Thesis on Human-Centered AI Earns Honors from International Computing Organization

Related links

Keywords

Categories

Machine Learning Encoder Improves Weather Forecasting and Tsunami Prediction

Related links

Keywords

News room topics

Categories

Minority English Dialects Vulnerable to Automatic Speech Recognition Inaccuracy

Addressing underrepresentation

TikTok barriers

Keywords

News room topics

Categories

Study Highlights Challenges in Detecting Violent Speech Aimed at Asian Communities

Related links

Keywords

News room topics

Categories

Meet VAL, an AI Teammate That Can Adapt to Your Tendencies

How Fast Can Val Learn?

Beyond Gaming

Design Your Own Agent

Keywords

News room topics

Categories