This article is based the research of Kelley Assistant Professor of Operations and Decision Technologies, Nick Brown. This work was co-authored by Dezhi Wu (U. of South Carolina), Jun Zhang (Wuhan University), Greg Moody (U. of Nevada Las Vegas), Paul Lowry (Virginia Tech).

Introduction

When a device user is at risk of a security breach, they likely receive a push notification designed to interrupt what they’re doing and direct immediate attention to mitigating the threat. These notifications use fear-based appeals to prompt a response—most often, to heed the security recommendation and follow the protective actions.

Existing research shows that users frequently ignore or dismiss these security notifications, particularly when they are perceived as intrusive or require cognitive effort to process. Alarmingly, up to 87% of crucial security notifications go unheeded, and the disruptive nature of how they are delivered may potentially undermine their effectiveness. (more…)

This article is based on a presentation at the Fall 2023 Institute of Business Analytics Conference on Generative AI and Cybersecurity: Navigating the New Era of Threats and Safeguards, by Mikel Rodriguez, leader of the AI Red Team at Google DeepMind.

For the past 20 years, if someone wanted to interact with machine learning (ML) systems, they would have to go to a state-of-the-art lab. Today, anyone who is interested in ML can pull out their phone and easily interact with these interfaces.

“The world is accelerating so quickly,” says Mikel Rodriguez, who leads AI assurance at Google DeepMind. “There isn’t a week that goes by without some interesting new development in artificial intelligence capabilities.” (more…)

This article is based on a presentation at the Fall 2023 Institute of Business Analytics Conference on Generative AI and Cybersecurity: Navigating the New Era of Threats and Safeguards. by Tori Westerhoff, senior manager of strategy and business planning for mixed reality on the Microsoft AI Red Team

Amid the bustle of innovation in artificial intelligence, thought leaders in both the private and public sectors are more clearly seeing the importance of security measures to make sure AI technology is as powerful as possible without doing harm.

To that end, in late 2023, the Biden Administration issued a landmark Executive Order that established unprecedented standards for AI safety and security. The Executive Order was designed to more carefully regulate the development of trustworthy technology and to protect privacy while promoting innovation and competition in the AI landscape.

Alongside these efforts, companies like Microsoft are finding new ways to stay ahead of the prolific acceleration of the technology with security processes like red teaming.

Tori Westerhoff, senior manager of strategy and business planning for mixed reality on the Microsoft AI Red Team, has found that the awareness of the entire risk landscape for AI is generally low.

(more…)

This article is based on a presentation at the Fall 2023 Institute of Business Analytics Conference on Generative AI and Cybersecurity: Navigating the New Era of Threats and Safeguards by Christina Liaghati, AI Strategy Execution & Operations Manager at MITRE.

Over a two-and-a-half-year period, an unsophisticated hacker duo was able to steal $77 million from the Chinese government by attacking a machine learning-enabled facial identification system that gave them access to the Shanghai Tax Authority.

The attackers created a fake shell company, which they used to acquire black market cell phones and headshots of Chinese citizens, as well as a long list of personal identifiable information. They were able to tweak the headshots and use the identifiable information to register accounts that evaded the ML-enabled facial recognition system used to verify identity.

With the privileged access into the Shanghai Tax Authority’s verified account environment, they repeatedly submitted fraudulent invoices that totaled $77 million.

(more…)