Customer Chatbots Are Convenient But Are They Secure

The proliferation of customer chatbots built on general-purpose AI engines has made them easy to develop but challenging to secure. A notable incident in January 2024 highlighted these security challenges when Ashley Beauchamp managed to trick DPD's chatbot into behaving unconventionally. The chatbot ended up criticizing DPD’s service, using inappropriate language, and even composing a disparaging haiku about its owner. This incident, often referred to as "jailbreaking," involves breaching AI's guardrails through a technique called prompt engineering.

The Challenge of Securing Chatbots

From June to September 2023, Immersive Labs conducted a public challenge to assess how easily a chatbot could be jailbroken through prompt engineering. The results, involving over 34,500 participants, were alarming. The challenge revealed significant vulnerabilities in chatbot security, with a high success rate of participants bypassing the chatbot's protective measures to extract sensitive information.

How Chatbots Work

Chatbots typically operate on top of large-scale AI systems like ChatGPT. They are constructed using the ChatGPT API and are given customer-specific instructions and guardrails. When a user submits a query, it is processed by ChatGPT, which returns an answer to the chatbot for delivery to the user. Although these interactions are theoretically protected by both ChatGPT’s and the chatbot's own guardrails, the Immersive Labs challenge demonstrated that these protections are often insufficient.

The Immersive Labs Challenge Findings

In the challenge, participants faced ten levels of increasing difficulty in tricking the ILGPT chatbot into revealing a forbidden word, "password." At the lowest level, where the chatbot was simply instructed not to reveal the word, 88% of participants succeeded. Even at higher levels, where additional guardrails were implemented, a significant percentage of participants continued to bypass the protections. By the final level, 17% of participants could still defeat the chatbot’s defenses using engineered prompts.

Who Is at Risk?

Any organization deploying chatbots without robust security measures is at risk. This includes companies across various industries that use chatbots for customer service, support, and engagement. The primary risks include:

1. Reputation Damage: A compromised chatbot can damage a company’s reputation, leading to loss of customer trust and potential financial losses.

2. Data Theft: Inadequately secured chatbots can be exploited to steal sensitive and proprietary information.

3. Operational Disruptions: Malicious actors can manipulate chatbots to disrupt operations, leading to significant business interruptions.

How to Protect Your Systems

To protect against these risks, organizations should implement the following measures:

1. Deploy Robust Guardrails: Ensure that chatbots are equipped with multiple layers of security measures, including industry-standard data loss prevention (DLP) techniques.

2. Regular Security Audits: Conduct frequent security assessments to identify and address vulnerabilities in chatbot systems.

3. User Education: Train staff and users to recognize and report suspicious chatbot behavior.

4. Utilize Advanced AI Security Solutions: Invest in advanced AI security solutions that can detect and mitigate sophisticated prompt engineering attacks.

The Dangers of Using Unvetted Python Libraries

Another area of concern is the use of unvetted Python libraries in developing chatbots. Libraries that have not been properly vetted can introduce significant security risks:

1. Unknown Vulnerabilities: Unvetted libraries may contain hidden vulnerabilities that can be exploited by attackers.

2. Malicious Code: Some libraries might include malicious code intentionally embedded by threat actors.

3. Lack of Updates: Libraries that are not actively maintained may not receive timely updates, leaving them susceptible to new threats.

Best Practices for Using Python Libraries

1. Research and Verification: Before using a library, thoroughly research its background and verify its security credentials.

2. Regular Updates: Keep all libraries and dependencies up to date with the latest security patches.

3. Use Trusted Sources: Download libraries from trusted sources and repositories to ensure their integrity.

4. Conduct Security Audits: Regularly audit your codebase and dependencies for potential security issues.

The Evolving Threat Landscape

The use of chatbots and AI is still in its early stages, and the potential for misuse is significant. As chatbots become more sophisticated, the risks associated with their use will only increase. Kevin Breen, director of cyber threat research at Immersive Labs, emphasized that relying solely on AI's built-in guardrails is insufficient. Comprehensive security measures, including prompt engineering defenses and regular updates, are essential.

In conclusion, while chatbots offer significant benefits in customer service and engagement, their security must be a priority. Organizations should implement robust security measures, conduct regular audits, and stay informed about the latest threats to protect their systems and data. By doing so, they can leverage the advantages of AI while mitigating the associated risks.