Generative AI (GenAI) is transforming industries, bringing new levels of automation, personalization, and efficiency. But as GenAI’s capabilities grow, so do the associated security risks. From potential data breaches to risky, unintended outputs, organizations face a wide range of security challenges that demand attention. Despite the fact that 80% of executives acknowledge the importance of trustworthy AI, less than 25% have implemented effective security protocols, creating a notable gap between awareness and practical action.

To help you bridge this gap, we’ve gathered five essential security practices presented at the recent AI Waves webinar, where expert panelists Diana Kelley, CISO at ProtectAI; Łukasz Boruń, AI Solution Architect at Miquido; and Deepak Prabhakara, CEO of BoxHQ, shared practical, real-world examples of why these practices are critical to securing GenAI applications.

1. Prioritize data security at every stage

Data protection is crucial when working with GenAI applications. These systems rely on large datasets, often containing sensitive information, to train models and produce accurate outputs.

Experts have pointed out that Generative AI tools may inadvertently retain data from their training, posing significant risks of exposing PII through model outputs. Given stringent regulations like GDPR in Europe and CCPA in California, organizations must ensure their GenAI models are compliant and do not accidentally breach privacy laws by revealing sensitive information.Diana Kelley explained that it’s all too easy for GenAI models to retain and inadvertently reveal sensitive information embedded in training data, noting,

A GenAI model can sometimes reproduce fragments of its training data, such as names or contact details, when prompted repeatedly. This unintended disclosure highlights why encryption and anonymization are essential to minimize data retention risks. By encrypting private data during every phase—from training to transmission—and anonymizing information wherever possible, organizations can drastically reduce their vulnerability to data leaks. Regular audits are also vital to identify and remove outdated data, minimizing the overall exposure of sensitive information. Techniques like federated learning, which allow models to process data locally, provide another layer of protection by avoiding the need to centralize personal data in a single location.

Steps to mitigate data privacy risks:

• Always encrypt sensitive data during training and deployment to minimize exposure risks.
• Conduct routine audits of training data to eliminate unnecessary or outdated information, thereby reducing the risk of sensitive data leakage.
• Utilize methods such as federated learning, which allows for decentralized training and mitigates data exposure risks.

2. Implement rigorous human oversight

GenAI-generated outputs often seem credible and authoritative, but without human oversight, organizations risk over-reliance on AI, especially in sensitive sectors like healthcare or finance. Diana Kelley emphasized that overtrust is one of the biggest risks in AI security:

In one case, an AI-driven customer service bot advised a traveler to purchase a new ticket without explaining refund options, ultimately causing a financial and legal issue for the company. This incident highlights the need for rigorous human oversight to catch such critical errors. For applications where accuracy is paramount, implementing a multi-tiered review AI system with human validation can prevent AI-generated errors before they become problems. Additionally, fostering a culture that treats AI as a support tool rather than a final authority encourages employees to think critically about AI outputs. In high-stakes cases, cross-validating information across multiple AI models helps reduce the risk of relying on a single, potentially biased output.

3. Secure the AI infrastructure

The security of your AI infrastructure is fundamental to protecting data integrity and ensuring a resilient system. Free-tier AI models may be attractive from a budget any perspective, but they often lack the security features needed for enterprise applications. Deepak Prabhakara underscored this point by noting that while many security challenges with GenAI may seem familiar, the scale and complexity of the risks are entirely new:

For instance, a large enterprise planning to deploy an AI tool  on a public cloud service had to delay its launch when internal security teams identified vulnerabilities in access control. Although this delay incurred significant costs, it ultimately protected the organization’s sensitive data from potential exposure. Opting for enterprise-grade AI models with advanced data isolation and encryption is essential, especially for sensitive applications. For high-sensitivity use cases, on-premise deployments can offer greater control over data handling, reducing reliance on third-party security measures. Additionally, conducting regular penetration tests can help identify and mitigate vulnerabilities, keeping infrastructure defenses strong and up-to-date.

4. Defend against prompt injection attacks

Prompt injection attacks exploit GenAI’s tendency to interpret inputs flexibly, sometimes allowing malicious actors to manipulate the model’s responses. This is particularly concerning for customer-facing GenAI applications that rely on user inputs. Łukasz Boruń highlighted how vital it is to protect these models from such attacks, explaining:

A chatbot developed for NYC services inadvertently responded to a prompt suggesting that food partially eaten by rodents was safe to serve. In another instance, a similar chatbot even approved a hypothetical business selling human meat. Such responses demonstrate how easily prompts can be manipulated and underscore the need for robust defenses against these attacks. Validating and sanitizing inputs before they reach the model is a crucial first step in preventing prompt injections. By exposing models to adversarial examples during training, organizations can prepare them to recognize and appropriately respond to potentially harmful inputs. Real-time monitoring of input data further enables immediate action if suspicious prompts arise, helping to mitigate the risks associated with prompt injection.

5. Carefully vet third-party models

Third-party pre-trained models can speed up AI development, but they also bring security and compliance risks if not thoroughly vetted. Many organizations overlook the biases and vulnerabilities that machine learning models may introduce, which can lead to legal and reputational consequences. Deepak Prabhakara noted the risks of relying on unvetted models:

A Polish e-commerce platform experienced this firsthand when an unvetted AI bot misinterpreted a prompt and issued a 100% discount on a high-value item. The result was not only a significant financial loss but also damaged customer trust. To avoid such pitfalls, it’s essential to source models from reputable providers with clear data handling standards. Regularly assessing these models for biases or hidden vulnerabilities ensures they align with the organization’s ethical and operational requirements. Using Retrieval-Augmented Generation (RAG) to pull real-time data from trusted sources can further reduce reliance on potentially outdated or flawed pre-trained models, keeping AI outputs more accurate and reliable.

Choosing between fine-tuning and RAG for enhanced security in GenAI applications

As AI developers, we often find ourselves at a crossroads, deciding between two powerful strategies for customizing our GenAI applications: fine-tuning and Retrieval-Augmented Generation (RAG). Each approach offers unique security advantages that cater to different application needs, so let’s dive into what makes each option stand out!

Fine-tuning: high control with added complexity

Fine-tuning is like giving your pre-existing model a focused makeover. By retraining it on specific domain data, you gain tighter control over its outputs. This is fantastic for scenarios where accuracy is king. However, let’s be real—it’s not the easiest path. Fine-tuning can be resource-intensive and requires a commitment to retraining. This method shines in industries where precision is non-negotiable, such as healthcare or finance.

When to choose fine-tuning:

• When accuracy and control are paramount, and you have the resources for retraining.
• In highly regulated sectors where precise AI outputs are essential.

RAG: speed and security through real-time data access

On the flip side, we have RAG, which brings a whole new level of agility to the table. This approach allows your applications to pull information from trusted sources in real-time, reducing reliance on static pre-trained data. It’s perfect for scenarios where speed and scalability are key—think customer data support applications that need quick, dynamic responses.

When to opt for RAG:

• When speed and scalability take precedence.
• For applications that thrive on real-time access to ever-changing datasets.

In the end, your choice between fine-tuning and RAG will depend on your specific needs and goals. By carefully weighing the benefits of each approach, you can enhance the security and performance of your GenAI applications, all while catering to the demands of your users. 

Conclusion: building a secure foundation for GenAI applications

As GenAI solutions become an integral part of business operations, securing these applications is more important than ever. By implementing these five best practices—prioritizing data privacy, incorporating human oversight, securing the infrastructure, defending against injection attacks, and vetting third-party models—organizations can lay a solid foundation for safe and resilient GenAI applications. Addressing security proactively allows you to unlock GenAI’s transformative potential with confidence, knowing that your data and reputation are protected. This article serves as a summary of the webinar’s findings, but if you’re hungry for more and want to delve into how to safeguard your GenAI applications with insights from security experts, download the recording.

Top AI innovations delivered monthly!

The administrator of your personal data is Miquido sp. z o.o. sp.k., with its ... registered office in Kraków at Zabłocie 43A, 30 - 701. We process the provided information in order to send you a newsletter. The basis for processing of your data is your consent and Miquido’s legitimate interest. You may withdraw your consent at any time by contacting us at marketing@miquido.com. You have the right to object, the right to access your data, the right to request rectification, deletion or restriction of data processing. For detailed information on the processing of your personal data, please see Privacy Policy.

Show more