ChatGPT processes over a billion queries daily from businesses. How much sensitive data is being leaked in the process?

The appeal of AI tools like ChatGPT is hard to ignore. With just a few clicks, employees can write emails, create code, study data, and solve challenging problems in seconds instead of hours.

This tech breakthrough has caught the attention of businesses everywhere, with AI use growing rapidly across companies — now used by an impressive 92% of Fortune 500 companies.

But underneath this productivity boost lies a serious problem that many companies haven't fully understood yet.

As ChatGPT and other Generative AI platforms handle over a billion questions daily — from more than 800 million weekly users — security experts are raising more and more urgent warnings about the risks these public AI tools create for business security, data protection, and following regulations.

Nearly half of companies have no AI-focused security measures in place, which creates a perfect recipe for trouble — especially in an age where AI-related data leaks are the #1 security worry for 69% of businesses in 2025. In this detailed guide, we'll look at why public AI tools pose real threats to your business and explain what a truly safe AI setup should look like.

The Hidden Dangers of Public AI Tools

Data Leakage and Retention: Your Business Secrets Aren't Safe

When employees paste sensitive information into public AI tools like ChatGPT, they're essentially broadcasting company secrets to a third-party system with limited accountability.

Research shows that sensitive data makes up approximately 11% of employee inputs to ChatGPT — a staggering figure when you consider the volume of daily queries.

But the problem extends far beyond traditional personal information.

Employees regularly share:

• Proprietary source code (as demonstrated in the Samsung incident where engineers leaked confidential semiconductor code)
• Strategic planning documents and internal meeting notes
• Customer data for "quick analysis"
• Financial projections and business intelligence
• Product development roadmaps and patent-worthy ideas

Even though OpenAI has strengthened its data protection measures, ChatGPT still retains chat history for (at least) 30 days and uses this information to improve its services.

Even with enterprise versions offering improved data handling, the risks remain substantial — particularly when employees default to the consumer version out of convenience or habit.

Compromised Credentials: The Keys to Your Kingdom

A security breach in 2024 exposed over 225,000 OpenAI credentials on dark web markets, stolen through various infostealer malware (with LummaC2 being particularly prevalent). These credentials provide complete access to chat history — including any sensitive business information previously shared.

Consider this scenario: An employee uses their work email to create a ChatGPT account, shares sensitive company information during their interactions, and then has their credentials stolen through a phishing attack. Now criminals have access not just to the employee's account but to all the proprietary information they've shared with the AI.

AI-Powered Social Engineering: Phishing on Steroids

Bad actors now leverage ChatGPT to craft highly convincing phishing campaigns that can fool even security-conscious employees. These aren't your typical grammatically challenged scam emails — they're sophisticated, personalized attacks that can mimic specific individuals within your organization with frightening accuracy.

Recent research demonstrates that AI-generated phishing emails have significantly higher success rates, with some AI-driven attacks bypassing multi-factor authentication through convincingly crafted urgent messages.

Real-World Consequences of Public AI Tool Usage

The Samsung incident provides a sobering example of what can go wrong. Engineers from Samsung's semiconductor division inadvertently leaked confidential code through ChatGPT while seeking debugging help, potentially exposing trade secrets worth billions. According to a company-wide survey conducted afterward, 65% of Samsung employees expressed serious concerns about generative AI security risks — but only after the damage was done.

More recently, a bug in the Redis open-source library used by ChatGPT allowed certain users to view the titles and first messages of other users' conversations, creating yet another vector for potential data exposure.

These incidents highlight a critical reality: Even when the AI platform itself implements strong security measures, the way organizations and employees use these tools — particularly without proper governance frameworks — creates substantial risk.

The Regulatory Storm on the Horizon

EU AI Act: A Game-Changing Regulatory Framework

The EU AI Act categorizes AI applications by risk level, from prohibited uses to minimal risk categories. High-risk AI applications in sectors like law enforcement and employment face particularly strict compliance standards. Non-compliance will result in penalties of up to €35 million or 7% of worldwide annual turnover — whichever is higher. Key provisions begin taking effect in early 2025, with full compliance required by August 2026.

U.S. State-Level Regulations: A Patchwork of Requirements

California's updated CCPA now treats AI-generated data as personal data, while other states are introducing their own regulations, creating a complex compliance landscape that varies by jurisdiction. This patchwork approach makes uniform governance challenging for multi-state operations.

The regulatory preparedness gap is alarming: 52% of business leaders admit uncertainty about navigating AI regulations, while only 18% of organizations have an enterprise-wide council authorized to make decisions on responsible AI governance.

What a Secure AI Deployment Should Look Like

Given these risks, how can businesses safely leverage AI technologies? The answer lies in implementing a comprehensive security framework specifically designed for AI usage:

1. Strong Governance and Security Policies

Any secure AI deployment begins with clear governance structures:

• Establish an AI governance council with representatives from IT, legal, compliance, and risk management. For smaller organizations, ensure the leadership team is involved in discussions.
• Develop codified security policies outlining acceptable AI use cases and security protocols. For example, develop a Responsible AI Use Policy for staff.
• Create role-specific AI training addressing unique departmental risks
• Implement approval workflows for sensitive AI applications

2. Enterprise-Grade AI Solutions

Rather than relying on public AI tools, organizations should consider:

• Deploying enterprise versions of AI platforms with enhanced security features (like OpenAI Enterprise or Microsoft Azure OpenAI)
• Exploring private AI deployments that keep data within your security perimeter
• Implementing AI-specific Data Loss Prevention (DLP) solutions designed to identify and block sensitive information before it reaches external AI systems
• Using AI governance platforms that provide visibility and control over all AI interactions

3. Comprehensive Technical Safeguards

Technical controls that should be standard for any AI deployment include:

• Zero Trust architecture requiring strict verification for all AI interactions
• Multi-factor authentication for all AI tool access
• Network monitoring for unusual AI-related behaviors
• Content filtering to prevent sensitive data sharing
• Automated prompt scanning to detect potential security risks

4. Clear Data Handling Policies

Establish strict guidelines for how data interacts with AI tools:

• Never share customer data through public AI tools
• Use anonymized examples or fictional scenarios instead of real data
• Implement approval processes for AI use in sensitive contexts
• Define clear consequences for AI policy violations
• Create data classification schemes that explicitly address AI usage scenarios
• Never use AI for critical business decisions around HR, legal, or finance

5. Continuous Monitoring and Assessment

Security isn't a one-time implementation but an ongoing process:

• Conduct regular AI risk assessments aligned with frameworks like NIST AI RMF
• Implement behavioral analytics to detect unusual AI usage patterns
• Establish incident response plans specifically for AI-related breaches
• Perform regular security audits of AI systems and usage
• Ensure employees are all trained on AI policies and proper usage of generative AI tools

The Future of Secure AI in Business

As AI continues its rapid evolution, the security challenges will only grow more complex. The distinction between organizations that thrive with AI and those that fall victim to its risks will increasingly depend on how seriously they take AI security from the outset.

For business leaders, the path forward is clear: recognize the substantial risks that public AI tools present, develop comprehensive governance frameworks, invest in secure AI infrastructures, and create a culture of AI security awareness throughout your organization. In doing so, you can harness the transformative power of artificial intelligence while protecting your most valuable assets — your data, your reputation, and your customers' trust.

The question isn't whether your organization will use AI — it's whether you'll use it securely before a preventable incident forces your hand.

Building Responsible AI at Unio Digital

The rapid growth of generative AI brings promising new innovations and raises new challenges. At Unió Digital, we are committed to developing AI responsibly, taking a people-centric approach that prioritizes security and confidentiality to integrate responsible AI for day-to-day operations.