Why AI’s Confident Mistakes Matter at Work

AI tools are now part of many employees’ workdays, helping with tasks such as drafting emails, summarizing content and preparing reports. But as organizations rely on them more, the risks of misuse and misplaced trust grow.

In 2023, US lawyers were sanctioned after filing legal documents with false citations invented by ChatGPT. The example highlights the danger of acting on AI-generated information without checking primary sources. Hallucinations are only one concern: employees may also enter sensitive information into unapproved tools, creating security, privacy and compliance risks.

The EU AI Act’s transparency requirements apply from 2 August 2026 to organizations within scope. The source says these include transparency about business interactions with AI and AI-generated or manipulated content, as well as appropriate AI literacy for staff. It reports potential fines of up to €15 million or 3% of an organization’s total annual turnover for breaches of the transparency requirements.

MetaCompliance’s chief technology officer argues that verification should reflect the stakes. Light spot-checks may be enough for low-risk tasks such as internal drafting or brainstorming. Outputs used in employment or financial decisions, or in customer-facing content, need stricter approval and monitoring, with a human involved.

The source also cites research in which 40% of CISOs fear staff are sharing sensitive information with generative AI platforms, while 68% identify employees as their organizations’ biggest cyber risk. It recommends mapping unapproved AI use, providing secure alternatives, and giving employees clear rules, training and verification routines.