CAISI Assesses Risks in DeepSeek AI Models

The Center for AI Standards and Innovation (CAISI) at the National Institute of Standards and Technology (NIST) has evaluated AI models from China's DeepSeek, finding them inferior to U.S. models in performance, cost, and security.

Commerce Secretary Howard Lutnick stated that the evaluation, part of President Trump's AI Action Plan, highlights the dominance of American AI over foreign counterparts. The report underscores the risks of relying on foreign AI systems, emphasizing the importance of U.S. leadership in setting standards and driving innovation.

The CAISI evaluation points out that DeepSeek's models have security and censorship issues that could pose risks to developers, consumers, and national security. Despite these risks, DeepSeek remains a leading developer, contributing to the increased global use of Chinese models.

Experts from CAISI assessed three DeepSeek models and four U.S. models across 19 benchmarks. These benchmarks include both public and private evaluations developed in collaboration with academic and federal partners.

The evaluation aligns with the AI Action Plan's directive to assess the capabilities of U.S. and foreign AI systems, their adoption, and potential security vulnerabilities. CAISI plays a crucial role in facilitating testing and research to secure U.S. leadership in AI.

DeepSeek's performance lags behind U.S. reference models, particularly in software engineering and cybersecurity tasks. The models are more costly and vulnerable to hijacking and jailbreaking attacks, and they reflect more Chinese Communist Party narratives.