Navigating AI Security in Response to the White House Executive Order

This week, the White House published a comprehensive executive order on artificial intelligence (AI), introducing new federal mandates designed to ensure the safe, reliable, and responsible deployment of AI technologies. The order is the most significant regulatory action taken by the U.S. federal government to date, building on earlier initiatives, like the Blueprint for an AI Bill of Rights and previous directives aimed at combatting algorithmic discrimination.

The executive order places a major focus on guiding federal agencies on how to regulate AI, setting clear expectations around AI safety, security, privacy, algorithmic discrimination, and the preservation of civil rights. These guidelines carry significant implications for the private sector, as organizations must now align with stringent standards to ensure their AI systems meet federal requirements.

President Biden has highlighted AI safety and security as foundational principles in this 111-page document. Vice President Harris reinforced this by announcing the creation of new AI Safety Institute within the Department of Commerce. This institute will develop standards for testing AI models and draft policy guidance for federal AI use.

At Dynamo AI, we are in active discussions with key policymakers from both the Legislative and Executive branches who are shaping the federal government's collective approach to AI regulation. We specialize in helping organizations navigate these new regulations, ensuring compliance with the heightened standards for AI safety and security.

In anticipation of this executive order, Dynamo AI tailors its large language model (LLM) platform to address emerging regulatory requirements around the world, including those set forth in this U.S. executive order and the EU AI Act. Our platform helps clients efficiently and safely productionize Generative AI (GenAI) solutions, while maintaining rigorous compliance documentation.

The executive order specifically addresses the risks associated with data leakage and data extraction, highlighting how AI can both exploit and enhance personal data collection:

“AI not only makes it easier to extract, identify, and exploit personal data, but it also heightens incentives to do so because companies use data to train AI systems."

Dynamo AI's cutting-edge, privacy-preserving techniques support end-to-end LLM training and an AI safety evaluation platform that integrates seamlessly with existing ML workflows. Our solutions have already helped some of the largest enterprises mitigate the risks outlined in the executive order through automated, ongoing penetration testing of LLMs to detect and prevent malicious data extraction.

As regulatory bodies continue to address the technical complexities of LLMs, as evidenced by the recent actions from the FTC, enterprises deploying AI services should expect regulators to have a strong grasp of these technologies and prepare accordingly.

At Dynamo AI, we proactively engage with policymakers and leverage our world-class team of AI experts to navigate emerging regulatory frameworks. Our team reviews and filters hundreds of new machine learning research papers to develop and refine regulation-compliant methods tailored to your specific AI use case and business needs. This approach ensures that your AI solutions not only meet current regulations but are also prepared for future developments.

Red-teaming and penetration testing: Ensuring AI security and compliance

The recent executive order emphasizes the need for guidelines on AI audits and red-teaming, particularly for AI systems that could potentially cause harm. This focus on red-teaming builds on prior frameworks, including the “external red teaming” language in the voluntary AI commitments declaration and independent testing highlighed in the NIST AI RMF and EU AI Act.

Red-teaming involves structured testing to identify flaws and vulnerabilities in AI systems, typically conducted in a controlled environment and in collaboration with AI developers.

 “Testing and evaluations will help ensure that AI systems function as intended, are resilient against misuse or dangerous modifications, are ethically developed and operated in a secure manner, and are compliant with applicable Federal laws and policies.”

With the rise of GenAI and LLMs, red-teaming is increasingly crucial for identifying potential compliance incidents, data leakage, and malicious attacks. These advanced models, which often train on larger and more sensitive datasets, introduce new attack vectors and vulnerabilities.

For instance, Dynamo AI's team of academic experts and privacy researchers recently discovered significant data and privacy flaws in GPT-3, a popular LLM from OpenAI. 

Since early 2023, Dynamo AI has been providing LLM penetration testing solutions to financial services institutions, hardware companies, and other major organizations deploying GenAI. Our evaluation suite rigorously stress-tests both open- and closed-source LLMs, focusing on critical risks such as sensitive data leakage and AI hallucinations, ensuring robust security and compliance.

Standardized safety and security documentation

Safety and security evaluations need to be accompanied by comprehensive and standardized reporting. Dynamo AI leads the industry in establishing what standardized AI safety reports should include. In response to the executive order, developers of dual-use foundational models are required to provide “a description of any associated measures the company has taken to meet safety objectives, such as mitigations to improve performance on these red-team tests and strengthen overall model security.”

Dynamo AI offers a suite of customizable, auto-generated AI model evaluation reports designed for seamless sharing with company leadership, regulators, and other key stakeholders.

Next steps with Dynamo AI 

At Dynamo AI, our mission is to enable Fortune 500 companies, federal agencies, and other large organizations to deploy state-of-the-art AI solutions securely and safely. We are dedicated to implementing effective risk mitigations that safeguard users, communities, and organizations. Contact us for more information on how we can help.