Apollo Research is becoming a PBC

TLDR: Apollo is spinning off from our fiscal sponsor into a Public Benefit Corporation (PBC). We think this is the best way for us to achieve our mission of reducing extreme risks from frontier AI systems. We are increasing our investment in research, evals and governance. We’re standing up a separate product team to build AGI safety products starting with AI agent monitoring. We introduced new corporate governance mechanisms to ensure we stay true to our mission well into the future.

Motivation for the PBC

Mission: We’ve seen a quickly growing market demand for our frontier AI safety research and evals work. We expect this demand to continue with increasing AI capabilities and risks, and that it will be hard to serve this demand at scale as a non-profit. Therefore, we think our mission of understanding and addressing extreme risks from frontier AI systems will be better fulfilled under a PBC model. 

Scale: The state of safety research and tooling is drastically behind what it needs to be to adequately address risks from frontier AI systems. We believe that an appropriate response requires a significantly larger scale, which is easier to achieve as a PBC.

Long-term vision: AI capabilities will continue to grow rapidly and AI systems will be integrated everywhere transforming the entire economy. We therefore think that frontier AI safety is going to be a large market. We actively intend to build out and shape this market. We’re particularly focused on making frontier AI systems safer for developers and users.

Details of the new organization

Frontier first approach: We will continue to work with frontier AI companies to evaluate their models and do joint research collaborations, and we’re actively growing our frontier safety research efforts. For the product, we’re prioritizing companies at the frontier of AI agent development or usage. Then, we will build out and adapt these tools for a broader market of consumers interested in improving the safety of their frontier AI agents. 

AGI safety products: We’re building AGI safety products, i.e. tools that increase the safety of frontier agents. We work on problems and solutions that we anticipate to transfer meaningfully to more capable, transformative AI systems. Right now, we are focusing on AI coding agent observability and control. If you’re interested in working with us to build better monitors or try early versions of our monitoring tools, please reach out. Marius, our CEO, has written more about AGI safety products here. 

Research independence: We think the most important AI safety improvements require substantial additional research. Therefore, we will continue to grow and double down on our research efforts. In parallel, we are building up a separate product arm of the organization that productizes the research. Our governance and some of our research efforts will continue to be partially supported by philanthropy.

Investment: We’re working with investors who understand and support our mission. To build out our product efforts, we have raised a Seed round led by 50Y with further investments from Juniper Ventures, Macroscopic Ventures, Common Metal, SAIF, Srikar Varadaraj, Sarah Meyohas, Ocean Investment, Progress Fund, and Bryan Johnson. The round was oversubscribed.  

Details of the transition

Mission board seats & other governance mechanisms: We strive to establish a robust governance structure, safeguarding our mission well into the future. As part of this effort, we have designated the 3rd and 6th (once the board expands to that size; currently 3 seats) seats of the board of directors as “mission seats”. These mission seats will be held by “mission directors” who are independent to Apollo Research PBC and our funders, and who have a special mandate that enables them to prioritize mission over other considerations. Our first mission director is Daniel Kokotajlo. We’re continuing our work to establish additional corporate governance mechanisms.

External IP valuation and transfer: We engaged a leading external IP valuation firm to assess the value of the IP we wanted to transfer and subsequently bought this IP at the assessed value, compensating the non-profit entity accordingly. 

Engagement with philanthropic foundations: Throughout our engagements with philanthropic foundations in the past, we remained transparent that  a transition to a PBC model might occur in the future. We have publicly contemplated this possibility since our public announcement in 2023.

Future of the non-profit entity: The precise details around the continuation of the non-profit  entity have not yet been solidified, regardless, the goal will be to spend the remaining funds in accordance with  the mission of the non-profit, for example, by supporting the building of the field of evaluations, the science of scheming and loss of control governance.