On September 11, 2026, OpenAI announced Habitat, its proprietary online storage platform for ChatGPT, the OpenAI API, Codex, and internal services. It provides fast and reliable access to data required for tasks such as logging in, loading Codex settings, and launching chats in ChatGPT.
Habitat processes over 70 million requests per second and supports products used by more than 1 billion people weekly. The platform operates in nearly 40 regions and stores over 500 PB of data.
The system provides a unified access layer for Azure Cosmos DB, Valkey caches, Blob Storage, Nanobase, and other storage solutions. Habitat handles routing, caching, authorization, encryption, tenant isolation, data placement, and load limiting for underlying components.
The project began in 2023 as a Python client library connected to Azure Cosmos DB. OpenAI later transformed it into a centralized service. This simplified deployment, system monitoring, auditing, and the enforcement of security policies.
Habitat uses a restricted NoSQL API based on objects and relationships rather than arbitrary SQL queries. This approach makes the workload more predictable and reduces the risk of resource-intensive queries overloading the online store. For complex queries, OpenAI uses a separate offline data representation in Rockset. Updates are propagated there via CDC in near real-time, isolating analytical and search workloads from the online store.
When scaling Habitat, OpenAI addressed asyncio scheduler latency and connection imbalances in Python's aiohttp, while also mitigating the risk of overloading underlying systems. To achieve this, they employed FIFO connection reuse, Envoy, HTTP/2, multiplexing, connection pools, limits, and circuit breakers.
At its peak, the Python version of Habitat handled over 20 million requests per second. It became OpenAI’s second-largest Envoy service by CPU core count and fourth-largest by scale.
In the second quarter of 2026, two engineers, assisted by Codex and GPT-5.5, rewrote Habitat from Python to Rust. At the time of publication, the Rust version was handling 95% of production requests and utilizing CPU and memory six times and 15 times more efficiently, respectively. It also delivered lower average and tail latencies. OpenAI planned to completely phase out the Python version in the coming weeks.
Habitat remains an internal infrastructure platform at OpenAI. The company has not indicated that it is available as a standalone public product.
Source: OpenAI
0 Comments