Xodus, an embedded database running in the JVM

Xodus is a high-performance, embedded database developed by JetBrains. The database is written in Java and Kotlin. Xodus operates entirely in-process, meaning it runs within your application’s JVM without requiring a separate server or complex setup. This also means that only one process can access the database at a time.

The database is licensed under the Apache 2.0 license, it is schema-less, transactional, full ACID compliance with snapshot isolation, and optimized for high performance. Xodus provides a low-level key-value store API, an object-oriented Entity Store API, and a Virtual File System (VFS) API for hierarchical file-like data storage.

The source code of Xodus is hosted on GitHub: https://github.com/JetBrains/xodus

In this blog post we will explore some of the key features of Xodus and how to use them in Java.

Xodus Architecture ¶

Schema-Less Design
Xodus doesn't enforce rigid schemas. You can store data dynamically, making it ideal for applications with evolving or unpredictable data structures.

Snapshot Isolation
Every transaction in Xodus operates at the snapshot isolation level. This ensures:

Consistency: Reads see a consistent snapshot of the database.
No Write Skew: Prevents anomalies in concurrent transactions.
Lock-Free Reads: Multiple readers can access data without blocking writers.

Append-Only Storage
Xodus uses an append-only storage mechanism:

New data is written without overwriting old data.
Reduces random I/O overhead (common in update-in-place databases).
Improves durability and crash recovery.

Garbage Collection
Old data versions are automatically cleaned up by a background garbage collector, ensuring storage efficiency without manual intervention.

Embedded and Serverless
Zero configuration: Add the dependency and start coding.
No network latency: Runs in the same JVM as your application.
Portable: Database files are cross-platform.

API Layers ¶

Xodus provides three API layers:

Environments
The foundation of Xodus. Manages transactions, storage, and garbage collection. Provides a key-value interface for raw data storage.

Entity Stores
Built on top of Environments.
Offers an object-oriented model with:

Entities: Objects with types (like "User" or "Order").
Properties: Named attributes (e.g., "name", "email").
Relations: Links between entities (e.g., a "User" owns multiple "Orders").

Virtual File Systems (VFS)
Built on top of Environments.
An API for storing hierarchical file-like data within the database.

Environment API ¶

The Environment API forms the core of Xodus, providing a low-level key-value store interface.

To work with the Environment API, you need to add the following dependency to your project:

    <dependency>
      <groupId>org.jetbrains.xodus</groupId>
      <artifactId>xodus-environment</artifactId>
      <version>2.0.1</version>
    </dependency>

pom.xml

Then you can create an Environment instance, which represents a database. As the first argument, you pass the path to the database directory.

Inside an environment, data is stored in Stores, which are similar to tables in relational databases. An environment can contain multiple stores, and each store holds multiple key-value pairs.

The Environment API can only store ByteIterable instances, a Xodus class that mixes byte iterable and array. For convenience the library provides bindings to convert Java types to ByteIterable and vice versa. For example: BooleanBinding, IntegerBinding, FloatBinding, StringBinding and many more.

    try (Environment env = Environments.newInstance("./data")) {
      // insert
      env.executeInTransaction(txn -> {
        Store store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES, txn);
        store.put(txn, StringBinding.stringToEntry("user1"),
            StringBinding.stringToEntry("Alice"));
      });

      // update
      env.executeInTransaction(txn -> {
        Store store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES, txn);
        store.put(txn, StringBinding.stringToEntry("user1"),
            StringBinding.stringToEntry("Alice Smith"));
      });

      // read
      env.executeInReadonlyTransaction(txn -> {
        Store store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES, txn);
        ByteIterable value = store.get(txn, StringBinding.stringToEntry("user1"));
        System.out.println("User1: " + StringBinding.entryToString(value));
      });

      // delete
      env.executeInTransaction(txn -> {
        Store store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES, txn);
        store.delete(txn, StringBinding.stringToEntry("user1"));
      });

EnvironmentDemo.java

An application can also pass an EnvironmentConfig instance when opening an environment. With this config you can control various aspects of the environment.

Here is an example that opens the environment in read-only mode:

  File dbDir = new File("./data");
  EnvironmentConfig config = new EnvironmentConfig();
  config.setEnvIsReadonly(true);
  try (Environment env = Environments.newInstance(dbDir, config)) {
    // ...
  }

It is important to close the environment when you are done with it, either with the close method or by using a try-with-resources block.

Transactions ¶

Every database operation must run within a transaction. Xodus supports both read-only and read-write transactions. An application can start multiple transactions concurrently. If you want to make sure that no other transaction can modify the data you are working with, you can use an exclusive transaction. Read-only transactions are still allowed when an exclusive transaction is running. Xodus throws an exception if an application tries to modify data in a read-only transaction.

The Environment API also provides computeIn* methods that run a block of code in a transaction and return a result.

  try (Environment env = Environments.newInstance("./data")) {
     String result = env.computeInReadonlyTransaction(txn -> {
     Store store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES, txn);
       var value = store.get(txn, StringBinding.stringToEntry("user1"));
       if (value != null) {
         return StringBinding.entryToString(value);
       } else {
         return null;
       }
    });
  }

Another way to handle the transaction is to start one with a env.begin* method and then commit or abort it manually. Changes made in the current transaction, can be undone by calling the revert method on the transaction instance.

This example starts a readonly transaction with beginReadonlyTransaction and shows you how to check if a store exists and list all stores in an environment:

    try (Environment env = Environments.newInstance("./data")) {
      Transaction txn = env.beginReadonlyTransaction();
      try {
        // Check if store exists
        boolean storeExists = env.storeExists("Users", txn);
        System.out.println("Store 'Users' exists: " + storeExists);

        // List all stores
        List<String> storeNames = env.getAllStoreNames(txn);
        System.out.println("Stores in the environment:");
        for (String name : storeNames) {
          System.out.println("- " + name);
        }
      }
      finally {
        txn.abort(); // Readonly transaction does not need to be committed
      }
    }

EnvironmentDemo.java

Another way to handle transactions is to use a ContextualEnvironment. This kind of environment internally uses a ThreadLocal variable to keep track of the current transaction. This simplifies the code because you do not have to pass the transaction instance into every method call. You still need to start and commit transactions either with env.begin* or run the code in a env.executeIn* or env.computeIn* block.

    try (ContextualEnvironment env = Environments.newContextualInstance("./data")) {
      try {
        env.beginTransaction();
        ContextualStore store = env.openStore("Users", StoreConfig.WITHOUT_DUPLICATES);
        store.put(StringBinding.stringToEntry("user1"),
            StringBinding.stringToEntry("Alice"));
      }
      finally {
        env.getCurrentTransaction().commit();
      }
    }

EnvironmentDemo.java

Store config ¶

In the examples above you see that the code passes StoreConfig.WITHOUT_DUPLICATES as argument when opening a store. This specifies that this store does not allow duplicate keys. Xodus supports several configurations for stores, which control how keys are handled regarding duplicates and prefixing.

Here is a summary of the available store configurations:

WITHOUT_DUPLICATES** ¶

Description:
The store does not allow duplicate keys, and key prefixing is not supported.

Behavior:
Each key must be unique. Attempting to insert a duplicate key will either overwrite the existing entry or fail, depending on the operation.

Use Case:
Ideal for simple key-value stores where keys are inherently unique (e.g., user IDs, session tokens). When you don't need prefix-based queries (e.g., finding all keys starting with "user_").

WITH_DUPLICATES ¶

Description:
The store allows duplicate keys, and key prefixing is not supported.

Behavior:
Multiple entries can share the same key. For example, you could store several values for the key "order_123". Retrieving a key returns all associated values (or an iterator over them).

Use Case:
Useful for logging or time-series data where multiple entries might share the same key (e.g., timestamps for the same event). When you don’t need prefix-based queries but require flexibility in key-value mapping.

WITHOUT_DUPLICATES_WITH_PREFIXING ¶

Description:
The store does not allow duplicate keys, but key prefixing is supported.

Behavior:
Keys must be unique, but the store optimizes for prefix-based operations. Efficiently supports range queries (e.g., find all keys starting with "user_").

Use Case:
Hierarchical or categorized data (e.g., "user:1:name", "user:1:email"). When you need to perform range queries or group keys by a common prefix (e.g., retrieving all keys for a specific user).

WITH_DUPLICATES_WITH_PREFIXING ¶

Description:
The store allows duplicate keys, and key prefixing is supported.

Behavior:
Combines the flexibility of duplicate keys with prefix-based optimizations. Supports both multiple values per key and efficient prefix queries.

Use Case:
Complex scenarios like multi-tenant systems where keys might be duplicated across tenants (e.g., "tenant_A:order_123", "tenant_B:order_123"), but you still need prefix-based access (e.g., all orders for "tenant_A"). Logging or event systems with duplicate keys and hierarchical organization.

USE_EXISTING ¶

Description:
Opens an existing store without needing to know its original configuration (duplicates or prefixing).

Behavior:
The store must already exist; attempting to open a non-existent store will fail. The runtime configuration (duplicates/prefixing) is inferred from the store's metadata. No new store is created.

Use Case:
When accessing a store created earlier, but you don't want to (or can't) specify its original settings. Useful in scenarios where the store's configuration is managed externally or dynamically.

Store Operations ¶

The store provides a set of operations to store data in a store, update it, retrieve it, and delete it. Depending on the store configuration, the behavior of operations like put, get, and delete varies:

put
Puts a key/value pair into the store and returns a boolean as result.

For stores with key duplicates, it returns true if the pair didn't exist in the store. It inserts the pair if the key doesn't exist or the key exists, but all the values associated with the key are different from the new value.

For stores without key duplicates, it returns true if the key didn't exist or the new value differs from the existing one. It overwrites the existing value if the key already exists.

putRight
This method is very similar to the put method. It can be used if it is certain that the key is definitely greater than any other key in the store. In that case, no search is being done before insertion, so putRight can perform several times faster than put. For example, you use a timestamp as a key, and you know that the application never inserts the same timestamp twice.

add
Adds a key/value pair to the store if the key doesn't exist. This method behaves the same with and without key duplicates. It returns true if the key doesn't exist, meaning it adds the pair to the store. If the key already exists, it does nothing and returns false. So even in a store that allows duplicates, this method will not add a new value for an existing key.

get
This method retrieves a value by a key. For stores without key duplicates, it returns the value associated with the key if the key exists, otherwise it returns null.
For stores with key duplicates, it returns the smallest not-null value associated with the key or null if the key doesn't exist. This means that if there are multiple values for a key, get will return the first one in the natural order of keys.

exists
This method checks if a key/value pair exists in the store. It returns true if the key exists in the store, regardless of whether the store allows duplicates or not. If the key does not exist, it returns false.

count
This method returns the total number of key/value pairs in the store. For stores without key duplicates, it returns the number of unique keys. For stores with key duplicates, it returns the total number of key/value pairs, including duplicates.

delete
This method deletes a key/value pair by a key. For stores without key duplicates, it deletes a single key/value pair and returns true if the pair was deleted. If the key does not exist, it returns false.
For stores with key duplicates, it deletes all pairs with the given key and returns true if any were deleted. If no pairs were deleted, it returns false. You cannot delete a particular key/value pair with this method, you have to use a cursor for this purpose.

openCursor
This method returns a Cursor instance that allows an application to access key/value pairs in the store in both ascending and descending order and to perform random access operations.

Cursor ¶

To open a cursor, an application calls the openCursor method on a store instance. The cursor can then be used to iterate over key/value pairs in the store. It's important to close the cursor after use to release resources. The cursor has access to the current key and value, and it can move through the store in both directions. In this example the cursor moves forward with getNext and retrieves the current key and value:

try (Cursor cursor = store.openCursor(txn)) {
   while (cursor.getNext()) {
     cursor.getKey();   // current key
     cursor.getValue(); // current value
   }
}

A newly opened cursor always points to a "virtual" key/value pair before the first (leftmost) pair in the store. The application then needs to move the cursor first before reading the current key and value. A cursor can be moved to the first key/value pair with getNext, to the last pair with getLast, or to a specific key with getSearchKey. Continuously calling getNext will move the cursor to the next key/value pair in ascending order, while getPrev will move it to the previous pair in descending order.

For stores that allow key duplicates, additional methods are available to iterate over all values for a specific key. getNextDup moves the cursor to the next key/value pair with the same key, while getNextNoDup moves it to the next pair with a different key. The same applies for the previous direction with getPrevDup and getPrevNoDup.

All get* methods return a boolean indicating whether the operation was successful or not. If there are no more pairs in the store, the cursor will return false.

A cursor can also be used to search for a specific key or key/value pair.

The getSearchKey method moves the cursor to the first (leftmost) value associated with the specified key. If the key exists, it returns the value associated with that key.
In stores that allow key duplicates, the cursor will be moved to the first (leftmost) key/value pair for the specified key, and it returns the associated value. To list all values for a key, you can use the getNextDup method after calling getSearchKey. getSearchKey returns null if it cannot find the specified key.

The getSearchKeyRange method moves the cursor to the first pair whose key is equal to or greater than the specified key. It returns the value of this pair if it exists, or null if nothing is found.

To find a pair with a specific key and value, you can use the getSearchBoth method. This method moves the cursor to the key/value pair with the specified key and value, returning true if the pair exists, otherwise false.

The getSearchBothRange method is similar to getSearchKeyRange, but it searches for a key/value pair where the key matches the specified key and the value is equal to or greater than the specified value. It returns the value of this pair if it exists, or null if nothing is found.

All the search methods do not move the cursor if they cannot find the specified key or key/value pair.

The cursor can also delete entries with the deleteCurrent method. This method deletes the key/value pair that the cursor is currently positioned on and returns true. If the cursor is not positioned on a valid key/value pair, it returns false.

Examples ¶

Here is an example with two stores, one that allows duplicates and one that does not. We see in this example that calling get on a store that allows duplicates only returns the first value. To read all values for a key, we need to use a cursor. The code moves the cursor with getSearchKey to the first value for the key and then iterates over all values with getNextDup.

    File dbDir = new File("./demo2");
    try (Environment env = Environments.newInstance(dbDir)) {
      Transaction rwTxn = env.beginTransaction();
      Store userStore = env.openStore("User", StoreConfig.WITHOUT_DUPLICATES, rwTxn);
      Store permissionsStore = env.openStore("Permissions", StoreConfig.WITH_DUPLICATES,
          rwTxn);

      userStore.put(rwTxn, LongBinding.longToEntry(1),
          StringBinding.stringToEntry("admin"));
      userStore.put(rwTxn, LongBinding.longToEntry(2),
          StringBinding.stringToEntry("user"));

      permissionsStore.put(rwTxn, LongBinding.longToEntry(1),
          StringBinding.stringToEntry("read"));
      permissionsStore.put(rwTxn, LongBinding.longToEntry(1),
          StringBinding.stringToEntry("write"));
      permissionsStore.put(rwTxn, LongBinding.longToEntry(2),
          StringBinding.stringToEntry("read"));
      rwTxn.commit();

      env.executeInReadonlyTransaction(txn -> {

        String adminUser = StringBinding
            .entryToString(userStore.get(txn, LongBinding.longToEntry(1)));
        System.out.println(adminUser); // admin

        String permission = StringBinding
            .entryToString(permissionsStore.get(txn, LongBinding.longToEntry(1)));
        System.out.println(permission); // read

        try (Cursor cursor = permissionsStore.openCursor(txn)) {
          ByteIterable bi = cursor.getSearchKey(LongBinding.longToEntry(1));
          if (bi != null) {
            System.out.println(StringBinding.entryToString(bi));
            while (cursor.getNextDup()) {
              bi = cursor.getValue();
              System.out.println(StringBinding.entryToString(bi)); // first read then
                                                                   // write
            }
          }
        }

      });

    }

EnvironmentDuplicateDemo.java

Here is an example with key prefixing. The example inserts sensor data with prefixes "sensor1:" and "sensor2:". It then uses a cursor to retrieve all values for the prefix "sensor1:". The getSearchKeyRange method is used to find the first key that matches the prefix, and then it iterates over all matching keys. Note that the cursor method getNext does not stop when it reaches a different prefix, this method will continue until it reaches the end of the store. Therefore, this code checks if the key no longer starts with the specific prefix.

    String sensor1Prefix = "sensor1:";
    String sensor2Prefix = "sensor2:";

    File dbDir = new File("./demo3");
    try (Environment env = Environments.newInstance(dbDir)) {
      Transaction rwTxn = env.beginTransaction();
      try {
        Store sensorStore = env.openStore("Sensor",
            StoreConfig.WITHOUT_DUPLICATES_WITH_PREFIXING, rwTxn);

        for (int i = 0; i < 10; i++) {
          sensorStore.put(rwTxn, StringBinding.stringToEntry(sensor1Prefix + i),
              IntegerBinding.intToEntry((int) (Math.random() * 100)));
          sensorStore.put(rwTxn, StringBinding.stringToEntry(sensor2Prefix + i),
              IntegerBinding.intToEntry((int) (Math.random() * 100)));
        }
      }
      finally {
        rwTxn.commit();
      }

      env.executeInReadonlyTransaction(txn -> {
        Store sensorStore = env.openStore("Sensor", StoreConfig.USE_EXISTING, txn);

        try (Cursor cursor = sensorStore.openCursor(txn)) {
          ByteIterable value = cursor
              .getSearchKeyRange(StringBinding.stringToEntry(sensor1Prefix));
          if (value != null) {
            System.out.println(StringBinding.entryToString(cursor.getKey()));
            System.out.println(IntegerBinding.entryToInt(value));
            while (cursor.getNext()) {
              String key = StringBinding.entryToString(cursor.getKey());
              if (!key.startsWith(sensor1Prefix)) {
                break; // stop if we reach a different prefix
              }
              value = cursor.getValue();
              System.out.println(key);
              System.out.println(IntegerBinding.entryToInt(value));
            }
          }
        }

      });

    }

EnvironmentPrefixingDemo.java

Entity Stores ¶

Entity Stores build on top of Environments, providing a higher-level, object-oriented API. You can define entities with properties and links, making it easy to model complex relationships.

The programming model looks similar to the Environment API, but instead of key-value pairs, you work with entities that have types and properties.

To use the Entity Store API, you need to add the following dependency to your project:

    <dependency>
      <groupId>org.jetbrains.xodus</groupId>
      <artifactId>xodus-entity-store</artifactId>
      <version>2.0.1</version>
    </dependency>

pom.xml

The main entry point of the Entity Store API is the PersistentEntityStore class. An application instantiates this class by passing the path to the database directory. It is important to close the store to release resources either with calling the close method or by using a try-with-resources block.

    File dbDir = new File("my_entity_store_db");
    try (PersistentEntityStore entityStore = PersistentEntityStores.newInstance(dbDir)) {

Xodus Architecture ¶

API Layers ¶

Environment API ¶

Transactions ¶

Store config ¶

WITHOUT_DUPLICATES** ¶

WITH_DUPLICATES ¶

WITHOUT_DUPLICATES_WITH_PREFIXING ¶

WITH_DUPLICATES_WITH_PREFIXING ¶

USE_EXISTING ¶

Store Operations ¶

Cursor ¶

Examples ¶

Entity Stores ¶

Store ¶

Sequences ¶

Query ¶

Retrieve all entities ¶

Sort ¶

Count ¶

Find ¶

Link Traversal ¶

Virtual File Systems ¶

Advanced Concepts ¶

Encryption ¶

Backups ¶

Conclusion ¶