This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Current state: Accepted
Discussion thread: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
The main goal is supporting interactive queries in presence of versioned state stores (KIP-889) in AK. This KIP is the successor of KIP-960 and discusses single-key, multi-timestamp queries. Other types of IQs are explained in the following KIP (KIP-969)
Key Queries with multiple timestamps:
- single-key query with upper bound timestamp
- single-key query with lower bound timestamp
- single-key query with timestamp range
- single-key all versions query
In this KIP,
- we propose a public class, MultiVersionedKeyQuery
- and a public enum ResultOrder
- Moreover, the public interface VersionedRecordIterator is added to iterate over different values that are returned from a single-key query (each value corresponds to a timestamp).
- In addition, a new method is added to the VersionedKeyValueStore interface to support single-key_multi-timestamp queries.
- Finally, a field called validTo is added to the VersionedRecord class to enable us representing tombstones as well.
To be able to list the tombstones, the validTo Optional field is added to the VersionedRecord class. The default value of validTo is Optional.empty() which means the record is still valid.
For single-key queries, MultiVersionedKeyQuery and VersionedRecordIterator classes will be used.
It helps with specifying the order of the returned results by the query.
- The methods are composable. The fromTime(Instant fromTime) and toTime(Instant toTime) methods specify the time range.
If a user applies the same time limit multiple times such as MultiVersionedKeyQuery.withKey(k).fromTime(t1).fromTime(t2), then the last one wins (it will be translated to MultiVersionedKeyQuery.withKey(k).fromTime(t2)).
- Defining a query with time range (empty, t1] will be translated into [0, t1] (calling only the toTime(t1) method).
- Defining a query with time range [t1, empty) will be translated into [t1, MAX) (calling only the fromTime(t1) method).
- A query with no specified time range will be translated into [0, MAX). It means that the query will return all the versions of the records with specified key.
- As explained in the javadocs, the query returns all valid records within the specified time range.
- The fromTime specifies the starting point. There can be records which have been inserted before the fromTime and are valid in the time range. For example, if the record (k,v) has been inserted at time=0, it will be returned by the multi versioned key queries with key=k and fromTime>=0. Obviously, if the record (k,v) becomes tombstone at time=2, then the multi versioned key queries with key=k and fromTime>=2 will not return it any more. In this case, the multi versioned key queries with key=k and fromTime<2 will return the record (k,v) validTo=2.
- The toTime specifies the ending point. Records that have been inserted at toTime are returned by the query as well.
- No ordering is guaranteed for the results, but the results can be sorted by timestamp (in ascending or descending order) by calling the corresponding defined methods (withAscendingTimestamps() and withDescendingTimestamps() respectively).
The following example illustrates the use of the VersionedKeyQuery class to query a versioned state store.
Imagine we have the following records
put(1, 1, time=2023-01-01T10:00:00.00Z)
put(1, null, time=2023-01-05T10:00:00.00Z)
put(1, null, time=2023-01-10T10:00:00.00Z)
put(1, 2, time=2023-01-15T10:00:00.00Z)
put(1, 3, time=2023-01-20T10:00:00.00Z)
Compatibility, Deprecation, and Migration Plan
- Since this is a completely new set of APIs, no backward compatibility concerns are anticipated.
- Since nothing is deprecated in this KIP, users have no need to migrate unless they want to.
In order to be able to retrieve the consecutive tombstones, we can have a method or flag (disabled by default) to allow users to get all tombstones. If it is a real use case for the users, we will add it later.
The single-key_multi-timestamp interactive queries will be tested in versioned stored IQv2 integration test (like non-versioned key queries). Moreover , there will be unit tests where ever needed.