To be Reviewed By: Monday, 2020-04-13
Authors: Blake Bender
Status: Draft | Discussion | Development | Active | Dropped | Superseded
Superseded by: N/A
Related: N/A
Problem
The existing native client API is for C++ and C# only. This is acceptable for writing client applications in C++, but presents substantial difficulties when attempting to bind to the API from any other language. The current native client has language bindings for C# via the Microsoft "IJW" language, a.k.a. C++/CLI, which Microsoft has subsequently abandoned, more-or-less. The .net framework has also subsequently been deprecated in favor of .net core, which has even less support for IJW than the original framework. The preferred way to interface with C/C++ libraries for .net core, along with many, many other languages, is via C bindings. I propose to add these to the native client, starting with the most basic functionality, and working towards eventually supporting the entire existing API.
Anti-Goals
This proposal is strictly limited to adding C bindings to the native client. We do not wish to make any specific proposals for supporting Geode access via any other languages (.net core, Python, etc) here.
Solution
A simple C binding for a native client library function would look something like this:
struct CacheFactory; typedef struct CacheFactory CacheFactory; APACHE_GEODE_EXPORT CacheFactory* CreateCacheFactory(); APACHE_GEODE_EXPORT DestroyCacheFactory(CacheFactory* factory);
And be implemented something like this:
CacheFactory* CreateCacheFactory() { CacheFactoryWrapper* cacheFactory = new CacheFactoryWrapper(); return reinterpret_cast<CacheFactory*>(cacheFactory); } void destroyCacheFactory(CacheFactory* cacheFactory) { delete reinterpret_cast<CacheFactoryWrapper*>(cacheFactory); }
Binding to this function and calling it is trivial for most other languages. Here's an example for .net core:
[DllImport(Constants.libPath, CharSet = CharSet.Auto)] private static extern IntPtr CreateCacheFactory(); var factory = CreateCacheFactory(); // Do stuff with factory DestroyCacheFactory(factory);
And from Python:
from ctypes import cdll, c_void_p nativeclient_ = cdll.LoadLibrary( "/Users/build/Workspace/native_client_install/libpivotal-gemfire.dylib" ) nativeclient_.CreateCacheFactory.restype = c_void_p nativeclient_.DestroyCacheFactory.argtypes = [c_void_p] factory_ = self.nativeclient_.CreateCacheFactory() # Do stuff with factory self.nativeclient_.DestroyCacheFactory(factory_)
Other languages follow a similar pattern.
Changes and Additions to Public Interfaces
General
We propose to add C bindings of a form similar to the above to the public API for the native client. The new API would need to be fully documented, but we believe implementation can start with coverage of a very small subset of the existing C++ API and still be extremely useful.
Organization of the Code
- The C bindings must reside in a shared library or DLL, as required by the method(s) used by various languages to interact with C code. Our initial plan to organize the code is as follows:
i. Code for C bindings shall reside in the existing geode-native repository
ii. The code for C bindings will be organized under its own top-level directory, alongside the existing cppcache and clicache directories, with separate include directory for public headers and src directory for the implementation.
iii. The C bindings will consume the geode-native classes as a static library, rather than importing them from a shared library.
iv. Only C bindings will be exported from the new shared library, i.e. geode-native C++ classes will be hidden
Naming Convention
- We will be adopting a specific naming convention for the C bindings to attempt to ensure unique names, since we can't take advantage of namespaces. Tentatively, we will prefix function names with `apache_geode_`, followed by 'Create' for methods that allocate a 'class instance', i.e. an opaque context pointer, and 'Destroy' for the dealloc/free function. For class methods, we will use 'apache_geode' + class name + '_' + method name. As a trivial example, a partial CacheFactory implementation with just new/delete and the method getVersion would be exported via C functions as follows:
- Since C functions can't be namespaced, we will adopt a naming convention for the C functions that distinguishes them as best they can be from application code. This will most likely be a string prefix representing the namespace, e.g. "apache_geode_client_CreateCacheFactory".
apache_geode_CreateCacheFactory apache_geode_CacheFactory_getVersion apache_geode_DestroyCacheFactory
Serialization
- Serialization in Geode is, of course, a large and complex topic. We do not, at the time of this writing, know all of the minute details of the ultimate implementation of serialization in any of the potential client languages which will use our C bindings. Generally, however, we plan to implement a mechanism sufficient to enable a client implementation of a PDX type handler and other necessary classes that matches the current C++/CLI implementation in geode-native.
Performance Impact
Any performance impact of calling through a C binding should be very minimal. A C function exposing a native client method should only involve a pointer cast and a call to the "wrapped" C++ API. Additionally, there will be no performance impact whatsoever for existing applications, since we're not proposing to modify any of the existing C++ code. Eventually we should be able to benchmark the C vs C++ calls to quantify the performance difference, but this is a fairly low priority. Benchmarking API performance from another language calling through the C bindings is a much more interesting and necessary task, which we believe will be undertaken as part of any future support for said language.
Backwards Compatibility and Upgrade Path
Since this is entirely new functionality, and makes no modification to existing APIs, there is no backwards compatibility burden to be considered here.
Prior Art
C bindings allow a client written in C to access the native client directly, but primarily they are interesting because they allow easy access to the API from a much larger overall set of (non-C) languages. One other alternative to access Geode from another language is to provide a full implementation of the Geode wire protocol in that language. At the moment, this is a very large undertaking, because the Geode protocol is complex, and not fully documented outside of the existing implementations in Geode and the Java and native clients. Another alternative would be to define an entirely new protocol via some language or tooling that would allow us to generate bindings for different languages. Google's protobuf, for instance, provides this functionality. Unfortunately, this is an even taller order than either of the first two options, since it involves fairly extensive modification of the server code, in addition to the clients. We believe the simple C bindings proposed provide a workable solution for many languages and applications which can be completed in a reasonable time frame.
FAQ
Errata