DirectMemory

Abstract

The following proposal is about Apache DirectMemory, a Java OpenSource multi-layered cache implementation featuring off-heap memory storage (a-la Terracotta BigMemory) to enable caching of Java objects without degrading JVM performance

Proposal

DirectMemory's main purpose is to to act as a second level cache (after a heap based one) able to store large amounts of data without filling up the Java heap and thus avoiding long garbage collection cycles. Although serialization has a runtime cost store/retrieve operations are in the sub-millisecond range being pretty acceptable in every usage scenario even as a first level cache and, most of all, outperforms heap storage when the count of the entries goes over a certain amount. DirectMemory implements cache eviction based on a simple LFU (Least Frequently Used) algorythm and also on item expiration. Included in the box is a small set of utility classes to easily handle off-heap memory buffers.

Background

DirectMemory is a project was born in the 2010 thanks to Raffaele P. Guidi initial effort under GitHub and already licensed under the Apache License 2.0.

Rationale

The rationale behind DirectMemory is bringing off-heap caching to the open source world, empowering FOSS developers and products with a tool that enables breaking the heap barrier and override the JVM garbage collection mechanism collection - which could be useful in scenarios where RAM needs are over the usual limits (more than 8, 12, 24gb) and to ease usage of off-heap memory in general

Current Status

Meritocracy

As a majority of the initial project members are existing ASF committers, we recognize the desirability of running the project as a meritocracy. We are eager to engage other members of the community and operate to the standard of meritocracy that Apache emphasizes; we believe this is the most effective method of growing our community and enabling widespread adoption.

Core Developers

In alphabetical order:

  • Christian Grobmeier <grobmeier at apache dot org>
  • Maurizio Cucchiara <mcucchiara at apache dot org>
  • Olivier Lamy <olamy at apache dot org>
  • Raffaele P. Guidi <raffaele dot p dot guidi at gmail dot com>
  • Simone Gianni <simoneg at apache dot org>
  • Simone Tripodi <simonetripodi at apache dot org>
  • Tommaso Teofili <tommaso at apache dot org>

Alignment

The purpose of the project is to develop and maintain DirectMemory implementation that can be used by other Apache projects.

Known Risks

Orphaned Products

DirectMemory does not have any reported production usage, yet, but is getting traction with developers and being evaluated by potential users and thus the risks of it being orphaned are minimal

Inexperience with Open Source

All of the committers have experience working in one or more open source projects inside and outside ASF.

Homogeneous Developers

The list of initial committers are geographically distributed across the Europe with no one company being associated with a majority of the developers. Many of these initial developers are experienced Apache committers already and all are experienced with working in distributed development communities.

Reliance on Salaried Developers

To the best of our knowledge, none of the initial committers are being paid to develop code for this project.

Relationships with Other Apache Products

DirectMemory fits naturally in the ASF because it could be successfully employed together with a large number of ASF products ranging from JCS - as a new cache region between the heap and indexed file ones, to ORM systems like Cayenne (i.e. replacing current OSCache based implementation), Apache JDO and JPA implementations and also java based databases (i.e. Derby) and all systems managing large amounts of data from Hadoop to Cassandra

A Excessive Fascination with the Apache Brand

While the Apache Software Foundation would be a good home for the DirectMemory project it already has some traction and it could live on its own - however we see reciprocal benefits for both the ASF and the project in adopting the brand to better reach the community

Documentation

  1. The original DirectMemory HomePage 2. The original DirectMemory Wiki 3. The DirectMemory Issue Tracker (used also for enhancements, feature requests and discussion)

Initial Source

The intial source comprises code developed on GitHub contributed under Grant from Raffaele P. Guidi for Directmemory.

Source and Intellectual Property Submission Plan

Source code will be moved from GitHub space inside the SVN space of the podling.

External Dependencies

Build/Test time dependencies

Cryptography

The project does not handle cryptography in any way.

Required Resources

Initial Committers

Names of initial committers - in alphabetical order - with current ASF status:

  • Christian Grobmeier <grobmeier at apache dot org> (ASF Member)
  • Ioannis Canellos <iocanel at apache dor org> (ASF Committer)
  • Maurizio Cucchiara <mcucchiara at apache dot org> (ASF Committer)
  • Olivier Lamy <olamy at apache dot org> (ASF Member)
  • Raffaele P. Guidi <raffaele dot p dot guidi at gmail dot com> (ICLA Signed)
  • Simone Gianni <simoneg at apache dot org> (ASF Member)
  • Simone Tripodi <simonetripodi at apache dot org> (ASF Member)
  • Tommaso Teofili <tommaso at apache dot org> (ASF Member)

Sponsors

Champion

  • Christian Grobmeier <grobmeier at apache dot org>

Nominated Mentors

  • Anthony Elder <antelder at apache dor org>
  • Christian Grobmeier <grobmeier at apache dot org>
  • Olivier Lamy <olamy at apache dot org>
  • Sylvain Wallez <sylvain at apache dot org>
  • Tim Williams <twilliams at apache dot org>

Sponsoring Entity

  • The Apache Incubator

Other interested people (in alphabetical order)

  • TBD
  • No labels