This site is obsolete.  The Apache Drill documentation has moved to the Apache Drill site: http://drill.apache.org/docs/  
Skip to end of metadata
Go to start of metadata

Project Name: Apache Drill

Q3 Summary in progress

(scroll down for previous summaries)

Issues:

Discussions on these key areas were _very focused_ and productive toward completion of goals for graduation:
1.) Develop momentum for community building
2.) Work toward first milestone release

How has the community developed since the last report?

Apache Drill community for this developing project has been very active during Q3. The most exciting event is the process of getting approval for the 1st milestone release. Developers worked to meet specific goals for the benchmark. The community was active in reviewing and voting on the proposal (both binding and non-binding voters participated). And the result is that the 1st milestone is approved for release late in September.

In addition, the user side of the community is beginning to be active. There is beginning to be activity on the user mail list, people have tried Drill in a pre-release hands-on workshop, and a new contributor has joined the project to begin testing queries and offering feedback.

Q3 events include:

  • Vote for 1st milestone released resulted in approval.
  • Subscription to the mailing list has grown to >383.

Activity has mainly been in the form of developing code, with some new engineers participating. Activity by users for early versions of Drill has also begun. Participation in the user mailing list is growing and of course the main progress is on the developer list.

Discussion on developer mailing list include discussion of next steps in REST API; Maven build checkstyle; many JIRAs created, some commented, some resolved (such as 156 for union POP implementation, 166 to generalize code generation and function holder)

Selection of Presentations, Articles and Outreach:

  • Standing event: Drill Hangout Tuesday mornings 9am Pacific Time at Google Hangout
  • Article in Big Data "Apache Drill: Interactive Query at Scale" by @mhausenblas & @intjesus http://bit.ly/14uhGJS
  • Apache Drill Hands-On Workshop in July at OSCON in Portland by @ted_dunning and @intjesus
  • Participation in Hive Think Tank panel "Hadoop + SQL" @tshiran in Palo Alto 7 Aug http://bit.ly/14Es3XK
Community Outreach
  • Twitter: @ApacheDrill is growing. Follower base currently 474.

Code

For details, see http://bit.ly/14YPXN9
Q3 code commits include but not limited to: see first milestone release

Mailing List Summary:

User Mailing List Posting

Sept 2013, 13
Aug 2013, 34
July 2013, 18
June 2013, 10
May 2013, 24
April 2013, 10
March 2013, 7
February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
Sept 2013, 644
Aug 2013, 394
July 2013, 370
June 2013, 247
May 2013, 135
April 2013, 188
March 2013, 260.
February 2013, 219
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) 1st milestone version approved for release!

-Ellen Friedman

June Summary in progress

(scroll down for previous summaries)

Issues:

Discussions on these key areas were _very focused_ and productive toward completion of goals for graduation:
1.) Develop momentum for community building
2.) Work toward first viable release

How has the community developed since the last report? In June:

Activity in the Drill community for this developing project has increased a lot in the first half of June. Subscription to the mailing list has grown to >364.

Activity has mainly been in the form of developing code. Participation in the user mailing list is growing slightly but of course the main progress is on the developer list.

Discussion on developer mailing list include discussion of SQL support; additional primitives;

Selection of Presentations, Articles and Outreach:

  • Standing event: Drill Hangout Tuesday mornings 9am Pacific Time at Google Hangout
  • Apache Drill Implementation: Berlin Buzzwords 3 June @ted_dunning & @mhausenblas
  • Apache Drill talk at Hadoop User Group France 6 June by @mhausenblas
  • @ApacheDrill is growing. Follower base currently 285.

Code

For details, see http://bit.ly/14YPXN9
June code commits include but not limited to: TBA

Mailing List Summary:

User Mailing List Posting

June 2013 10
May 2013, 24
April 2013, 10
March 2013, 7
February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
June 2013 247
May 2013, 135
April 2013, 188
March 2013, 260.
February 2013, 219
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) TBA

-Ellen Friedman

May Summary in progress

(scroll down for April + Q1 2013 summaries)

Issues:

Discussions on these key areas were _very focused_ and productive toward completion of goals for graduation:
1.) TBA
2.)

How has the community developed since the last report? In May:

Activity in the Drill community for this developing project continues to be very active. Subscription to the mailing list has grown to >313. Activity has mainly been in the form of developing code. Participation in the user mailing list is very small as the project is not yet at that stage. Discussions on the developer list: Active but slower in first half of May.

Selection of Presentations, Articles and Outreach:

  • Standing event: Drill Hangout Tuesday mornings 9am Pacific Time at Google Hangout
  • Apache Drill Implementation: Berlin Buzzwords 3 June @ted_dunning & @mhausenblas
  • Interactive Ad hoc query at scale: talk at Hadoop User Group UK by @mhausenblas
  • @ApacheDrill is growing. Follower base currently 251.

Code

For details, see http://bit.ly/14YPXN9
May code commits include but not limited to: TBA

Mailing List Summary:

User Mailing List Posting

April User mailing list 10
March 2013, 7
February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
May 77 to date
April 2013 188
March 2013, 260.
February 2013, 219
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) TBA

-Ellen Friedman

Project Report for Q2 of 2013: April Summary

(scroll down for Q1 2013 summaries)

Issues:

Discussions on these key areas were _very focused_ and productive toward completion of goals for graduation:
1.) TBA
2.)

How has the community developed since the last report? In April:

Activity in the Drill community ... Subscription to the mailing list has grown to 313.

Discussions on the developer list: Very active in April, with some new voices including but not limited to David Alves, Lisen Mu, Owen O'Malley, kishore g on these topics: timeline for dist execution; storage file format; use of Apache Helix; storage engine; schema.

Selection of Presentations, Articles and Outreach:

  • Standing event: Drill Hangout Tuesday mornings 9am Pacific Time at Google Hangout
  • All You Need to Know About Drill, talk during Big Data Week #bdw13 by Michael Hausenblas on 26 April http://bit.ly/17L1rDr
  • Introduction to Apache Drill talk at new Bay Area Analytics Meet-up: 2 April, speaker Tomer Shiran Details at http://bit.ly/167uVxh
  • @ApacheDrill is active. Follower base currently 223.

Code

For details, see http://bit.ly/14YPXN9
April code commits include but not limited to:

added flatten & join test execution
basic framework fro physical plan & later exploration modification to physical plan representation
added basic system test
Drill 55: RSERegistry correctly store built engines

Mailing List Summary:

User Mailing List Posting

Remain small as the project is still under development.

April User mailing list 11
March 2013, 7
February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
April 2013 188
March 2013, 260.
February 2013, 219
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) TBA

-Ellen Friedman

Project Report for Q1 of 2013: March Summary (end Q1; see Podling report for Q1 for final)

(scroll down for January, February 2013 summaries)

Note that the Q1 podling report has already been filed and approved. This summary completes the month of March.

Issues:

Discussions on these key areas were _very focused_ and productive toward completion of goals for graduation:
1.) TBA
2.)

How has the community developed since the last report? In March:

Activity in the Drill community continues to be lively. Many in the community are participating. Subscription to the mailing list has grown to 313.

Discussions on the developer list are active from the start of the month. A variety of people are participating, including some newcomers. Examples of participation on the developer list are too numerous to list entirely. They include but are not limited to:

March discussions included

  • report of clean-ups and fixes of various issues including the SQL parser;
  • a discussion led by Timothy and Jacques on the WindowFrame reference operator is open
  • specific suggestions of how new community members can contribute
  • another columnar format: parquet
  • HBase storage engine implementation
  • storage engine: define java interface

Selection of Presentations, Articles and Outreach:

  • Introduction to Apache Drill talk at new Bay Area Analytics Meet-up: 2 April, speaker Tomer Shiran Details at http://bit.ly/167uVxh
  • Value and Architecture of Apache Drill, Hadoop Summit Europe in Amsterdam 20 March 3pm, speaker Michael Hausenblas Details at http://bit.ly/XXLllg
  • Apache Drill hands-on workshop in Amsterdam led by Ted Dunning with help from NL HUG 19 March 2013.
  • Project Update & talk on SQL parser, Bay Area Apache Drill Users Group: 13 March, speakers Julian Hyde and Jacques Nadeau Details at http://bit.ly/WrC89P
  • @ApacheDrill is active. Follower base currently 179.

Code

For details, see http://bit.ly/14YPXN9
March code commits include but not limited to:

Mailing List Summary:

User Mailing List Posting

March 2013, 7
February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
March 2013, 260.
February 2013, 219
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) SQL parser committed
2.) Increase in community participating on mailing list with new contributors and increased public outreach and interest
3.) TBA

-Ellen Friedman

Project Report for Q1 of 2013: February Summary

Issues:

Discussions on these key areas were _very focused_ and productive toward this project's graduation goals:
1.) Implementation of reference operators
2.) Mocking library
3.) Build SQL parser that generates logical plan

How has the community developed since the last report? In February:

Activity in the Drill community continues to grow during February. Many in the community participated. Subscription to the mailing list has grown to 313.

Discussions on the developer list were active and frequent with a variety of people participating. Examples of participation on the developer list are too numerous to list entirely. They include but are not limited to:

Mailing lists started the month with user list discussions between Brian O’Neill & Ted Dunning about and on dev list activity about JIRAs opened for implementation of key operators of the reference interpreter, with Timothy Chen & Jacques Nadeau. A new discussion thread on junit version has opened, with Andrew Psaltis and Keys Botzum joining Jacques, Ted and Tim.

New discussion thread Mocking Library was started by Christopher Merrick and joined by Ed Kohlwey,Stevo Slavić, Jacques Nadeau, Timothy Chen and Ted Dunning. JIRAs continue to be posted on Implementation of reference operators including Flatten, Sequence, CollapseAggregate, RunningAggregate, WindowsFrame and from Julian Hyde with Jacques JIRAs for SQL parser that generates logical plan; Ted Dunning JIRA for building synthetic log dataset generator. A new discussion of Drill + behavioral data began with Jacques, Ted, and Ben Johnson. Sree Vadi contributed to a discussion of optic maven references with Ted, Timothy, and Julian Hyde.

Selection of Presentations, Articles and Outreach:

  • "Introduction to Apache Drill" by Ted Dunning Boulder/Denver Big Data Meetup 13 Feb. (Example of interest: slideshare views 776)
  • "Introduction to Apache Drill" by Keys Botzum New Jersey HUG Asbury Park, NJ on 19 Feb.
  • "Introduction to Apache Drill - interactive query and analysis at scale" at Munich HUG by Michael Hausenblas 22 Feb.(Example of interest: slideshare views 388)
  • "Introduction to Apache Drill" Strata Conference 2013 Santa Clara, CA by Tomer Shiran 27 Feb.
  • Meeting of Bay Area Apache Drill Users Group: Meeting in March; speakers will include Julian Hyde.
  • "Introduce Apache Drill and How People Will Use It" topic selected for new Bay Area Analytics Group Meet-up, Tomer Shiran to speak 2nd April.
  • @ApacheDrill is active. Follower base currently 144.
  • Traffic to wiki has included interest in the Drill high-level architecture diagram (announced via Twitter)

Code

For details, see http://bit.ly/14YPXN9
February code commits include but not limited to:

Synthetic Log Generator
JDBC driver; SQL parser & planner
Convert JSON result into immutable tree
Flatten 1st pass; out a new output field
Update logical plan

Mailing List Summary:

User Mailing List Posting

February 2013, 7

Developer Mailing List Posting:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
February 2013, 227
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) High level architecture diagram was added to Wiki
2.) 1st version SQL parser checked in
3.) Activity on developer mailing list increased.
4.) Community interest grew as reflected in part by increased following on @ApacheDrill.
5.) Code was committed (7 significant commits for February)

-Ellen Friedman

Project Report for Q1 of 2013: January Summary

Issues:

Discussions on these key areas were _very focused_ and productive toward this project's graduation goals:
1.) Logical Plan reference interpreter, syntax
2.) Schema-less Management
3.) Thrift-based Protocols

How has the community developed since the last report Dec 2012?

Drill community started with year with a burst of activity that has steadily grown.

User activity on mailing list has not yet gained momentum, but developer interest increased sharply with a variety of activities. Posting to the developer mailing list was up to 169 for January (up from 51 in December 2012 during the holiday season).

In addition to the activity on mailing lists, there have also been offline discussions. Participants are encouraged to bring discussions online as soon as possible for enriched discussion.

Many in the community participated. Examples include but are not limited to:

A Drill Wiki was set up by Ted Dunning and linked to the Drill website. New participant Stefan Siprell is working with Michael Hausenblas collecting data sets, experimenting with queries and exploring use cases to be reported via Drill Wiki. Q1 Report in progress (January summary) is posted to Wiki. Added demo how-to reference interpreter on Wiki. Michael Hausenblas added a high level architecture diagram to the Wiki.

Progess in the logical plan was a primary focus in January, both in discussion, development, code and external communication. This work on logical plan included development of two versions of a reference interpreter by Ted Dunning and development of a third by Jacques Nadeau. Jacques’ version was chosen as the version to use and further refined by Jacques and Gera Shegalov.

Jacques Nadeau has been working with Timothy Chen to characterize the JSON scanner API.

Newcomer Chris Merrick began work on Limit Operator Reference Implementation, with discussions including Jacques Nadeau and Julian Hyde.

A discussion of Cassandra storage engine implementation was furthered mainly by Henry Saputra, Brian O’Neill and Jacques Nadeau.

Selection of Presentations, Articles and Outreach:

*The first live demonstation of Drill and the reference interpreter was conducted at a JAVA meet-up in Portland 15 January 2013 by Gera Shegalov. Slides for the presentation were made available online, with 240 views to date.

*The Drill project was also described in a presentation at Hadoop Conference 2013 Winter in Tokyo in 21 January by Ted Dunning, and slides were made available online, with 1740 views to date.

*Slides and video from a December presentation on Drill by Ted Dunning at HUG France are now online thanks to Julien Muller: http://hugfrance.fr/drill-presented-by-ted-dunning/

*Drill presentation by Jacques Nadeau was given 28 January at the Global Big Data Conference in Santa Clara, CA and slides were made available online.

*An invited article on Drill by Ted Dunning and Jacques Nadeau 30 January 2013 edition of Software Developers journal. Article is titled “Apache Drill: Newcomer in the Hadoop Ecosystem”.

  • @ApacheDrill follower base grew to 107.

Code

Git-hub Code was checked in…

5 significant commits in December (December commits showed up in Jan report)

3 significant commits in January 2013

Mailing List Summary:

User Mailing List

New users continued to join the list as well as asking formally to be part of the community. Activity on mailing list for January: 21

Developer Mailing List:

http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/
January 2013, 169
Dec 2012, 51holidays
Nov 2012, 73 (jira, focused discussions)
Oct 2012, 214 (svn, discussions, jira)
Sep 2012, 413
Aug 2012, 85

Summary: How has the project developed since the last report:

== Milestones ==
1.) During the month of January, the reference interpreter was developed and checked in
2.) 1st live demo of Drill reference interpreter was conducted in Portland, Oregon (USA).
3.) Wiki was set up
4.) Community was grown through the use of Twitter entity @ApacheDrill
5.) Code was committed (8 significant commits for Dec-January)

-Ellen Friedman

Labels
  • No labels