...
SQE seems to be designed as in-house solution. No flexible for data sources, and it seems to require custom version of storm-redis and storm-kafka.
It might take some time to become open-source ready. That’s why Heron is still in beta even though Twitter uses it in production.
The needed effort for replacing SQL-like interface with SQL interface via Calcite is nondeterministic.
Most of code on storm-sql-core is purely for supporting SQL interface via Calcite.
Even though we drop ‘standalone’ mode of Storm SQL, still huge part of code on storm-sql-core would be needed.
When we address SQL interface via Calcite, SQL like JSON interface will be removed, and JW Player should make some efforts to change their queries anyway.
SQE has limited expressions what JW Player need in their use cases.
For example, MAX and SUM provided but not MIN and AVG, and RLIKE provided but not LIKE.
Limited expressions could be annoying for other use cases.
Storm SQL is trying plans to try to cover Calcite SQL language with best effort.
Contributors of Storm SQL need to overcome learning curve of SQE.
We can’t continue putting the efforts to this area before IP clearance is done.
...
Calcite provides implementations of ‘LIKE’ / ‘SIMILAR TO’.
‘SIMILAR TO’ handles regular expression so we can just use this expression.
Supported regex is far from Java regex, so someone might still want to have a function for Java regex.
GetIf
It works like elvis operator, but predicate should be just a boolean value.
It’d be ideal to implement ‘CASE’ instead of using this.
...