Delete comments <a href="http://www.aptis.info/portfolio/delete-all-comments/" target="_blank" title="Help"> <span class="aui-icon aui-icon-small aui-iconfont-help aptis-delete-all-comments-title-icon">Help</span> </a>

nit on the naming here. Don't all indexes skip data? even our bloom index does ...

Vinoth Chandar | Apr 12, 2021

So far we are not using this index information in query engines. So I thought w...

Satish Kotha | Apr 13, 2021

satish.kotha rmpifer uditme This is a great discussion, we now also plan to us...

liwei | Apr 19, 2021

Just as discussion in . Now list marker files for rollback is expensive and sl...

liwei | Apr 22, 2021

Today the metadata table has a single partition files, which contains the file ...

Vinoth Chandar | Apr 22, 2021

vinoth Yes, can we unify the concept to metadata? Which resolve the performan...

liwei | Apr 23, 2021

I am not sure if marker files belong there. Once a commit is over, marker files...

Vinoth Chandar | Apr 24, 2021

satish.kotha do I need to checkout some branch you have? HoodieSnapshotMetadata...

Vinoth Chandar | Apr 24, 2021

Yes,this is available in Uber's version. For now, you can replace snapshot met...

Satish Kotha | Apr 24, 2021

option("hoodie.datasource.write.recordkey.field", "partitionPath,fileName,colNa...

Vinoth Chandar | Apr 24, 2021

Even without sorting, parquet was doing better. I also tried sorting using bulk...

Satish Kotha | Apr 24, 2021

Few things I thought of. 1) I think this is touched upon in pwason's original...

Vinoth Chandar | May 17, 2021

ath="hdfs://tables/my_metadata_table_parquet" satish.kotha Can we also evalua...

liwei | May 18, 2021

Yes, the scripts in appendix can be used to measure performance on cloud stores...

Satish Kotha | May 18, 2021

satish.kothahello , what the different between "parquet" and "parquet spark sql...

liwei | May 18, 2021

For 3rd row, parquet folder is registered as temp table in spark. We then used ...

Satish Kotha | May 18, 2021

satish.kotha I think we should use "parquet-spark sql" for performance comparis...

liwei | May 25, 2021

also "Appendix" can generate the results? I am happy to test these case in Alib...

liwei | May 18, 2021

Yes, that would be very helpful. Please reach out on slack if commands in Appen...

Satish Kotha | May 18, 2021

I wrote a JMH benchmark (reusing parts of satish's code from the appendix) that...

Vinoth Chandar | May 19, 2021

This seems to be different from results I observed. Couple questions: 1) Are y...

Satish Kotha | May 19, 2021

I wrote some code on top of HFile.Reader, reusing what HoodieHFileReader does. ...

Vinoth Chandar | May 20, 2021

vinoth satish.kotha Hello , About parquet performance: I see MetadataBenchm...

liwei | May 25, 2021

lw309637554 >>But in spark sql datasource use VectorizedParquetRecordReader w...

Vinoth Chandar | Jun 23, 2021

vinoth >>But in spark sql datasource use VectorizedParquetRecordReader which w...

liwei | Jul 09, 2021

lw309637554 Even with parquet 1.11.1 version, benchmark shows similar performan...

Satish Kotha | Aug 06, 2021

satish.kotha With predicate push down and large files parquet will be better. ...

liwei | Aug 10, 2021

Delete comments Help