Delete comments <a href="http://www.aptis.info/portfolio/delete-all-comments/" target="_blank" title="Help"> <span class="aui-icon aui-icon-small aui-iconfont-help aptis-delete-all-comments-title-icon">Help</span> </a>

scbai can we please fill out the proposers list ? Also please add codope here ...

Vinoth Chandar | Sep 08, 2021

While I acknowledge that typical backwards incompatible evolutions are not supp...

Vinoth Chandar | Sep 08, 2021

I am actually trying to move away from partitions and build indexes instead for...

Vinoth Chandar | Sep 08, 2021

i think we should not add this to the commit metadata in the timeline. We can e...

Vinoth Chandar | Sep 08, 2021

if there are two concurrent writers, how would the +1 work? Hudi is moving towa...

Vinoth Chandar | Sep 08, 2021

meaning, it will be a special internal field in each base file? what about orc ...

Vinoth Chandar | Sep 08, 2021

again, we don't need a special version-id for the schema in Hudi. We already ha...

Vinoth Chandar | Sep 08, 2021

do we track the column position in the schema?

Vinoth Chandar | Sep 08, 2021

yes

Jack | Sep 09, 2021

this will increase the commit file size over time. lets explore keeping this in...

Vinoth Chandar | Sep 08, 2021

Voila. We can just deal with it using metadata table cleaning, compaction

Vinoth Chandar | Sep 08, 2021

I am really hesitant to introduce this kind of translation in the physical data...

Vinoth Chandar | Sep 08, 2021

yet to review this in detail

Vinoth Chandar | Sep 08, 2021

I think this can be follow on work. not needed in the first phase IMO

Vinoth Chandar | Sep 08, 2021

This is actually what I was talking about before.

Vinoth Chandar | Sep 08, 2021

codopecould you explain how this flow works without the `ID`. my take is that w...

Vinoth Chandar | Sep 08, 2021

Like I mentioned we also need to check this across DML and DML. Not just DDL vs...

Vinoth Chandar | Sep 08, 2021

We need to support reading of existing data without changes and IIUC you are su...

Vinoth Chandar | Sep 08, 2021

IMO we should do an upgrade step regardless, if we go with the id based approac...

Vinoth Chandar | Sep 08, 2021

but the physical columns in old parquet files will not be id named? so we need ...

Vinoth Chandar | Sep 08, 2021

We probably need to update the query engines first to read id-based files in th...

Vinoth Chandar | Sep 08, 2021

scbai I don't think we are very apart. I am convinced of your high level approa...

Vinoth Chandar | Sep 08, 2021

vinoth I think we almost reached an agreement. But there are some issues I need...

Jack | Sep 09, 2021

562622367 RFC - 33 Hudi supports more comprehensive Schema Evolution this pict...

Vinoth Chandar | Sep 09, 2021

sorry for leading your misunderstanding.

Jack | Sep 09, 2021

562622367 If I understand correctly, the `id` for each field in the schema will...

Sagar Sumit | Sep 09, 2021

codope . no need to be mapped to the Parquet ID for the field for schema mergi...

Jack | Sep 09, 2021

More aspects we can flesh out in this RFC IMO Support across all query engine...

Vinoth Chandar | Sep 09, 2021

vinoth codope vbalaji i agree with that more aspects we can flesh out in this...

Jack | Sep 09, 2021

Delete comments Help