I have a job to get ConsumerRecord from kafka and using AvroParquetWriter to write to a parquet file. Now I would like to add a field in the parquet but getting below error:

.apache.avro.AvroRuntimeException: Not a valid schema field

Just wonder, can I disable the validation?

I have a job to get ConsumerRecord from kafka and using AvroParquetWriter to write to a parquet file. Now I would like to add a field in the parquet but getting below error:

.apache.avro.AvroRuntimeException: Not a valid schema field

Just wonder, can I disable the validation?

• apache-kafka
• apache-nifi

Share Follow edited Mar 5 at 8:36 OneCricketeer 192k2020 gold badges142142 silver badges268268 bronze badges asked Mar 4 at 7:39 carfieldcarfield 2,09122 gold badges1515 silver badges1515 bronze badges 1

• Show entire stack trace – OneCricketeer Commented Mar 5 at 8:36

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

This should be able to done by set enable.schema.validation=false. But what I actually need is Extend Avro schema via Java API by adding one field