apache nifi - How to read csv with binary delimiter - Stack Overflow

The source csv file is:123456TextValue1654321TextValue2where 123456 and TextValue1 are different valu

The source csv file is:

123456TextValue1
654321TextValue2

where 123456 and TextValue1 are different values, separated by binary delimiter (\u0001)

Similary 654321 TextValue2

I use ConvertRecord for updating the delimiter from "\u0001" to ";"

RecordReader is CSVReader with the following properties:

  • Schema Access Strategy: Use 'Schema Text' Property
  • Schema Text: #{text_schema}
  • Value Separator: \u0001
  • Treat First Line as Header: false
  • Ignore CSV Header Column Names: true

RecordWriter is CSVRecordSetWriter:

  • Schema Access Strategy: Use 'Schema Text' Property
  • Schema Text: #{text_schema}
  • Value Separator: ;
  • Include Header Line: true

text_schema is

 {
  "type": "record",
  "name": "test_schema",
  "fields": [
    {
      "name": "FIELD_1",
      "type": ["int","null"],
    "description": "FIELD_1"
  },
  {
    "name": "FIELD_2",
    "type": ["string","null"],
    "description": "FIELD_2"
  }
 ]
}

Expected output is:

   FIELD_1;FIELD_2;
   123456;TextValue1
   654321;TextValue2

But I got the following error:

ERROR
ConvertRecord[id=01931001-0d7e-1e43-146d-1a380e6d43b7] Failed     to process FlowFile[filename=7365b509-7100-4bc2-a070-    4cc8ce8377b9]; will route to failure:     .apache.nifi.processor.exception.ProcessException: Could not     parse incoming data
   Caused by:                   .apache.nifi.serialization.MalformedRecordException: Error                  while getting next record
  Caused by: java.lang.NumberFormatException: For input string:       "123456TextValue1"

发布者:admin,转转请注明出处:http://www.yc00.com/questions/1745611178a4635958.html

相关推荐

  • apache nifi - How to read csv with binary delimiter - Stack Overflow

    The source csv file is:123456TextValue1654321TextValue2where 123456 and TextValue1 are different valu

    11小时前
    40

发表回复

评论列表(0条)

  • 暂无评论

联系我们

400-800-8888

在线咨询: QQ交谈

邮件:admin@example.com

工作时间:周一至周五,9:30-18:30,节假日休息

关注微信