amazon web services - Drophide fields from s3 based data lake consumable Athena table - Stack Overflow

I have a data lake implemented using AWS s3.Bronze and silver layer are implemented, with data in silv

I have a data lake implemented using AWS s3. Bronze and silver layer are implemented, with data in silver layer exposed for access via API and also via jdbc/odbc based sql client.

We have a requirement to drop/hide a field from the consumable Athena table, so far below is the option I've known and related limitations:

Option 1: Drop data for the table from silver layer and reprocess with glue script updated to drop the field

Limitation: costly, need to reprocess huge volume

Athena doesn't support dropping field, and using views every time crawler updates the silver table catalogue gets stale.

Could anyone please suggest whats a proper way handling this?

发布者:admin,转转请注明出处:http://www.yc00.com/questions/1744802681a4594586.html

相关推荐

发表回复

评论列表(0条)

  • 暂无评论

联系我们

400-800-8888

在线咨询: QQ交谈

邮件:admin@example.com

工作时间:周一至周五,9:30-18:30,节假日休息

关注微信