I have a data lake implemented using AWS s3. Bronze and silver layer are implemented, with data in silver layer exposed for access via API and also via jdbc/odbc based sql client.
We have a requirement to drop/hide a field from the consumable Athena table, so far below is the option I've known and related limitations:
Option 1: Drop data for the table from silver layer and reprocess with glue script updated to drop the field
Limitation: costly, need to reprocess huge volume
Athena doesn't support dropping field, and using views every time crawler updates the silver table catalogue gets stale.
Could anyone please suggest whats a proper way handling this?
发布者:admin,转转请注明出处:http://www.yc00.com/questions/1744802681a4594586.html
评论列表(0条)