Is there any way to stop or pause an already running optmize on delta table started using

val deltaTable = DeltaTable.forPath(spark, deltaTablePath) 
deltaTable.optimize().executeCompaction()

Is there any way to stop or pause an already running optmize on delta table started using

val deltaTable = DeltaTable.forPath(spark, deltaTablePath) 
deltaTable.optimize().executeCompaction()

• apache-spark
• databricks
• delta

Share Improve this question Follow asked Jan 29 at 8:27 Shailendra KirtikarShailendra Kirtikar 4744 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

To stop or pause a running OPTIMIZE operation on a Delta table (started via executeCompaction()), there is no direct API in Delta Lake 2.x or earlier. If the OPTIMIZE is running as a Spark job, you can kill the associated Spark job.