google cloud platform - Airflow SSH Operator bash commands are failing Intermittently - Stack Overflow

We are currently running multiple bash commands through Airflow 2.7.3 version under Google Cloud Compos

We are currently running multiple bash commands through Airflow 2.7.3 version under Google Cloud Composer.

from airflow.providers.ssh.operators.ssh import SSHOperator

task_exec = SSHOperator(
    task_id='additonal_info',
    command="ksh -x execute.ksh paramter1",
    ssh_conn_id="SSH_CONNECTION",
    conn_timeout=432000,       
)

Sometimes command runs for 1 hour , sometime to 8 hours and sometimes with in 30-40 minutes commands fails with returning no error at the bottom of Airflow task but it will be failed state in Airflow.

Could any one pls suggest what can be done to fix this issue

发布者:admin,转转请注明出处:http://www.yc00.com/questions/1745601910a4635451.html

相关推荐

发表回复

评论列表(0条)

  • 暂无评论

联系我们

400-800-8888

在线咨询: QQ交谈

邮件:admin@example.com

工作时间:周一至周五,9:30-18:30,节假日休息

关注微信