2

I have a Logstash server I need to run commands in through Cloud Composer which uses Airflow. I am using the SH Operator. I have made the SSH connection and added my RSA Private Key to the extras field. I've tried to connect while having a username entered and while not having a username entered. Everytime, after connecting, I get the error: "Error: SSH Operator Error: No authentication methods available." I can use this RSA key to connect through Putty. I can PING the server as well. There is no password for this server - only the keys. SSHOperator Code:

t1 =SSHOperator(
                task_id = 'SSHOperator',

                ssh_hook=sshHook,
                do_xcom_push=True,
                command='ls /logstash/scripts',
                dag=dag
            )

SSH Connection: ssh connection image

maximuminfiniti
  • 21
  • 1
  • 4

1 Answers1

2

In order to properly propagate Extra field parameters within Connection definition approaching SSHOperator connection type, you should explicitly set the path to your RSA key file in JSON data format, as shared in Airflow documentation, for instance:

{
   "key_file": "/PATH_TO_YOUR_KEY/.ssh/KEY_NAME_rsa"
}
Nick_Kh
  • 5,089
  • 2
  • 10
  • 16
  • 1
    that was already set up as I said in my post. The solution to this issue was found to be that the RSA key we had been provided had been generated on a WIndows machine and the VMs are Linux. To fix the issue, we logged into the Clusters associated with the Composer environment and generated keys from there. – maximuminfiniti Jan 03 '20 at 17:48