Python 101: How to Move Files between Servers

If you do much system administration, then you know that sometimes you have to write scripts that can move files between servers. I’m not really a system administrator by trade, but I did have to do this sort of thing in some of my programs anyway. Python has several 3rd party packages that provide this ability. We’ll be looking at how to do it with paramiko which depends on PyCrypto (or download PyCrypto from PyPI).

Writing the Code

Assuming you have all the 3rd party packages mentioned earlier, we can get to coding. To keep things super simple, we’ll just use paramiko for our first example. The following code is loosely based on some code I use at work. Let’s take a look!

import paramiko

########################################################################
class SSHConnection(object):
    """"""

    #----------------------------------------------------------------------
    def __init__(self, host, username, password, port=22):
        """Initialize and setup connection"""
        self.sftp = None
        self.sftp_open = False
        
        # open SSH Transport stream
        self.transport = paramiko.Transport((host, port))
        
        self.transport.connect(username=username, password=password)
        
    #----------------------------------------------------------------------
    def _openSFTPConnection(self):
        """
        Opens an SFTP connection if not already open
        """
        if not self.sftp_open:
            self.sftp = paramiko.SFTPClient.from_transport(self.transport)
            self.sftp_open = True
    
    #----------------------------------------------------------------------
    def get(self, remote_path, local_path=None):
        """
        Copies a file from the remote host to the local host.
        """
        self._openSFTPConnection()        
        self.sftp.get(remote_path, local_path)        
            
    #----------------------------------------------------------------------
    def put(self, local_path, remote_path=None):
        """
        Copies a file from the local host to the remote host
        """
        self._openSFTPConnection()
        self.sftp.put(local_path, remote_path)
        
    #----------------------------------------------------------------------
    def close(self):
        """
        Close SFTP connection and ssh connection
        """
        if self.sftp_open:
            self.sftp.close()
            self.sftp_open = False
        self.transport.close()
        
if __name__ == "__main__":
    host = "myserver"
    username = "mike"
    pw = "dingbat!"
    
    origin = '/home/mld/projects/ssh/random_file.txt'
    dst = '/home/mdriscoll/random_file.txt'
    
    ssh = SSHConnection(host, username, pw)
    ssh.put(origin, dst)
    ssh.close()

Let’s spend some time breaking this down a bit. In our class’s __init__ we need to pass in at least parameters. In this example, we pass it our host, a username and a password. Then we open an SSH Transport stream object. Next we call our put method to send a file from our machine to the server. If you want to download a file, see the get method. Finally, we call our close method to close our connections. You will note that in the put and get methods, we have them call a semi-private method that checks to see if our SFTPClient is initialized and if not, it goes ahead and creates it.

Wrapping Up

Paramiko sure makes this stuff easy. I highly recommend reading Jesse’s old article on the subject (linked below) as he goes into a lot more detail. I am curious about what packages other people use for ssh and scp though, so feel free to leave me some tips in the comments. I’ve been hearing some good things about Fabric.

Further Reading

• Jesse Noller’s SSH Programming with Paramiko