Downloading text files with Python and ftplib.FTP from z/os

Brent.Longborough picture Brent.Longborough · Jul 26, 2009 · Viewed 11.4k times · Source

I'm trying to automate downloading of some text files from a z/os PDS, using Python and ftplib.

Since the host files are EBCDIC, I can't simply use FTP.retrbinary().

FTP.retrlines(), when used with open(file,w).writelines as its callback, doesn't, of course, provide EOLs.

So, for starters, I've come up with this piece of code which "looks OK to me", but as I'm a relative Python noob, can anyone suggest a better approach? Obviously, to keep this question simple, this isn't the final, bells-and-whistles thing.

Many thanks.

#!python.exe
from ftplib import FTP

class xfile (file):
    def writelineswitheol(self, sequence):
        for s in sequence:
            self.write(s+"\r\n")

sess = FTP("zos.server.to.be", "myid", "mypassword")
sess.sendcmd("site sbd=(IBM-1047,ISO8859-1)")
sess.cwd("'FOO.BAR.PDS'")
a = sess.nlst("RTB*")
for i in a:
    sess.retrlines("RETR "+i, xfile(i, 'w').writelineswitheol)
sess.quit()

Update: Python 3.0, platform is MingW under Windows XP.

z/os PDSs have a fixed record structure, rather than relying on line endings as record separators. However, the z/os FTP server, when transmitting in text mode, provides the record endings, which retrlines() strips off.

Closing update:

Here's my revised solution, which will be the basis for ongoing development (removing built-in passwords, for example):

import ftplib
import os
from sys import exc_info

sess = ftplib.FTP("undisclosed.server.com", "userid", "password")
sess.sendcmd("site sbd=(IBM-1047,ISO8859-1)")
for dir in ["ASM", "ASML", "ASMM", "C", "CPP", "DLLA", "DLLC", "DLMC", "GEN", "HDR", "MAC"]:
    sess.cwd("'ZLTALM.PREP.%s'" % dir)
    try:
        filelist = sess.nlst()
    except ftplib.error_perm as x:
        if (x.args[0][:3] != '550'):
            raise
    else:
        try:
            os.mkdir(dir)
        except:
            continue
        for hostfile in filelist:
            lines = []
            sess.retrlines("RETR "+hostfile, lines.append)
            pcfile = open("%s/%s"% (dir,hostfile), 'w')
            for line in lines:
                pcfile.write(line+"\n")
            pcfile.close()
        print ("Done: " + dir)
sess.quit()

My thanks to both John and Vinay

Answer

Dave Griffiths picture Dave Griffiths · Nov 1, 2011

Just came across this question as I was trying to figure out how to recursively download datasets from z/OS. I've been using a simple python script for years now to download ebcdic files from the mainframe. It effectively just does this:

def writeline(line):
    file.write(line + "\n")

file = open(filename, "w")
ftp.retrlines("retr " + filename, writeline)