Cannot load pickled object

Houdini picture Houdini · Aug 15, 2013 · Viewed 48.3k times · Source

The problem I am having is when I try to load the pickled object. I have tried using both pickle.loads and pickle.load Here are the results:

pickle.loads:

TypeError: 'str' does not support the buffer interface

pickle.load:

TypeError: file must have 'read' and 'readline' attributes

Can someone please tell me what I am doing wrong in this process?

elif str(parser) == "SwissWithdrawn_Parser":
    # swissprot name changes
    print("Gathering SwissProt update info...")
    cache_hits = 0
    cache_misses = 0
    files = set()

    for f in os.listdir("out/cache/"):
        if os.path.isfile("out/cache/" + f):
            files.add(f)

    for name in sp_lost_names:

        cached = False
        url = (
            "http://www.uniprot.org/uniprot/?query=mnemonic%3a"
            + name
            + "+active%3ayes&format=tab&columns=entry%20name"
        )
        hashed_url = str(hash(url))

        ################### For Testing Only - use cache ##################
        if hashed_url in files:
            cached = True
            cache_hits += 1
            content = pickle.loads("out/cache/" + hashed_url)  # <-- problematic line
        else:
            cache_misses += 1
            content = urllib.request.urlopen(url)

        # get the contents returned from the HTTPResponse object
        content_list = [x.decode().strip() for x in content.readlines()]
        if not cached:
            with open("out/cache/" + hashed_url, "wb") as fp:
                pickle.dump(content_list, fp)
        ####################################################################

        # no replacement
        if len(content_list) is 0:
            change_log["swiss-names"] = {name: "withdrawn"}
        # get the new name
        else:
            new_name = content_list[1]
            change_log["swiss-names"] = {name: new_name}

Answer

Martijn Pieters picture Martijn Pieters · Aug 15, 2013

You need to either read the file first (as binary bytes) and use pickle.loads(), or pass an open file object to the pickle.load() command. The latter is preferable:

with open('out/cache/' +hashed_url, 'rb') as pickle_file:
    content = pickle.load(pickle_file)

Neither method supports loading a pickle from a filename.