Dynamic class loading in Python 2.6: RuntimeWarning: Parent module 'plugins' not found while handling absolute import

Question

I am working on a plugin system where plugin modules are loaded like this:

def load_plugins():
   plugins=glob.glob("plugins/*.py")
   instances=[]
   for p in plugins:
      try:
         name=p.split("/")[-1]
         name=name.split(".py")[0]
         log.debug("Possible plugin: %s", name)
         f, file, desc=imp.find_module(name, ["plugins"])
         plugin=imp.load_module('plugins.'+name, f, file, desc)
         getattr(plugin, "__init__")(log)
         instances=instances+plugin.get_instances()
      except Exception as e:
         log.info("Failed to load plugin: "+str(p))
         log.info("Error: %s " % (e))
         log.info(traceback.format_exc(e))
   return instances

The code works, but for each import statement in the plugin code i get a warning like this:

plugins/plugin.py:2: RuntimeWarning: Parent module 'plugins' not found while handling absolute import
  import os

No errors are reported for the main program code, and the plugins work.

Can somebody explain what the warning means and what I doing wrong. Do I need to create an empty plugins module separately and import it to keep python happy?

Answer 1

If the plugins directory does not have an __init__.py, it isn't a package, so when you create plugins.whatever, Python warns you that such a thing shouldn't really exist. (It couldn't be created by "import plugins.whatever" no matter what your path is.)

Also,

• Don't split on /, which is unportable. Use os.path.split.
• Don't use .split(".py") to get the name without the extension, which is buggy. Use os.path.splitext.
• Don't use getattr with a string literal. getattr(plugin, "__init__") is spelled plugin.__init__.
• I am confused why you are calling a module-level __init__ function. This doesn't seem right. Perhaps you want a "set_logger" function or better, to instantiate a class that takes a logger.
• Don't use L = L + some_other_list to extend a list, use the extend method, which has better performance and is more idiomatic.
• Don't squash unknown exceptions by except Exception. If you cannot plan to do something sane in response to an exception, your program cannot go on sanely.