Copy an entity in Google App Engine datastore in Python without knowing property names at 'compile' time

Question

In a Python Google App Engine app I'm writing, I have an entity stored in the datastore that I need to retrieve, make an exact copy of it (with the exception of the key), and then put this entity back in.

How should I do this? In particular, are there any caveats or tricks I need to be aware of when doing this so that I get a copy of the sort I expect and not something else.

ETA: Well, I tried it out and I did run into problems. I would like to make my copy in such a way that I don't have to know the names of the properties when I write the code. My thinking was to do this:

#theThing = a particular entity we pull from the datastore with model Thing
copyThing = Thing(user = user)
for thingProperty in theThing.properties():
    copyThing.__setattr__(thingProperty[0], thingProperty[1])

This executes without any errors... until I try to pull copyThing from the datastore, at which point I discover that all of the properties are set to None (with the exception of the user and key, obviously). So clearly this code is doing something, since it's replacing the defaults with None (all of the properties have a default value set), but not at all what I want. Suggestions?

Answer 1

Here you go:

def clone_entity(e, **extra_args):
  """Clones an entity, adding or overriding constructor attributes.

  The cloned entity will have exactly the same property values as the original
  entity, except where overridden. By default it will have no parent entity or
  key name, unless supplied.

  Args:
    e: The entity to clone
    extra_args: Keyword arguments to override from the cloned entity and pass
      to the constructor.
  Returns:
    A cloned, possibly modified, copy of entity e.
  """
  klass = e.__class__
  props = dict((k, v.__get__(e, klass)) for k, v in klass.properties().iteritems())
  props.update(extra_args)
  return klass(**props)

Example usage:

b = clone_entity(a)
c = clone_entity(a, key_name='foo')
d = clone_entity(a, parent=a.key().parent())

EDIT: Changes if using NDB

Combining Gus' comment below with a fix for properties that specify a different datastore name, the following code works for NDB:

def clone_entity(e, **extra_args):
  klass = e.__class__
  props = dict((v._code_name, v.__get__(e, klass)) for v in klass._properties.itervalues() if type(v) is not ndb.ComputedProperty)
  props.update(extra_args)
  return klass(**props)

Example usage (note key_name becomes id in NDB):

b = clone_entity(a, id='new_id_here')

Side note: see the use of _code_name to get the Python-friendly property name. Without this, a property like name = ndb.StringProperty('n') would cause the model constructor to raise an AttributeError: type object 'foo' has no attribute 'n'.