I want to find a caffe python data layer example to learn.
I know that Fast-RCNN has a python data layer, but it's rather complicated since I
am not familiar with object detection.
So my question is, is there a python data layer example where I can learn how to define my own data preparation procedure?
For example, how to do define a python data layer do much more data augmentation
(such as translation, rotation etc.) than caffe "ImageDataLayer"
.
Thank you very much
You can use a "Python"
layer: a layer implemented in python to feed data into your net. (See an example for adding a type: "Python"
layer here).
import sys, os
sys.path.insert(0, os.environ['CAFFE_ROOT']+'/python')
import caffe
class myInputLayer(caffe.Layer):
def setup(self,bottom,top):
# read parameters from `self.param_str`
...
def reshape(self,bottom,top):
# no "bottom"s for input layer
if len(bottom)>0:
raise Exception('cannot have bottoms for input layer')
# make sure you have the right number of "top"s
if len(top)!= ...
raise ...
top[0].reshape( ... ) # reshape the outputs to the proper sizes
def forward(self,bottom,top):
# do your magic here... feed **one** batch to `top`
top[0].data[...] = one_batch_of_data
def backward(self, top, propagate_down, bottom):
# no back-prop for input layers
pass
For more information on param_str
see this thread.
You can find a sketch of a data loading layer with pre-fetch here.