Is it possible to expand the ioctl interface in Linux so that the user-space application can send a pointer to a function to the kernel space driver?
I'm in particular thinking of ways to handle the stream in user-controllable way but doing it in the kernel. Those operations could be attached to the kernel module but this would make development a lot easier as I wouldn't need to mess with the kernel during development.
More specifically, this would be the process:
I think you can achieve what you want by having your driver provide one or more character devices (or block devices) that your user space applications opens.
Then you could use inotify (linux journal article) for kernel->user space event communication. Ioctl or writing to the device for user space->kernel event communication. Data exchange could also be achieved by reading/writing to one or more device files.
Alternatively you can provide /proc or /sys filesystem entries or use netlink.
You might also consider ksocket:
Ksocket is a linux 2.6 kernel module that provides bsd-style socket interfaces (namely socket, bind, listen, connect, accept, ...) for kernel developers to facilitate their network progaramming in linux kernel space. The interfaces ksocket presents are much the same as their equivalent in glibc, so even new developers for kernel space will have no barrier in developing kernel network-related programms.