Trouble Calling ioctl from user-space C

Question

I'm trying to implement a program to access memory on an embedded system. I need to access some control register so I think that ioctl is the best way to do it. I have added the ioctl to the fops:

struct file_operations aes_fops = {
  read: aes_read,
  write: aes_write,
  unlocked_ioctl: aes_ioctl,
  open: aes_open,
  release: aes_release
};

And have set up the function:

int aes_ioctl(struct inode *inode,  
     struct file *file, 
     unsigned int ioctl_num,    
     unsigned long ioctl_param){

     printk(KERN_INFO "in ioctl\n");
....
}

But I am not getting inside of this function. Here is my user space code. Please help me understand if I am doing this totally wrong.

int main(int argc, char* argv[]){
    int fd = fopen("/dev/aes", "r+");
    ioctl(fd, 0, 1);
    fclose(fd);
}

Some of the code is apparently for older kernels, because I am compiling for an embedded system where an older version of Linux has been modified.

Answer 1

The problem with your code is the request number you are using - 0. The kernel has some request number reserved for internal use. The kernel regards the request number as a struct, separates it to fields and calls the right subsystem for it.

See Documentation/ioctl/ioctl-number.txt (from Linux 3.4.6):

Code  Seq#(hex) Include File            Comments
========================================================
0x00    00-1F   linux/fs.h              conflict!
0x00    00-1F   scsi/scsi_ioctl.h       conflict!
0x00    00-1F   linux/fb.h              conflict!
0x00    00-1F   linux/wavefront.h       conflict!
0x02    all     linux/fd.h
0x03    all     linux/hdreg.h
...

Depending on what you are during, you'd have to follow the kernel guidelines for adding new ioctls():

If you are adding new ioctl's to the kernel, you should use the _IO
macros defined in <linux/ioctl.h>:

    _IO    an ioctl with no parameters
    _IOW   an ioctl with write parameters (copy_from_user)
    _IOR   an ioctl with read parameters  (copy_to_user)
    _IOWR  an ioctl with both write and read parameters.

See the kernel's own Documentation/ioctl/ioctl-decoding.txt document for further details on how those numbers are structured.

In practice, if you pick Code 1, which means starting from 0x100 up until 0x1ff, you'd be fine.