How to access the system call from user-space?

Question

I read some paragraphs in LKD¹ and I just cannot understand the contents below:

Accessing the System Call from User-Space

Generally, the C library provides support for system calls. User applications can pull in function prototypes from the standard headers and link with the C library to use your system call (or the library routine that, in turn, uses your syscall call). If you just wrote the system call, however, it is doubtful that glibc already supports it!

Thankfully, Linux provides a set of macros for wrapping access to system calls. It sets up the register contents and issues the trap instructions. These macros are named _syscalln(), where n is between zero and six. The number corresponds to the number of parameters passed into the syscall because the macro needs to know how many parameters to expect and, consequently, push into registers. For example, consider the system call open(), defined as

long open(const char *filename, int flags, int mode)

The syscall macro to use this system call without explicit library support would be

#define __NR_open 5
_syscall3(long, open, const char *, filename, int, flags, int, mode)

Then, the application can simply call open().

For each macro, there are 2+2×n parameters. The first parameter corresponds to the return type of the syscall. The second is the name of the system call. Next follows the type and name for each parameter in order of the system call. The __NR_open define is in <asm/unistd.h>; it is the system call number. The _syscall3 macro expands into a C function with inline assembly; the assembly performs the steps discussed in the previous section to push the system call number and parameters into the correct registers and issue the software interrupt to trap into the kernel. Placing this macro in an application is all that is required to use the open() system call.

Let's write the macro to use our splendid new foo() system call and then write some test code to show off our efforts.

#define __NR_foo 283
__syscall0(long, foo)

int main ()
{
        long stack_size;

        stack_size = foo ();
        printf ("The kernel stack size is %ld\n", stack_size);
        return 0;
}

What does the application can simply call open() mean?

Besides, for the last piece of code, where is the declaration of foo()? And how can I make this piece of code compilable and runnable? What are the header files I need to include?

__________
¹ Linux Kernel Development, by Robert Love.  PDF file at wordpress.com (go to page 81); Google Books result.

Answer 1

You first should understand what is the role of the linux kernel, and that applications interact with the kernel only thru system calls.

In effect, an application runs on the "virtual machine" provided by the kernel: it is running in the user space and can only do (at the lowest machine level) the set of machine instructions permitted in user CPU mode augmented by the instruction (e.g. SYSENTER or INT 0x80 ...) used to make system calls. So, from the user-level application point of view, a syscall is an atomic pseudo machine instruction.

The Linux Assembly Howto explains how a syscall can be done at the assembly (i.e. machine instruction) level.

The GNU libc is providing C functions corresponding to the syscalls. So for example the open function is a tiny glue (i.e. a wrapper) above the syscall of number NR__open (it is making the syscall then updating errno). Application usually call such C functions in libc instead of doing the syscall.

^{You could use some other libc. For instance the MUSL libc is somhow "simpler" and its code is perhaps easier to read. It also is wrapping the raw syscalls into corresponding C functions.}

If you add your own syscall, you better also implement a similar C function (in your own library). So you should have also a header file for your library.

See also intro(2) and syscall(2) and syscalls(2) man pages, and the role of VDSO in syscalls.

Notice that syscalls are not C functions. They don't use the call stack (they could even be invoked without any stack). A syscall is basically a number like NR__open from <asm/unistd.h>, a SYSENTER machine instruction with conventions about which registers hold before the arguments to the syscall and which ones hold after the result[s] of the syscall (including the failure result, to set errno in the C library wrapping the syscall). The conventions for syscalls are not the calling conventions for C functions in the ABI spec (e.g. x86-64 psABI). So you need a C wrapper.