insmod fails with "Unknown symbol in module" for a symbol defined in another module

Question

I am working in Ubuntu. I am trying to make two kernel modules which uses each other functions. My problem is that I got modules properly compiled, but the symbol is not resolved for one of them.

To make things simple, let's call these modules as m1 and m2.

m2 is exporting function void func_m2(void). The m1 is calling this function. Both modules properly compile.

After it all compiles, I need to load first the m2 module (because it has exported func_m2 function) and afterwards m1 module. So, let's make it:

volodymyr@sv1:~/development/kmodules/m2$ sudo insmod ./m2.ko

Now, lets load m1 module which is trying to use func_m2:

volodymyr@sv1:~/development/kmodules/m1$ sudo insmod ./m1.ko
insmod: error inserting './m1.ko': -1 Unknown symbol in module

Following is what I see in logs:

volodymyr@sv1:~/development/kmodules/m1$ dmesg | tail
[ 3938.166616] Loading m2 module ...
[ 3963.078055] m1: no symbol version for func_m2
[ 3963.078059] m1: Unknown symbol func_m2

So, it seems like the references to symbol func_m2 is not resolved. Interesting. Let's check if it is present in symbol table:

volodymyr@sv1:~/development/kmodules$ cat /proc/kallsyms | grep 'func_m2'
ffffffffa00530d0 r __ksymtab_func_m2    [m2]
ffffffffa00530e8 r __kstrtab_func_m2    [m2]
ffffffffa00530e0 r __kcrctab_func_m2    [m2]
ffffffffa0053000 T func_m2      [m2]
000000004edd543f a __crc_func_m2        [m2]

As you can see, the func_m2 is actually present in symbol table. So why m1 can't be loaded?

I have installed properly Linux headers for my kernel and Linux sources. I did not make any modifications in kernel, it is untouched, and it's version is: 2.6.31-16-generic (I run x64)

Now, to show you full picture I am putting here the source code and Makefile I used for this test for both m1 and m2 modules.

m1 module:

m1.c:

#include <linux/module.h>
#include <linux/kernel.h>

extern void func_m2(void);

int hello_start(void)
{
    printk(KERN_INFO "Loading m1 module ...\n");

    func_m2();

    return 0;
 }

 void hello_end(void)
 {
    printk(KERN_INFO "Unloading m1 ...\n");
 }

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");

Makefile:

obj-m := m1.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

m2 module:

m2.c:

#include <linux/module.h>
#include <linux/kernel.h>

int hello_start(void)
{
    printk(KERN_INFO "Loading m2 module ...\n");

    return 0;
}

void hello_end(void)
{
    printk(KERN_INFO "Unloading m2 ...\n");
}

void func_m2(void)
{
    printk(KERN_INFO "This a function in m2\n");
}

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");
EXPORT_SYMBOL(func_m2);

Makefile:

obj-m := m2.o
export-objs := m2.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Basically my question is: Why can't m1 be loaded?

It would be helpful if someone could answer.

Answer 1

When you build m2, it creates a Module.symvers file.

Copy this file to where you are building m1. Then make m1, and insmod it.

You probably had a warning when you were building m1 before, something like:

WARNING: "func_m2" [/tmp/m1/m1.ko] undefined!

This should disappear once you use the Module.symvers from the m2 module.

From http://www.kernel.org/doc/Documentation/kbuild/modules.txt:

--- 6.2 Symbols and External Modules

When building an external module, the build system needs access to the symbols from the kernel to check if all external symbols are defined. This is done in the MODPOST step. modpost obtains the symbols by reading Module.symvers from the kernel source tree. If a Module.symvers file is present in the directory where the external module is being built, this file will be read too. During the MODPOST step, a new Module.symvers file will be written containing all exported symbols that were not defined in the kernel.

And this is also worth reading, from the same file:

--- 6.3 Symbols From Another External Module

Sometimes, an external module uses exported symbols from another external module. kbuild needs to have full knowledge of all symbols to avoid spitting out warnings about undefined symbols. Three solutions exist for this situation.

NOTE: The method with a top-level kbuild file is recommended but may be impractical in certain situations.

Use a top-level kbuild file If you have two modules, foo.ko and bar.ko, where foo.ko needs symbols from bar.ko, you can use a common top-level kbuild file so both modules are compiled in the same build. Consider the following directory layout:

./foo/ <= contains foo.ko ./bar/ <= contains bar.ko

The top-level kbuild file would then look like:

$ ./Kbuild (or ./Makefile): obj-y := foo/ bar/

And executing

$ make -C $KDIR M=$PWD

will then do the expected and compile both modules with full knowledge of symbols from either module.

Use an extra Module.symvers file When an external module is built, a Module.symvers file is generated containing all exported symbols which are not defined in the kernel. To get access to symbols from bar.ko, copy the Module.symvers file from the compilation of bar.ko to the directory where foo.ko is built. During the module build, kbuild will read the Module.symvers file in the directory of the external module, and when the build is finished, a new Module.symvers file is created containing the sum of all symbols defined and not part of the kernel.

Use "make" variable KBUILD_EXTRA_SYMBOLS If it is impractical to copy Module.symvers from another module, you can assign a space separated list of files to KBUILD_EXTRA_SYMBOLS in your build file. These files will be loaded by modpost during the initialization of its symbol tables.