June 22, 2012

Patching vmware vmnet module for Linux 3.2.*

When installing vmware player 4.0.4, I had the nice surprise of finding out that the vmnet kernel module would not properly compile while starting the vmware service.

2012-06-22T15:34:41.039+01:00| vthread-3| I120: Building module vmnet.
2012-06-22T15:34:41.040+01:00| vthread-3| I120: Extracting the sources of the vmnet module.
2012-06-22T15:34:41.046+01:00| vthread-3| I120: Building module with command: /usr/bin/make -j -C /tmp/vmware-root/modules/vmnet-only auto-build SUPPORT_SMP=1 HEADER_DIR=/lib/modules/3.2.0-25-generic/build/include CC=/usr/bin/gcc GREP=/usr/bin/make IS_GCC_3=no VMCCVER=4.6
2012-06-22T15:34:41.835+01:00| vthread-3| I120: Failed to compile module vmnet!

Manually directly compiling the module, we see a couple of incompatibility with the kernel header definitions

[email protected]:~/code$ tar xf /usr/lib/vmware/modules/source/vmnet.tar -C ./ && cd vmnet-only/
[email protected]:~/code/vmnet-only$ make -j -C ./ auto-build SUPPORT_SMP=1 HEADER_DIR=/lib/modules/3.2.0-25-generic/build/include CC=/usr/bin/gcc GREP=/usr/bin/make IS_GCC_3=no VMCCVER=4.6
[...skipping redundant lines...]
/home/aabbas/code/vmnet-only/filter.c:60:16: error: ‘THIS_MODULE’ undeclared here (not in a function)
make[2]: *** [/home/aabbas/code/vmnet-only/filter.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/aabbas/code/vmnet-only/userif.c: In function ‘VNetCsumCopyDatagram’:
/home/aabbas/code/vmnet-only/userif.c:520:3: error: incompatible type for argument 1 of ‘kmap’
include/linux/highmem.h:48:21: note: expected ‘struct page *’ but argument is of type ‘const struct <anonymous>’
/home/aabbas/code/vmnet-only/userif.c:523:3: error: incompatible type for argument 1 of ‘kunmap’
include/linux/highmem.h:54:20: note: expected ‘struct page *’ but argument is of type ‘const struct <anonymous>’
make[2]: *** [/home/aabbas/code/vmnet-only/userif.o] Error 1
/home/aabbas/code/vmnet-only/netif.c: In function ‘VNetNetIfSetup’:
/home/aabbas/code/vmnet-only/netif.c:134:7: error: unknown field ‘ndo_set_multicast_list’ specified in initializer
/home/aabbas/code/vmnet-only/netif.c:134:7: warning: initialization from incompatible pointer type [enabled by default]
/home/aabbas/code/vmnet-only/netif.c:134:7: warning: (near initialization for ‘vnetNetifOps.ndo_validate_addr’) [enabled by default]

I have created a small repo at my github account @alouche, containing some code changes to fix these issues - the repo will continuously updated if necessary against different kernel versions (only from 3.2 up). To fix it, simply replace vmnet.tar in /usr/lib/vmware/modules/source with my modified version

[email protected]:/usr/lib/vmware/modules/source# wget https://github.com/alouche/vmware-vmnet-3.2-kernel/tarball/3.2.0 -O - | tar -xzf - --transform 's/alouche-vmware-vmnet-3.2-kernel-0dcdbb9/vmnet-only/' && tar cvf vmnet.tar vmnet-only

Start the vmware service and the vmnet module will automatically successfully compile.

Enjoy,