User Tools

Site Tools


developingpld:advanceddeveloping:fixingasneeded

This is an old revision of the document!


Fixing --as-needed problems

Quick Workaround

Put in spec file this line:

%devine filterout_ld -Wl,--as-needed

But don't use it unless you really need.

Why ? Because it's very stupid. If something fails to build normally it's a sign –as-needed did well it's job and disabled unneeded libraries.

How --as-needed works

With –as-needed enabled only libraries which contain symbols required by object files are linked.

About positions

Linker options are positional, and position is very important. Because it looks for missing symbols only in libraries coming after object file.

Correct ones are:

$ gcc $(LDFLAGS) -o $@ $(OBJECTS) $(LIBS)

and this generally works well.

If there is something like that:

$ gcc -Wl,--as-needed -l<library> <objects.o>

the library will never be linked, because it's not required at the time it's checked.

And with something like that:

$ gcc -l<library> -Wl,--as-needed <objects.o>

library will allways be linked, even if it's not required.

Short examples

Most of problems are with readline/ncurses and tinfo, and it's nice example: tinfo is required by both readline and ncurses, and both are linked with this library (now, when I've fixed readline;).

But some packages link with readline or ncurses while they use only symbols from tinfo. Without –as-needed those executables work because they are linked with tinfo from readline/ncurses library. With –as-needed it will not work, because readline/ncurses contain no symbols required by executable so they are not linked, it's dependencies naturally are neither linked. Thats why there is a need to pass -ltinfo. If it requires only symbols from tinfo it's ok to s/ncurses/tinfo/. But if it really requires readline/ncurses but there is some executable (or ./configure) which requires only tinfo both -lreadline/-lncurses and -ltinfo should be passed.

For a longer example take a look at real example nr. 1.

Other common problem is when package produces some shared libraries without linking them with all required libraries, and everything is linked at the end to one binary. So we have:

$ gcc -Wl,--as-needed -o executable <objects.o> -l2 -l1

it's usual when object files require only library 2, and -l1 is required by -l2. As I said linker only checks for symbols from objects, so -l1 is not linked. Normally it is easy to fix it, simply make sure while linking -l2 it is linked to -l1 (take a look at the second real example).

Real examples

1. unresolved symbols in executable

Very common situation, xmoto.spec. Actually, it's nothing new. Every linking behaves this way if some library is missing.

With -Wl,–as-needed enabled it stops on something like this:

x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\"  -ggdb -O2 -Wl,--as-needed -o xmoto-edit  BuiltInFont.o [...object files...] Packager.o -lGL -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lbz2 -lz -lpng -ljpeg

VApp.o: In function `vapp::App::getRealTime()':
src/VApp.cpp:287: undefined reference to `SDL_GetTicks'
VApp.o: In function `vapp::App::getTime()':
src/VApp.cpp:284: undefined reference to `SDL_GetTicks'

[... bunch of missing SDL functions ...]

Editor.o: In function `vapp::EditorApp::viewDrawGrid()':
src/Editor.cpp:777: undefined reference to `SDL_GetMouseState'
Editor.o:src/Editor.cpp:46: more undefined references to `SDL_GetMouseState' follow
EditorMain.o: In function `main':
src/EditorMain.cpp:59: undefined reference to `SDL_Quit'

collect2: ld returned 1 exit status
make[1]: *** [xmoto-edit] Error 1

So let's try to find some of missing symbols:

$ grep SDL_GetMouseState /usr/lib64/libSDL*.so
Binary file /usr/lib64/libSDL.so matches

they are in -lSDL, but binary does not link with -lSDL; edit Makefile by hand and add -lSDL at the same place -lSDL_mixer is: LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg

What we get after running make in build tree:

x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\" -ggdb -O2 -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o  -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg 

VApp.o: In function `vapp::App::grabScreen()':
src/VApp.cpp:667: undefined reference to `glReadBuffer'
src/VApp.cpp:671: undefined reference to `glReadPixels'

[...more missing gl functions...]

src/Editor.cpp:1280: undefined reference to `glEnable'
src/Editor.cpp:1288: undefined reference to `glDisable'

collect2: ld returned 1 exit status

Same story:

$ grep glEnableClientState /usr/lib64/lib*.so
Binary file /usr/lib64/libGL.so matches

LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg

But take a look at spec file, -lGL thing was fixed there already:

%{__make} \
        GL_LIBS="-lGL"

Anyway, lets run make, and what we get ?

x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\"  -mfpmath=sse -ggdb -O2 -ftree-vectorize -ftree-vectorizer-verbose=1 -march=athlon64 -mmmx -msse -msse2 -m3dnow  -s -Wl,--as-needed -Wl,-s  -o xmoto-edit  BuiltInFont.o Image.o LevelSrc.o VApp.o VBezier.o VDraw.o VDrawText.o VFileIO.o VMath.o VTexture.o VXml.o tim.o tim_io_stdio.o tim_jpeg.o tim_memory_crt.o tim_png.o tinystr.o tinyxml.o tinyxmlerror.o tinyxmlparser.o md5.o md5file.o FileCompression.o SwapEndian.o DBuffer.o CRCHash.o Theme.o WWW.o Editor.o EditorMain.o EditorData.o EditorLog.o Packager.o  -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg

make[1]: Leaving directory `/home/users/sparky/rpm/BUILD/xmoto-0.2.0'

It worked !

Why was it working without --as-needed ?

Answer is really easy: libSDL is required by SDL_mixer:

$ ldd /usr/lib64/libSDL_mixer-1.2.so.0.2.4 | grep SDL
        libSDL-1.2.so.0 => /usr/lib64/libSDL-1.2.so.0 (0x00002ab425307000)

but xmoto-edit contains no SDL_mixer symbols, that's why it wasn't linked with SDL_mixer, and there was nothing providing SDL library.

Can you see now why was it stupid to disable –as-needed ? It worked just perfectly disabling unneeded library !

Finally, fix for this may be:

%configure \
        LIBS="-lSDL -lGL"

or patching configure.in, and the result is:

Wrote: /home/users/sparky/rpm/RPMS/xmoto-0.2.0-2.x86_64.rpm

2. unresolved symbols in library while linking executable

That's the most common and a little more difficult case, evolution-data-server:

Compilation stops at this place:

/bin/sh ../libtool --tag=CC --mode=link x86_64-pld-linux-gcc -ggdb -O2 -Wall -Wmissing-prototypes  -Wno-sign-compare  -Wl,--as-needed -o test-source-selector  test-source-selector.o libedataserverui-1.2.la ../libedataserver/libedataserver-1.2.la -pthread -lglade-2.0 -lgtk-x11-2.0 -lxml2 -lgdk-x11-2.0 -latk-1.0 -lgdk_pixbuf-2.0 -lpangocairo-1.0 -lXext -lXrender -lXinerama -lXi -lXrandr -lXcursor -lXfixes -lcairo -lpangoft2-1.0 -lfontconfig -lfreetype -lz -lpango-1.0 -lX11 -lgnome-2 -lpopt -lbonobo-2 -lgnomevfs-2 -lbonobo-activation -lgconf-2 -lgobject-2.0 -lORBit-2 -lm -lgmodule-2.0 -ldl -lgthread-2.0 -lgnome-keyring -lglib-2.0 -lpthread

x86_64-pld-linux-gcc -ggdb -O2 -ftree-vectorize -ftree-vectorizer-verbose=1 -march=athlon64 -mmmx -msse -msse2 -m3dnow -mfpmath=sse -Wall -Wmissing-prototypes -Wno-sign-compare -Wl,--as-needed -o .libs/test-source-selector test-source-selector.o -pthread ./.libs/libedataserverui-1.2.so [many, many .so and -l libraries] -lpthread

./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_new'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_find_items_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_get_default_keyring_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_attribute_list_free'
./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_get_widget'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_create_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_item_delete_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_item_create_sync'

collect2: ld returned 1 exit status
make[2]: *** [test-source-selector] Error 1

It's different case, now it isn't problem with binary file, but with shared library. And the difficulty is it shows up at executable-linking time, because undefined symbols in libraries are permited.

OK, lets try to fix that library:

$ grep gnome_keyring_item_delete_sync /usr/lib64/lib*.so
Binary file /usr/lib64/libgnome-keyring.so matches
$ grep glade_xml_new /usr/lib64/lib*.so
Binary file /usr/lib64/libglade-2.0.so matches

Editing Makefile we find:

GNOME_KEYRING_LIBS = -lgnome-keyring -lglib-2.0

So add this one and glade to that library deps:

libedataserverui_1_2_la_LIBADD = \
        $(top_builddir)/addressbook/libebook/libebook-1.2.la    \
        $(GNOME_KEYRING_LIBS) -lglade-2.0 \
        $(E_DATA_SERVER_LIBS)

As it uses libtool, remove .la to force regeneration: $ rm libedataserverui-1.2.la and try to build $ make passes without problems.

Now we only need to fix it correcly, after looking at configure.in and Makefile.am one can see it was only a typo:

libedataserverui_1_2_la_LIBADD = \
        $(top_builddir)/addressbook/libebook/libebook-1.2.la    \
        $(E_DATA_SERVER_UI_LIBS)
                        ^^^
Why was it working without --as-needed ?

test-source-selector binary was linked with all libraries needed by libedataserverui-1.2.so, and the binary was the one who provided missing symbols to libedataserverui-1.2.so

3. unresolved symbols caused by incorrect order

Two of most difficult of common problems at once, evolution.spec:

The difficulties are:

  1. it appears in configure, what may be difficult to fix and difficult to find what the real problem is
  2. the problem is not a missing library, but arguments order

configure stops with such message, which says nothing:

checking if pilot-link handles UTF-8 conversions... no
configure: error: evolution requires pilot-link to have working UTF-8 conversion routines

let's look at config.log:

x86_64-pld-linux-gcc -o conftest -ggdb -O2 -DORBIT2=1 -pthread -I/usr/include/libgnome-2.0 [...many -I...] -I/usr/include/libxml2 -Wl,--as-needed -pthread -lgpilotd [...many -l...] -lglib-2.0 conftest.c >&5

/home/users/sparky/tmp/ccgrL9ll.o: In function `main':
/home/users/sparky/rpm/BUILD/evolution-2.7.90/conftest.c:64: undefined reference to `convert_ToPilotChar'

collect2: ld returned 1 exit status

function 'convert_ToPilotChar', may be found in passed library, but take a look at section about positions, arguments order is incorrect: libraries go before objects (conftest.c) it's very common when someone puts -l<> in LDFLAGS instead of LIBS, that was the case too

just take a look at my fix:

-       LDFLAGS_save="$LDFLAGS"
-       LDFLAGS="$LDFLAGS $GNOME_PILOT_LIBS"
+       LIBS_save="$LIBS"
+       LIBS="$LIBS $GNOME_PILOT_LIBS"
[...]
-       LDFLAGS="$LDFLAGS_save"
+       LIBS="$LIBS_save"

with this simple change everything works perfectly.

developingpld/advanceddeveloping/fixingasneeded.1154911272.txt.gz · Last modified: 2006-08-07 02:41 by SamChi