User Tools

Site Tools


developingpld:advanceddeveloping:fixingasneeded

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
developingpld:advanceddeveloping:fixingasneeded [2006-08-07 01:19] – Created from my posts on pld-devel-en list, needs a lot of work SamChidevelopingpld:advanceddeveloping:fixingasneeded [2007-05-27 20:51] (current) arekm
Line 4: Line 4:
  
  
-==== Quick Workaround ==== +===== Quick and ugly workaround (DON'T USE IT!) ===== 
-Put in spec file this line+Put this line in ''spec'' file: 
  
-<file>%devine filterout_ld -Wl,--as-needed 
-</file> 
-But **don't use it** unless you realy need.  
  
-Why ? Because it's very stupid. If something fails to build normally it's a sign ''--as-needed'' did well it's job and disabled unneeded libraries. + 
 +<file>%define filterout_ld -Wl,--as-needed 
 +</file> 
 +But **don't use it** unless you really need to quickly build a package. Never ever try to commit such a change. It might be tempting to do so but if you think about it ''--as-needed'' did it's job well and disabled unneeded libraries. If building fails, it's the package that needs to get fixed, not the linker flags
  
  
  
 ===== How --as-needed works ===== ===== How --as-needed works =====
-With ''--as-needed'' enabled only libraries which contain symbols required by __object files__ are linked. +With ''--as-needed'' enabledonly libraries containing symbols required by __object files__ are linked. 
  
  
  
-== About positions == +==== Why it fails for some packages ==== 
-Linker options are positional, and position is very important+In other wordsif you list 10 libraries and only 3 of them contain referenced symbols, others are silently discarded. This is a good thing but unfortunately some projects fail to list all their dependencies properly
  
-Correct ones are: +This normally works because if project requires libraries //A// and //B// and //A// itself lists //B// as a dependency, both //A// and //B// get pulled in even if the project itself only lists //A//.  
 + 
 +Now assume that //A// is a superfluous (unneeded) dependency which gets dropped (or //A// requiring //B// is superfluous and //B// gets dropped from //A//). The symbols are no longer available.  
 + 
 + 
 + 
 +==== About positions ==== 
 +Linker options are positional, and position is very important. It looks for missing symbols only in libraries coming after object file.  
 + 
 +Correct positions are: 
  
 <file>$ gcc $(LDFLAGS) -o $@ $(OBJECTS) $(LIBS) <file>$ gcc $(LDFLAGS) -o $@ $(OBJECTS) $(LIBS)
Line 29: Line 38:
 and this generally works well.  and this generally works well. 
  
-If there is something like that: + 
 + If there is something like that: 
  
 <file>$ gcc -Wl,--as-needed -l<library> <objects.o> <file>$ gcc -Wl,--as-needed -l<library> <objects.o>
Line 35: Line 45:
 the library will __never__ be linked, because it's not required at the time it's checked.  the library will __never__ be linked, because it's not required at the time it's checked. 
  
-And with something like that: + 
 + And with something like that: 
  
 <file>$ gcc -l<library> -Wl,--as-needed <objects.o> <file>$ gcc -l<library> -Wl,--as-needed <objects.o>
 </file> </file>
-library will allways be linked, even if it's not required. +library will __allways__ be linked, even if it isn'required. 
  
  
  
 ==== Short examples ==== ==== Short examples ====
-Most of problems are with readline/ncurses and tinfo, and it's nice example: tinfo is required by both readline and ncurses, and both are linked with this library (now, when I've fixed readline). +Most of problems are with readline/ncurses and tinfo, and it's nice example: tinfo is required by both readline and ncurses, and both are linked with this library (now, when [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SOURCES/readline-shared.patch?r1=1.6&amp;r2=1.7|fixed readline]] is fixed). 
  
-But some packages link with readline or ncurses while they use __only__ symbols from tinfo.  //Without// ''--as-needed'' those executables work because they are linked with tinfo from readline/ncurses libraries //With// ''--as-needed'' it will not work, because readline/ncurses contain no symbols required by executable so they are not linked, it's dependencies naturally are neither linked. So there is a need to pass ''-ltinfo''. If it requires only symbols from tinfo it's ok to ''s/ncurses/tinfo/''. But if it realy requires readline/ncurses but there is some executable (or ./configure) which requires only tinfo both ''-lreadline''/''-lncurses'' and ''-ltinfo'' should be passed. +But some packages link with readline or ncurses while they use __only__ symbols from tinfo. //Without// ''--as-needed'' those executables work because they are linked with tinfo from readline/ncurses library. //With// ''--as-needed'' linking will not work, because readline/ncurses contain no symbols required by executable so they are not linked, it's dependencies naturally are neither linked. Thats why there is a need to pass ''-ltinfo''. If it requires only symbols from tinfo it's ok to ''s/ncurses/tinfo/''. But if it really requires readline/ncurses but there is some executable (or ./configure) which requires only tinfo both ''-lreadline''/''-lncurses'' and ''-ltinfo'' should be passed.  
 + 
 +For a longer example take a look at real example nr. 1
  
  
Line 57: Line 70:
 <file>$ gcc -Wl,--as-needed -o executable <objects.o> -l2 -l1 <file>$ gcc -Wl,--as-needed -o executable <objects.o> -l2 -l1
 </file> </file>
-and normally objects require only library 2, and -l1 is required by -l2. As I said it checks only for symbols from objects, so -l1 is not linked. Normally it is easy to fix it, simply make sure while linking -l2 it is linked to -l1. +It's usual situation when object files require only library 2, and //-l1// is required by //-l2//. As I said linker only checks for symbols from objects, so //-l1// is not linked. Normally it is easy to fix it, simply make sure while linking //-l2// it is linked to //-l1// (take a look at the second real example)
  
  
  
-===== Examples =====+===== Real examples =====
  
  
 ==== 1. unresolved symbols in executable ==== ==== 1. unresolved symbols in executable ====
-xmoto +Very common situation, [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/xmoto.spec?rev=HEAD|xmoto.spec]]. Actually, it's nothing new. Every linking behaves this way if some library is missing. 
  
 With ''-Wl,--as-needed'' enabled it stops on something like this:  With ''-Wl,--as-needed'' enabled it stops on something like this: 
Line 71: Line 84:
  
  
-<file>x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\"  -ggdb -O2 -Wl,--as-needed -o xmoto-edit  BuiltInFont.o [...object files...] Packager.o -lGL -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lbz2 -lz -lpng -ljpeg+<file>g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o -lGL -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lbz2 -lz -lpng -ljpeg
  
 VApp.o: In function `vapp::App::getRealTime()': VApp.o: In function `vapp::App::getRealTime()':
Line 96: Line 109:
 Binary file /usr/lib64/libSDL.so matches Binary file /usr/lib64/libSDL.so matches
 </file> </file>
-they are in ''-lSDL'', but binary does not link with ''-lSDL''; edit Makefile by hand and add ''-lSDL'' at the same place ''-lSDL_mixer'' is: ''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg'' +They are in ''-lSDL'', but binary does not link with ''-lSDL''. Edit Makefile by hand and add ''-lSDL'' at the same place ''-lSDL_mixer'' is: ''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg'' 
  
 What we get after running make in build tree:  What we get after running make in build tree: 
Line 102: Line 115:
  
  
-<file>x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\" -ggdb -O2 -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o  -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg +<file>g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg
  
 VApp.o: In function `vapp::App::grabScreen()': VApp.o: In function `vapp::App::grabScreen()':
Line 124: Line 137:
 ''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg''  ''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg'' 
  
-But take a look at spec file, -lGL thing was fixed there already: +But [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/xmoto.spec?rev=1.21|take a look at spec file]]''-lGL'' thing was fixed there already: 
  
  
Line 135: Line 148:
  
  
-<file>x86_64-pld-linux-g++ -DNOMMGR -Wall -DGAMEDATADIR=\"/usr/share/xmoto\"  -mfpmath=sse -ggdb -O2 -ftree-vectorize -ftree-vectorizer-verbose=1 -march=athlon64 -mmmx -msse -msse2 -m3dnow  -s -Wl,--as-needed -Wl,-s  -o xmoto-edit  BuiltInFont.o Image.o LevelSrc.o VApp.o VBezier.o VDraw.o VDrawText.o VFileIO.o VMath.o VTexture.o VXml.o tim.o tim_io_stdio.o tim_jpeg.o tim_memory_crt.o tim_png.o tinystr.o tinyxml.o tinyxmlerror.o tinyxmlparser.o md5.o md5file.o FileCompression.o SwapEndian.o DBuffer.o CRCHash.o Theme.o WWW.o Editor.o EditorMain.o EditorData.o EditorLog.o Packager.o  -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg+<file>g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...Packager.o -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg
  
 make[1]: Leaving directory `/home/users/sparky/rpm/BUILD/xmoto-0.2.0' make[1]: Leaving directory `/home/users/sparky/rpm/BUILD/xmoto-0.2.0'
 </file> </file>
-//It worked !// +**It worked !** 
  
  
  
 == Why was it working without --as-needed ? == == Why was it working without --as-needed ? ==
-Answer is realy easy: libSDL is required by SDL_mixer: +Answer is really easy: libSDL is required by SDL_mixer: 
  
  
Line 151: Line 164:
         libSDL-1.2.so.0 => /usr/lib64/libSDL-1.2.so.0 (0x00002ab425307000)         libSDL-1.2.so.0 => /usr/lib64/libSDL-1.2.so.0 (0x00002ab425307000)
 </file> </file>
-but xmoto-edit contains no SDL_mixer symbols, that's why xmoto-edit wasn't linked with SDL_mixer, and nothing provided SDL library. +but xmoto-edit contains no SDL_mixer symbols, that's why it wasn't linked with SDL_mixer, and there was nothing providing SDL library. 
  
-Can you see now why was it so stupid to disable ''--as-needed'' ? It worked just perfectly disabling unneeded library ! +Can you see now why was it stupid to disable ''--as-needed'' ? It worked just perfectly disabling unneeded library ! 
  
  
Line 163: Line 176:
         LIBS="-lSDL -lGL"         LIBS="-lSDL -lGL"
 </file> </file>
-or patching configure.in and the result is: +or patching configure.inand the result is: 
  
-Wrote: /home/users/sparky/rpm/RPMS/xmoto-0.2.0-2.x86_64.rpm +''Wrote: /home/users/sparky/rpm/RPMS/xmoto-0.2.0-2.x86_64.rpm'' 
  
  
  
 ==== 2. unresolved symbols in library while linking executable ==== ==== 2. unresolved symbols in library while linking executable ====
-That's the most common and a little more difficult case, evolution-data-server: +That's the most common and a little more difficult case, [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/evolution-data-server.spec?rev=HEAD|evolution-data-server]]
  
 Compilation stops at this place:  Compilation stops at this place: 
Line 176: Line 189:
  
  
-<file>/bin/sh ../libtool --tag=CC --mode=link x86_64-pld-linux-gcc -ggdb -O2 -Wall -Wmissing-prototypes  -Wno-sign-compare  -Wl,--as-needed -o test-source-selector  test-source-selector.o libedataserverui-1.2.la ../libedataserver/libedataserver-1.2.la -pthread -lglade-2.0 -lgtk-x11-2.0 -lxml2 -lgdk-x11-2.0 -latk-1.-lgdk_pixbuf-2.0 -lpangocairo-1.0 -lXext -lXrender -lXinerama -lXi -lXrandr -lXcursor -lXfixes -lcairo -lpangoft2-1.0 -lfontconfig -lfreetype -lz -lpango-1.0 -lX11 -lgnome-2 -lpopt -lbonobo-2 -lgnomevfs-2 -lbonobo-activation -lgconf-2 -lgobject-2.0 -lORBit-2 -lm -lgmodule-2.0 -ldl -lgthread-2.0 -lgnome-keyring -lglib-2.0 -lpthread+<file>/bin/sh ../libtool --tag=CC --mode=link gcc -Wl,--as-needed -o test-source-selector  test-source-selector.o libedataserverui-1.2.la ../libedataserver/libedataserver-1.2.la -pthread -lglade-2.0 [...many -l libraries...
 +-lgnome-keyring -lpthread
  
-x86_64-pld-linux-gcc -ggdb -O2 -ftree-vectorize -ftree-vectorizer-verbose=1 -march=athlon64 -mmmx -msse -msse2 -m3dnow -mfpmath=sse -Wall -Wmissing-prototypes -Wno-sign-compare -Wl,--as-needed -o .libs/test-source-selector test-source-selector.o -pthread ./.libs/libedataserverui-1.2.so [many, many .so and -l libraries] -lpthread+gcc -Wl,--as-needed -o .libs/test-source-selector test-source-selector.o -pthread ./.libs/libedataserverui-1.2.so [...many, many .so and -l libraries...] -lpthread
  
 ./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_new' ./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_new'
Line 207: Line 221:
 ''GNOME_KEYRING_LIBS = -lgnome-keyring -lglib-2.0''  ''GNOME_KEYRING_LIBS = -lgnome-keyring -lglib-2.0'' 
  
-So add this one and glade to that library deps: +So add this one and ''-lglade-2.0'' to that library deps: 
  
  
Line 230: Line 244:
  
 == Why was it working without --as-needed ? == == Why was it working without --as-needed ? ==
-test-source-selector binary was linked with all libraries needed by libedataserverui-1.2.so, and the binary was the one who provided missing symbols to libedataserverui-1.2.so +''test-source-selector'' binary was linked with all libraries needed by libedataserverui-1.2.so, and the binary was the one who provided missing symbols to libedataserverui-1.2.so  
 + 
 + 
 + 
 +=== What if broken library comes from other package ? === 
 +You can check manually does library has all required symbols, simply running ' gcc //library// ', like this:  
 + 
 +This is an example of correctly linked library:  
 + 
 +<file>$ gcc /usr/lib64/liblftp-tasks.so.0.0.0 
 +/usr/lib64/gcc/x86_64-pld-linux/4.1.2/../../../../lib64/crt1.o: In function `_start': 
 +(.text+0x20): undefined reference to `main' 
 +collect2: ld returned 1 exit status 
 +</file> 
 +And incorrectly linked one:  
 + 
 +<file>$ gcc /usr/lib64/liblftp-jobs.so.0.0.0 
 +/usr/lib64/gcc/x86_64-pld-linux/4.1.2/../../../../lib64/crt1.o: In function `_start': 
 +(.text+0x20): undefined reference to `main' 
 +/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `StringSet::Remove(int)' 
 +/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `Buffer::Format(char const*, ...)' 
 +[...] 
 +/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `SMTask::SuspendSlave()' 
 +collect2: ld returned 1 exit status 
 +</file> 
 +**Warning:** some libraries may require to have unresolved symbols. 
  
  
  
-==== 3. ==== +==== 3. unresolved symbols caused by incorrect order ==== 
-Two of most difficult of common problems at one, evolution.spec: +Two of most difficult of common problems at once[[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/evolution.spec?rev=HEAD|evolution.spec]]
  
 The difficulties are:  The difficulties are: 
Line 253: Line 292:
  
  
-<file>x86_64-pld-linux-gcc -o conftest -ggdb -O2 -DORBIT2=1 -pthread -I/usr/include/libgnome-2.0 [...many -I...] -I/usr/include/libxml2 -Wl,--as-needed -pthread -lgpilotd [...many -l...] -lglib-2.0 conftest.c >&5+<file>gcc -o conftest -ggdb -O2 -DORBIT2=1 -pthread -I/usr/include/libgnome-2.0 [...many -I...] -I/usr/include/libxml2 -Wl,--as-needed -pthread -lgpilotd [...many -l...] -lglib-2.0 conftest.c >&5
  
 /home/users/sparky/tmp/ccgrL9ll.o: In function `main': /home/users/sparky/tmp/ccgrL9ll.o: In function `main':
Line 260: Line 299:
 collect2: ld returned 1 exit status collect2: ld returned 1 exit status
 </file> </file>
-function '''convert_ToPilotChar''', may be found in passed library, but take a look at section about positions, arguments order is incorrect: libraries go before objects (conftest.c) it's very common when someone puts ''-l<>'' in ''LDFLAGS'' instead of ''LIBS'', that was the case too +function '''convert_ToPilotChar''', may be found in passed library, but take a look at section about positions, arguments order is incorrect: libraries go before objects (conftest.c). It's very common when someone puts ''-l<>'' in ''LDFLAGS'' instead of ''LIBS'', that was the case too
  
-just take a look at my fix: +Just take a look at my fix: 
  
  
Line 274: Line 313:
 +       LIBS="$LIBS_save" +       LIBS="$LIBS_save"
 </file> </file>
-with this simple change everything works perfectly +With this simple change everything works perfectly
  
  
developingpld/advanceddeveloping/fixingasneeded.1154906347.txt.gz · Last modified: 2006-08-07 01:19 by SamChi

Except where otherwise noted, content on this wiki is licensed under the following license: Public Domain
Public Domain Donate Powered by PHP Valid HTML5 Valid CSS PLD Linux Driven by DokuWiki