Opened 14 years ago

Closed 12 years ago

Last modified 12 years ago

#2402 closed defect (fixed)

qmake build system has bad dependencies which causes --as-needed and future binutils to fail to link

Reported by: cardoe@… Owned by: Nigel
Priority: minor Milestone: unknown
Component: mythtv Version: 0.20
Severity: medium Keywords:
Cc: Ticket locked: no


The build system has dependencies on libraries incorrect and poorly laid out. This causes anyone compiling with --as-needed to have missing symbols and libraries aren't built in the correct order for resolving dependencies since apparently qmake respects order rather then dependencies.

The following patch is for release-0_20-fixes, it cleans this whole mess up.

Attachments (2)

mythtv-0.20-as-needed.patch (4.2 KB) - added by anonymous 14 years ago.
mythtv-0.20-as-needed.2.patch (4.2 KB) - added by cardoe@… 14 years ago.

Download all attachments as: .zip

Change History (17)

Changed 14 years ago by anonymous

Attachment: mythtv-0.20-as-needed.patch added

Changed 14 years ago by cardoe@…

comment:1 Changed 14 years ago by Nigel

Hi Doug. I have never used '--as-needed' (it is not an option in my ld), but yes, the dependencies need fixing. Sadly, your re-ordering of SUBDIRS causes link failure on Mac OS X:

rm -f libmythui- libmythui-0.20.dylib libmythui-0.20.0.dylib libmythui-
g++ -headerpad_max_install_names -Wl,-search_paths_first -prebind -dynamiclib -seg1addr 0xCC000000 -compatibility_version       0.20 -current_version   0.20.0 -install_name    libmythui-0.20.0.dylib -o libmythui- mythmainwindow.o mythpainter.o mythimage.o myththemebase.o mythpainter_qt.o xmlparsebase.o mythscreenstack.o mythscreentype.o mythgesture.o mythuitype.o mythuiimage.o mythuitext.o mythuistatetype.o mythlistbutton.o mythfontproperties.o mythuibutton.o myththemedmenu.o mythdialogbox.o moc_mythmainwindow.o moc_mythuitype.o moc_mythlistbutton.o moc_myththemedmenu.o moc_mythdialogbox.o  -L/Users/nigel/.osx-packager/src/qt-mac-free-3.3.6/lib -L/usr/lib -L/Users/nigel/.osx-packager/build/lib -framework Carbon -framework OpenGL -lqt-mt 
ld: warning prebinding disabled because dependent library: libqt-mt.3.dylib is not prebound
ld: Undefined symbols:
/usr/bin/libtool: internal link edit command failed
make[2]: *** [libmythui-] Error 1
make[1]: *** [sub-libmythui] Error 2

because there are actually circular dependencies. I am not a C++ expert, so I have been thinking about this for several months (since libmythui started being used) and can only think of three fixes:

  1. Re-engineer the classes so that MythDialog is moved into libmythui, and there is yet another source directory and library for all the fundamental Myth stuff (like MSqlQuery, MythContext, MythMediaMonitor, MythPluginManager), or
  1. Change the build system to make one big library, or
  1. Use late binding on Mac OS X

Only the first option is a proper fix

comment:2 Changed 14 years ago by anonymous


I'm well aware that there are circular dependencies. I've been saying that as far back as 0.18 or 0.17. There's A LOT of code that needs to get cleaned up. I've posted this here for Linux users with new binutils that use --as-needed since this will work on there systems. There is no guarentee it will work on other systems. I highlighted the Isaac the circular depends the other night.

comment:3 Changed 13 years ago by candrews@…

Is there any news on this ticket? I believe multiple distros now use --as-needed, so I assume this could be becoming a bigger issue, and the cause of more distro provided patches.

comment:4 Changed 13 years ago by Nigel

I am working on it in #4264

comment:5 Changed 13 years ago by Nigel

Owner: changed from Isaac Richards to Nigel
Status: newassigned

comment:6 Changed 12 years ago by Nigel

Hmm. I just hacked a build machine's Qt to use ld with --as-needed, deleted all libs and did a build of both 0-20-fixes and current SVN head. No linker errors:

% make
cd libmyth && make -f Makefile
make[2]: Entering directory `/home/nigel/mythtv/libs/libmyth'
rm -f
g++ -shared -v -Wl,--as-needed,--verbose -Wl,-soname, -o
GNU ld version 2.17 Debian GNU/Linux
attempt to open /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/crtn.o succeeded
ln -s
ln -s
ln -s
make[2]: Leaving directory `/home/nigel/mythtv/libs/libmyth'

I have looked at the binutils README, but there don't seem to be any real changes with --as-needed in 2.18 (except for a helper script). Am I going mad? Or does --as-needed do nothing different?

comment:7 Changed 12 years ago by Nigel

Resolution: worksforme
Status: assignedclosed

I tested again, on a 64bit machine which has binutils 2.18. Again, no problems in SVN head or 0-20-fixes.

comment:8 Changed 12 years ago by candrews@…

Resolution: worksforme
Status: closednew

I have this line in my /usr/qt/3/mkspecs/linux-g++/qmake.conf file: QMAKE_LFLAGS_RELEASE=-Wl,--hash-style=both,-z,relro,--enable-new-dtags,-O1,-z,now,--as-needed

My machine is an AMD64 running GCC 4.2.2, binutils 2.18, and qt 3.3.8.

When I build mythtv 15429, this is the error I get: g++ -c -pipe -fomit-frame-pointer -O3 -Wall -Wno-switch -Wpointer-arith -Wredundant-decls -Wno-non-virtual-dtor -DSTDC_CONSTANT_MACROS -D_REENTRANT -DMMX -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DPREFIX=\"/usr\" -DLIBDIR=\"/usr/lib64\" -DUSING_X11 -DUSING_XV -DUSING_IVTV -DUSING_XVMC -DUSING_XVMC_VLD -DUSING_XRANDR -DUSING_OPENGL_VSYNC -DUSING_ALSA -DUSING_OSS -DQT_NO_DEBUG -DQT_THREAD_SUPPORT -DQT_SHARED -DQT_TABLET_SUPPORT -I/usr/qt/3/mkspecs/linux-g++ -I. -I/usr/include -I/usr/include -I../.. -I../../libs -I../../libs/libmyth -I../../libs/libmythtv -I../../libs/libavutil -I../../libs/libavformat -I../../libs/libavcodec -I../../libs/libmythupnp -I../../libs/libmythui -I../../libs/libmythlivemedia -I/usr/qt/3/include -I/usr/X11R6/include -I/usr/X11R6/include -o manualschedule.o manualschedule.cpp g++ -c -pipe -fomit-frame-pointer -O3 -Wall -Wno-switch -Wpointer-arith -Wredundant-decls -Wno-non-virtual-dtor -DSTDC_CONSTANT_MACROS -D_REENTRANT -DMMX -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DPREFIX=\"/usr\" -DLIBDIR=\"/usr/lib64\" -DUSING_X11 -DUSING_XV -DUSING_IVTV -DUSING_XVMC -DUSING_XVMC_VLD -DUSING_XRANDR -DUSING_OPENGL_VSYNC -DUSING_ALSA -DUSING_OSS -DQT_NO_DEBUG -DQT_THREAD_SUPPORT -DQT_SHARED -DQT_TABLET_SUPPORT -I/usr/qt/3/mkspecs/linux-g++ -I. -I/usr/include -I/usr/include -I../.. -I../../libs -I../../libs/libmyth -I../../libs/libmythtv -I../../libs/libavutil -I../../libs/libavformat -I../../libs/libavcodec -I../../libs/libmythupnp -I../../libs/libmythui -I../../libs/libmythlivemedia -I/usr/qt/3/include -I/usr/X11R6/include -I/usr/X11R6/include -o programrecpriority.o programrecpriority.cpp ../../libs/libmyth/ undefined reference to `XmlConfiguration::XmlConfiguration?(QString const&)' ../../libs/libmyth/ undefined reference to `MythXMLClient::~MythXMLClient()' ../../libs/libmyth/ undefined reference to `UPnp::UPnp()' ../../libs/libmyth/ undefined reference to `SSDPCache::Find(QString const&)' ../../libs/libmyth/ undefined reference to `SSDPCache::Find(QString const&, QString const&)' ../../libs/libmyth/ undefined reference to `MythXMLClient::MythXMLClient(QUrl const&, bool)' ../../libs/libmyth/ undefined reference to `UPnp::SetConfiguration?(Configuration*)' ../../libs/libmyth/ undefined reference to `UPnp::Initialize(int, HttpServer?*)' ../../libs/libmyth/ undefined reference to `UPnpDeviceDesc::Retrieve(QString&, bool)' ../../libs/libmyth/ undefined reference to `UPnp::g_SSDPCache' ../../libs/libmyth/ undefined reference to `SSDP::PerformSearch?(QString const&)' ../../libs/libmyth/ undefined reference to `HttpServer::HttpServer?(int)' ../../libs/libmyth/ undefined reference to `UPnp::g_UPnpDeviceDesc' ../../libs/libmyth/ undefined reference to `MythXMLClient::GetConnectionInfo?(QString const&, DatabaseParams?*, QString&)' ../../libs/libmyth/ undefined reference to `UPnp::g_pConfig' ../../libs/libmyth/ undefined reference to `UPnp::g_pSSDP' collect2: ld returned 1 exit status make[2]: * [mythtv] Error 1 make[2]: Leaving directory `/var/tmp/portage/media-tv/mythtv-0.21_pre15429/work/mythtv-0.21/programs/mythtv' make[1]: * [sub-mythtv] Error 2 make[1]: * Waiting for unfinished jobs....

If I remove the "--as-needed" from the QMAKE_LDFLAGS_RELEASE line in qmake.conf, mythtv builds and runs fine.

comment:9 Changed 12 years ago by Nigel

Hooray. Someone actually showed me the real errors they are getting!
From the patches, I had assumed the problems were in the library linking (closure), not the program linking.
I'm not sure this will be fixable, because --as-needed just seems to be broken on the link line. Check out this output after I pruned LIBS down to -lGL -lmyth -lmythui:

/usr/bin/ld --eh-frame-hdr -m elf_i386 --hash-style=both -dynamic-linker /lib/ -o mythtv /usr/lib/gcc/i486-linux-gnu/4.1.3/../../../../lib/crt1.o /usr/lib/gcc/i486-linux-gnu/4.1.3/../../../../lib/crti.o /usr/lib/gcc/i486-linux-gnu/4.1.3/crtbegin.o -L/usr/share/qt3/lib -L../../libs/libmyth -L../../libs/libmythui -L/usr/lib/gcc/i486-linux-gnu/4.1.3 -L/usr/lib/gcc/i486-linux-gnu/4.1.3 -L/usr/lib/gcc/i486-linux-gnu/4.1.3/../../../../lib -L/lib/../lib -L/usr/lib/../lib --as-needed -v main.o -lmyth-0.20.2 -lGL -lmythui-0.20.2 -lqt-mt -lXext -lX11 -lpthread -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/i486-linux-gnu/4.1.3/crtend.o /usr/lib/gcc/i486-linux-gnu/4.1.3/../../../../lib/crtn.o
GNU ld (GNU Binutils for Debian) 2.18
main.o: In function `main':
main.cpp:(.text+0x3c86): undefined reference to `TV::InitKeys()'
main.cpp:(.text+0x3c9c): undefined reference to `TV::TV()'
main.cpp:(.text+0x3cad): undefined reference to `TV::Init(bool)'
main.cpp:(.text+0x431e): undefined reference to `ProgramInfo::ProgramInfo()'
main.cpp:(.text+0x43e1): undefined reference to `TV::Playback(ProgramInfo*)'
main.cpp:(.text+0x43ff): undefined reference to `TV::GetState() const'
main.cpp:(.text+0x449b): undefined reference to `TV::LiveTV(bool, bool)'
../../libs/libmythui/ undefined reference to `glClearDepth'
../../libs/libmythui/ undefined reference to `glMatrixMode'
../../libs/libmythui/ undefined reference to `glHint'...
% objdump -T /usr/lib/ | grep glClearDepth
0004ace0 g    DF .text  00000000  Base        glClearDepth

libGL is parsed first, it contains the symbols, but 'ld --as-needed' doesn't find it. I'm at a loss (again).

comment:10 Changed 12 years ago by Nigel

Status: newassigned

comment:11 Changed 12 years ago by Nigel

OK. I have worked it out. Linux ld --as-needed uses the dependencies that are compiled into the libraries that it is parsing (like Solaris's ld -R, except that they are used at runtime). Several checkins on the way!

comment:12 Changed 12 years ago by Nigel

(In [15472]) First of several Linux 'ld --as-needed' patches. Thanks to Doug (cardoe at See #2402.

comment:13 Changed 12 years ago by Nigel

(In [15476]) Reorder libs to get around ld --as-needed problems. See #2402. Basically, build libmythui earlier, and libmythtv later.

comment:14 Changed 12 years ago by Nigel

Resolution: fixed
Status: assignedclosed

(In [15477]) Also rebuild libmythtv if libmythui changes. Closes #2402

comment:15 Changed 12 years ago by anonymous

this cannot be marked as fixed, problem still there in last svn revisions

Note: See TracTickets for help on using tickets.