Issue 79500 - HelpLinker crash on building khmer content
Summary: HelpLinker crash on building khmer content
Status: CLOSED FIXED
Alias: None
Product: Build Tools
Classification: Code
Component: code (show other issues)
Version: 680m219
Hardware: PC Linux, all
: P1 (highest) Trivial (vote)
Target Milestone: OOo 2.3
Assignee: caolanm
QA Contact: issues@tools
URL:
Keywords:
: 79307 (view as issue list)
Depends on:
Blocks:
 
Reported: 2007-07-11 20:26 UTC by pavel
Modified: 2007-07-13 08:32 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
does this work for everyone (2.07 KB, patch)
2007-07-12 10:00 UTC, caolanm
no flags Details | Diff
patch, + hjs problem (2.22 KB, patch)
2007-07-12 13:01 UTC, caolanm
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description pavel 2007-07-11 20:26:00 UTC
Hi,

I do build for many language and my build stopped in module helpcontent2, because HelpLinekr 
crashed for language km.

My tree contains this GSI file merged in:

http://ftp.linux.cz/pub/localization/OpenOffice.org/devel/build/Files/SRC680/GSI_km.sdf.bz2

I now have a file, that contains 910 input files and helplinker:

pavel@oo:~/BuildDir/ooo_SRC680_m220_src/helpcontent2/util/swriter> HelpLinker @/tmp/qqq
Making /disk2/pavel/BuildDir/ooo_SRC680_m220_src/helpcontent2/unxlngi6.pro/bin/swriter_km.zip 
from 910 input files
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
............................................................................................................Segmentation fault
pavel@oo:~/BuildDir/ooo_SRC680_m220_src/helpcontent2/util/swriter> 

Running HelpLinker with the temp file containing only one file in one go was OK for them all:

pavel@oo:~/BuildDir/ooo_SRC680_m220_src/helpcontent2/util/swriter> for i in `cat /tmp/qqq.list`; 
do cat /tmp/qqq.header >/tmp/runme; echo $i >>/tmp/runme; cat /tmp/qqq.footer >>/tmp/runme; 
HelpLinker @/tmp/runme 2>&1 | grep done >/dev/null|| echo $i; done
pavel@oo:~/BuildDir/ooo_SRC680_m220_src/helpcontent2/util/swriter> 

GDB:

pavel@oo:~/BuildDir/ooo_SRC680_m220_src/helpcontent2/util/swriter> gdb HelpLinker 
GNU gdb 5.3.92
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i586-suse-linux"...
(gdb) run @/tmp/qqq
Starting program: /disk2/pavel/BuildDir/ooo_SRC680_m220_src/solver/680/unxlngi6.pro/bin/
HelpLinker @/tmp/qqq
setenv: Too many arguments.
[New Thread 16384 (LWP 7144)]
[New Thread 32769 (LWP 7168)]
[New Thread 16386 (LWP 7169)]
Making /disk2/pavel/BuildDir/ooo_SRC680_m220_src/helpcontent2/unxlngi6.pro/bin/swriter_km.zip 
from 909 input files
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
................................................................................................................................................................
............................................................................................................
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 16384 (LWP 7144)]
0x08066cd0 in FullBtreeDict::insert(FullDictBlock&, boost::shared_ptr<Entry>) (this=0x41708008, 
bl=@0x2008, ent=
        {px = 0x416f8068, pn = {pi_ = 0x41712040}}) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/
xmlhelp/source/com/sun/star/help/HelpLinker.cxx:1519
1519        if (bl._isLeaf)
(gdb) where
#0  0x08066cd0 in FullBtreeDict::insert(FullDictBlock&, boost::shared_ptr<Entry>) (this=0x41708008, 
bl=@0x2008, ent=
        {px = 0x416f8068, pn = {pi_ = 0x41712040}}) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/
xmlhelp/source/com/sun/star/help/HelpLinker.cxx:1519
#1  0x08066df0 in FullBtreeDict::insert(FullDictBlock&, boost::shared_ptr<Entry>) (this=0x41708008, 
bl=@0x41708490, ent=
        {px = 0x416f8068, pn = {pi_ = 0x41712040}}) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/
xmlhelp/source/com/sun/star/help/HelpLinker.cxx:1526
#2  0x08066df0 in FullBtreeDict::insert(FullDictBlock&, boost::shared_ptr<Entry>) (this=0x41708008, 
bl=@0x41cc30e4, ent=
        {px = 0x416f8068, pn = {pi_ = 0x41712040}}) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/
xmlhelp/source/com/sun/star/help/HelpLinker.cxx:1526
#3  0x080670c1 in FullBtreeDict::store(_STL::basic_string<char, _STL::char_traits<char>, 
_STL::allocator<char> > const&, int) (this=0x41708008, 
    bla=@0xbfc0b540, id=19154) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/xmlhelp/source/
com/sun/star/help/HelpLinker.cxx:1553
#4  0x080684f6 in Index::intern(_STL::basic_string<char, _STL::char_traits<char>, 
_STL::allocator<char> > const&) (this=0x416f3388, name=@0xbfc0b540)
    at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/xmlhelp/source/com/sun/star/help/
HelpLinker.cxx:2244
#5  0x08077f58 in IndexAdapter::intern(_STL::basic_string<char, _STL::char_traits<char>, 
_STL::allocator<char> > const&) (this=0x41716010, 
    name=@0xbfc0b540) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/xmlhelp/source/com/sun/
star/help/HelpLinker.cxx:3543
#6  0x0806cc5c in IndexAdapter::storeLocation(_STL::basic_string<char, _STL::char_traits<char>, 
_STL::allocator<char> > const&, int) (this=0x41716010, 
    token=@0xbfc0b540, number=6) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/xmlhelp/source/
com/sun/star/help/HelpLinker.cxx:3565
#7  0x08077f9b in IndexAdapter::storeLocation(_STL::basic_string<char, _STL::char_traits<char>, 
_STL::allocator<char> > const&) (this=0x41716010, 
    token=@0xbfc0b540) at /disk2/pavel/BuildDir/ooo_SRC680_m220_src/xmlhelp/source/com/sun/
star/help/HelpLinker.cxx:3508
Comment 1 pavel 2007-07-11 22:48:07 UTC
Exactly the same crash on unxmacxi.pro.
Comment 2 pavel 2007-07-11 22:57:35 UTC
Other languages are OK. Only km looks somehow problematic.

Caolan: do you haev any idea?
Comment 3 pavel 2007-07-12 06:09:08 UTC
*** Issue 79307 has been marked as a duplicate of this issue. ***
Comment 4 gregor.hartmann 2007-07-12 06:26:47 UTC
pavel, there are many gsicheck problems in that file. Maybe thats the reason the
linker crashes. some of them can be fixed automatically with latest gsicheck
(included in your build already) but many would have to get fixed manually
Comment 5 pavel 2007-07-12 06:34:13 UTC
gh: may be. But this is in my opinion different problem.

Maho reported it even on the merged Khmer translations...
Comment 6 pavel 2007-07-12 06:46:37 UTC
rt: can you please reproduce this in your environment as well?

Just build Khmer (km) helpcontent2.
Comment 7 caolanm 2007-07-12 08:55:12 UTC
km eh, let me configure to build that and I'll take a look.
Comment 8 hjs 2007-07-12 09:35:38 UTC
hmm, error 139 on solaris intel, sbasic_en-US.zip...
maybe it's just having lots of languages?
Comment 9 hjs 2007-07-12 09:36:14 UTC
btw., a cws based on m219
Comment 10 hjs 2007-07-12 10:00:13 UTC
something unrelated here. dies in ucnv_close in the dtor of tokenizer.
Comment 11 caolanm 2007-07-12 10:00:56 UTC
Created attachment 46716 [details]
does this work for everyone
Comment 12 caolanm 2007-07-12 10:04:19 UTC
The code in the HelpLinker is lifted from the java one, so I suspect that the
same thing happens in the java one, except perhaps silently caught as an
exception somewhere.

khmer has no direct space thing, so here we get some really big strings, for
whatever original impl reasons we're limited to 250 char long index strings. If
bigger we clip them, but we look them up in the cache before clipping, leading
to a string being mapped to an id, but not being found under that id later. 

So, if we're going to clip a string, we need to do it before we look it up
and/or subsequently store it.
Comment 13 pavel 2007-07-12 10:48:13 UTC
km build OK here with the patch - thanks!
Comment 14 rt 2007-07-12 11:05:49 UTC
Tested on Solaris Sparc:
build in unmodified environment breaks as described. Patch helps, dmake succeeds
now.

Masterfix?
Comment 15 caolanm 2007-07-12 13:00:35 UTC
masterfix sounds good to me. 

but I suggest the following patch. hjs reports an additional problem with the
static object dtor on solaris. I see no reason why there *should* be a problem,
but there are no ill side-effects of skipping dropping the troublesome resources
on exit.
Comment 16 caolanm 2007-07-12 13:01:08 UTC
Created attachment 46722 [details]
patch, + hjs problem
Comment 17 ivo.hinkelmann 2007-07-12 14:25:21 UTC
fixed in master
Comment 18 pavel 2007-07-13 08:32:57 UTC
Khmer crash is gone in SRC680_m221.

Closing.