G
George Potemkin
Guest
The customer had two incidents of db down due to the memory violation on a client holding the GST latch: [2018/02/23@10:46:21.000+0000] P-5684 T-1 I ABL 2462: (49) SYSTEM ERROR: Memory violation. [2018/02/23@10:46:21.000+0000] P-5684 T-1 I ABL 2462: (439) ** Save file named core for analysis by Progress Software Corporation. [2018/02/23@10:46:25.000+0000] P-5684 T-1 I ABL 2462: (-----) Generating /home/xxxxxx/protrace.5684 [2018/02/23@16:46:55.591+0600] P-6053 T-1 I WDOG 2462: (5028) SYSTEM ERROR: Releasing regular latch. latchId: 7 [2018/02/23@16:46:55.592+0600] P-6053 T-1 I WDOG 1104: (2522) User 2462 died holding 1 shared memory locks. We know that in both cases the sessions were preparing to quit from database. GST latch = Global Storage Table latch that control an access to the entire shared memory area. Its activity used to be very low compared to the activity of other latches. I guess GST latch is mainly used when Progress needs to change a size of any structure in shared memory by decreasing the free memory. The tests shown: 1. GST latch is activily used only at broker startup. 2. promon/R&D/1/14. Status: Shared Memory Segments screen creates one GST latch lock (of course, without changing shared memory). 3. When new session exceeds the previous maximum for the number of connected sessions or new login broker/remote client server is started then GST latch is locked twice and free shared memory is decreasing by 160 bytes. 4. When a remote session is connecting but it does not increase HWM of connected sessions then GST latch is locked twice but free shared memory is not changing. I guess it's just due to some legacy code used by remote client servers. 5. "Lock table overflow" error locks GST latch only once and free shared memory is decreasing by 88 bytes (= size of Lock Table Entry since 11.4, 14 bytes in V5 ;-). 6. 'proutil -C increaseto' locks GST latch many times and the free shared memory is decreasing when possible or new shared memory segment is added. Of course, it's not the complete list. Monitoring free shared memory during a day shown the changes that are multiples of 448 bytes (=64*7). It can be caused by an increase of HWM of connected sessions but then I would expect the changes that are multiples of 160 bytes (=32*5). Is it a correct assumption that ABL sessions will use GST latch only under the exceptional circumstances like "Lock table overflow" error? There are no reasons to use GST latch while disconnecting from a database, aren't there? I guess the session in my example locked GST latch only due to its "madness" (Memory violation). Protrace file contains: (4) 0xe0000001901cb480 ---- Signal 11 (SIGSEGV) delivered ---- (5) 0x40000000008d7b81 stVacate + 0x161 at /vobs_rkt/src/dbmgr/st/stm.c:803 [/usr/dlc/bin/_progres] (6) 0x4000000000963840 cxDeactivateCursor + 0x150 at /vobs_rkt/src/dbmgr/cx/cxnxt.c:222 [/usr/dlc/bin/_progres] (7) 0x40000000009634c0 dsmCursorDelete + 0x1c0 at /vobs_rkt/src/dbmgr/dsm/dsmcursr.c:1281 [/usr/dlc/bin/_progres] (8) 0x4000000001752430 proixdlc + 0x150 at /vobs_prgs/src/prsys/profnd.c:596 [/usr/dlc/bin/_progres] And I guess the session tried to use an index and according ABL Stack Trace it was the index in /another/ database the session was connected to. Is it a reasonable assumption?
Continue reading...
Continue reading...
