Mike
Moderator
Server :- HP-UX
Openedge version: - 11.2
We had a situation where all AI files become full and AI demon was in hung state. We tried make empty full ais but does not work, Db had down. So we re-boot the server. and got date time mismatch. Db got corrupt after re-boot or before no idea.Seeking for root cause analyst\ . Please help. Apologies coz Db logs in Spanish so i just took Few logs.
2023/08/18@09:54:05.052-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:10.063-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:10.063-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:15.072-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:15.073-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:15.964-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 17.
[2023/08/18@09:54:15.967-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 18.
[2023/08/18@09:54:15.968-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 20.
[2023/08/18@09:54:15.968-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 21.
[2023/08/18@09:54:15.969-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 22.
[2023/08/18@09:54:15.969-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 23.
[2023/08/18@09:54:15.970-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 25.
[2023/08/18@09:54:15.971-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 27.
[2023/08/18@09:54:15.971-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 29.
[2023/08/18@09:54:15.972-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 30.
[2023/08/18@09:54:15.972-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 31.
[2023/08/18@09:54:15.973-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 32.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 34.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 34: (2252) Beginning of transaction rollback.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 34: (2253) Transaction rollback has finished.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 35.
[2023/08/18@09:54:15.975-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 36.
[2023/08/18@09:54:15.976-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 37.
[2023/08/18@09:54:15.976-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 39.
[2023/08/18@09:54:20.084-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:20.084-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:25.094-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/chl/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:25.094-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:59:43.428-0400] P-11391 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@09:59:43.439-0400] P-11391 T-1 I ABL : (886) ** The database was last used on Fri Aug 18 05:20:03 2023.
[2023/08/18@09:59:43.440-0400] P-11391 T-1 I ABL : (887) ** Previous-image file expected Fri Aug 18 08:48:39 2023.
[2023/08/18@09:59:43.444-0400] P-11391 T-1 I ABL : (888) ** The dates do not match, this indicates that you have an incorrect copy of one of them.
[2023/08/18@09:59:43.445-0400] P-11391 T-1 I ABL : (334) Single-user session end.
[2023/08/18@09:59:45.734-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
Fri Aug 18 09:59:43 2023
[2023/08/18@09:59:43.428-0400] P-11391 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@09:59:43.439-0400] P-11391 T-1 I ABL : (886) ** The database was last used on Fri Aug 18 05:20:03 2023.
[2023/08/18@09:59:43.440-0400] P-11391 T-1 I ABL : (887) ** Previous-image file expected Fri Aug 18 08:48:39 2023.
[2023/08/18@09:59:43.444-0400] P-11391 T-1 I ABL : (888) ** The dates do not match, this in
6:02.757-0400] P-20592 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@10:36:02.766-0400] P-20592 T-1 I ABL : (5326) Begin physical redo phase at 512 .
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16793) BEGIN RL Control Structure Dump
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16794) Cluster Data | Bi Blocksize: 16384
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16795) rlcurr: 512 rlclused: 0 curBlk: 0 curOfst: 0 nxtBlk: 0 nxtOfst: 0
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16796) rlctr: 1934 rlsize: 2560 left: 1536 right: 1024 rlcap: 8376320 bytes: 8388608 lastblk: 0 lastOfst: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16797) Recovery Data
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16798) TranTableSize: 172 XIDTableSize: 100
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16799) RedoBlk: 939 RedoOfst: 3862 ReadBlk: 0 ReadOfst: 0 PrevBlk: 0 blkLog: 14
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16800) Cluster Timing
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16801) Sys: 1692369362 base: 1659785124 clst: 32584238
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16802) Dependency Control
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16803) depend: 0 written: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16803) depend: 3 written: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16790) START RL Bi Buffers 3
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16791) Bi Buffer: 0 state 4 dpend: 0 sdpend: 0 dbkey: -1
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0000: 0004 0000 0000 0000 000a 0000 0062 09c8
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0010: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0020: 000a 0000 0062 09c8 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0030: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0040: 000a 0000 006c 10a8 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0050: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0060: 0000 0000 0000 0000 ffff ffff ffff ffff
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0070: 0000 0003 ffff ffff 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0080: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0090: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00a0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00b0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00c0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00d0: 0002 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00e0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00f0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0100: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0110: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0000: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0010: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0020: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0030: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0040: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0050: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0060: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0070: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.
Openedge version: - 11.2
We had a situation where all AI files become full and AI demon was in hung state. We tried make empty full ais but does not work, Db had down. So we re-boot the server. and got date time mismatch. Db got corrupt after re-boot or before no idea.Seeking for root cause analyst\ . Please help. Apologies coz Db logs in Spanish so i just took Few logs.
2023/08/18@09:54:05.052-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:10.063-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:10.063-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:15.072-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:15.073-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:15.964-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 17.
[2023/08/18@09:54:15.967-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 18.
[2023/08/18@09:54:15.968-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 20.
[2023/08/18@09:54:15.968-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 21.
[2023/08/18@09:54:15.969-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 22.
[2023/08/18@09:54:15.969-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 23.
[2023/08/18@09:54:15.970-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 25.
[2023/08/18@09:54:15.971-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 27.
[2023/08/18@09:54:15.971-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 29.
[2023/08/18@09:54:15.972-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 30.
[2023/08/18@09:54:15.972-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 31.
[2023/08/18@09:54:15.973-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 32.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 34.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 34: (2252) Beginning of transaction rollback.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 34: (2253) Transaction rollback has finished.
[2023/08/18@09:54:15.974-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 35.
[2023/08/18@09:54:15.975-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 36.
[2023/08/18@09:54:15.976-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 37.
[2023/08/18@09:54:15.976-0400] P-17237 T-1 I WDOG 15: (2527) Disconnecting missing user 39.
[2023/08/18@09:54:20.084-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:20.084-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:54:25.094-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/chl/ai/ladb2/ladb2 .a4, is full.
[2023/08/18@09:54:25.094-0400] P-17252 T-1 I AIMGT 16: (3776) Backup the ai extension and mark it as empty.
[2023/08/18@09:59:43.428-0400] P-11391 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@09:59:43.439-0400] P-11391 T-1 I ABL : (886) ** The database was last used on Fri Aug 18 05:20:03 2023.
[2023/08/18@09:59:43.440-0400] P-11391 T-1 I ABL : (887) ** Previous-image file expected Fri Aug 18 08:48:39 2023.
[2023/08/18@09:59:43.444-0400] P-11391 T-1 I ABL : (888) ** The dates do not match, this indicates that you have an incorrect copy of one of them.
[2023/08/18@09:59:43.445-0400] P-11391 T-1 I ABL : (334) Single-user session end.
[2023/08/18@09:59:45.734-0400] P-17252 T-1 I AIMGT 16: (3775) Cannot switch to After-Image extension /chi/rt/ai/ladb2/ladb2 .a4, is full.
Fri Aug 18 09:59:43 2023
[2023/08/18@09:59:43.428-0400] P-11391 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@09:59:43.439-0400] P-11391 T-1 I ABL : (886) ** The database was last used on Fri Aug 18 05:20:03 2023.
[2023/08/18@09:59:43.440-0400] P-11391 T-1 I ABL : (887) ** Previous-image file expected Fri Aug 18 08:48:39 2023.
[2023/08/18@09:59:43.444-0400] P-11391 T-1 I ABL : (888) ** The dates do not match, this in
6:02.757-0400] P-20592 T-1 I ABL : (451) Single-user login for root at /dev/pts/1.
[2023/08/18@10:36:02.766-0400] P-20592 T-1 I ABL : (5326) Begin physical redo phase at 512 .
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16793) BEGIN RL Control Structure Dump
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16794) Cluster Data | Bi Blocksize: 16384
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16795) rlcurr: 512 rlclused: 0 curBlk: 0 curOfst: 0 nxtBlk: 0 nxtOfst: 0
[2023/08/18@10:36:03.032-0400] P-20592 T-1 I ABL : (16796) rlctr: 1934 rlsize: 2560 left: 1536 right: 1024 rlcap: 8376320 bytes: 8388608 lastblk: 0 lastOfst: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16797) Recovery Data
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16798) TranTableSize: 172 XIDTableSize: 100
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16799) RedoBlk: 939 RedoOfst: 3862 ReadBlk: 0 ReadOfst: 0 PrevBlk: 0 blkLog: 14
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16800) Cluster Timing
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16801) Sys: 1692369362 base: 1659785124 clst: 32584238
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16802) Dependency Control
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16803) depend: 0 written: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16803) depend: 3 written: 0
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16790) START RL Bi Buffers 3
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (16791) Bi Buffer: 0 state 4 dpend: 0 sdpend: 0 dbkey: -1
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0000: 0004 0000 0000 0000 000a 0000 0062 09c8
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0010: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0020: 000a 0000 0062 09c8 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0030: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0040: 000a 0000 006c 10a8 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0050: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0060: 0000 0000 0000 0000 ffff ffff ffff ffff
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0070: 0000 0003 ffff ffff 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0080: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0090: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00a0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00b0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00c0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00d0: 0002 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00e0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 00f0: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0100: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0110: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0000: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0010: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0020: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0030: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0040: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0050: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0060: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.033-0400] P-20592 T-1 I ABL : (-----) 0070: 0000 0000 0000 0000 0000 0000 0000 0000
[2023/08/18@10:36:03.