Your database was damaged. Dump its data and reload it.

Renaldo

New Member
Hi All,

we are running a progress 9.1C database on a Sco Unix server. We got the following error yesterday:

SRV 4: Out of free shared memory. Use -Mxs to increase.

we then ran prostrct

prostrct repair session begin for single on /dev/ttyp53. (451)
prostrct repair session end. (334)

and then started the database which gave the following information:

BROKER 0: Multi-user session begin. (333)
BROKER 0: Begin Physical Redo Phase at 5056 . (5326)
BROKER 0: SYSTEM ERROR: Invalid block 5967 for file /u1/mdsdat/automate.b1, max is 5967 (2329)
BROKER 0: SYSTEM ERROR: Possible file truncation, 5967 too big for database. (612)
BROKER 0: SYSTEM ERROR: The broker is exiting unexpectedly, beginning Abnormal Shutdown. (5292)

BROKER 0: drexit: Initiating Abnormal Shutdown
BROKER 0: ** Save file named core for analysis by Progress Software Corporation. (439)

BROKER 0: ** The database was last used Thu Mar 12 16:25:21 2009. (886)
BROKER 0: ** The before-image file expected Thu Mar 12 16:28:12 2009. (887)
BROKER 0: ** Those dates do not match, so you have the wrong copy of one of them. (888)
proutil -C truncate bi session begin for single on /dev/ttyp12. (451)
** The FORCE option was given, database recovery will be skipped. (33)
** Your database was damaged. Dump its data and reload it. (37)
.bi file truncated. (123)
proutil -C truncate bi session end. (334)

I then increased the -L lock table to 20 000 and started the database with these parameters. All users are currently working, but when i start the database we get the following error:

** Your database was damaged. Dump its data and reload it.

is there a way to repair database without a dump and reload?

Regards
Renaldo
 

TomBascom

Curmudgeon
Hi All,

we are running a progress 9.1C database on a Sco Unix server.

9.1C is obsolete, unsupported and no longer shipped. You are at the mercy of your hardware and OS provider for compatibility. And your OS provider is SCO. You have been warned.

The very last release of Progress version 9 was 9.1E service pack 4. It is now 5+ years old.

9.1C is beyond ancient and unsupported. It was also a fairly buggy release that is mostly used by a certain application that usually has lots of opportunities for performance improvement available. That package also isn't usually implemented with crucial fail-safes like after-imaging enabled.

If you insist on using version 9 you should be using 9.1E SP4. Regardless of what the aforementioned partner may have told you it will work just fine.

There will be no updates, service packs or bug fixes of any kind to version 9. Support of 9.1E SP4 is limited to "best efforts". Which means that if anyone in support can remember they will try to help you.

Expect to be encouraged to upgrade at every opportunity.

We got the following error yesterday:

SRV 4: Out of free shared memory. Use -Mxs to increase.

we then ran prostrct

Why? The error message clearly says to increase -Mxs. Not to run prostrct.

Was there a reason for doing something other than what the message indicated?

prostrct repair session begin for single on /dev/ttyp53. (451)
prostrct repair session end. (334)

and then started the database which gave the following information:

BROKER 0: Multi-user session begin. (333)
BROKER 0: Begin Physical Redo Phase at 5056 . (5326)
BROKER 0: SYSTEM ERROR: Invalid block 5967 for file /u1/mdsdat/automate.b1, max is 5967 (2329)
BROKER 0: SYSTEM ERROR: Possible file truncation, 5967 too big for database. (612)

I think that something happened which you are either not reporting or are unaware of.

BROKER 0: SYSTEM ERROR: The broker is exiting unexpectedly, beginning Abnormal Shutdown. (5292)
BROKER 0: drexit: Initiating Abnormal Shutdown
BROKER 0: ** Save file named core for analysis by Progress Software Corporation. (439)
BROKER 0: ** The database was last used Thu Mar 12 16:25:21 2009. (886)
BROKER 0: ** The before-image file expected Thu Mar 12 16:28:12 2009. (887)
BROKER 0: ** Those dates do not match, so you have the wrong copy of one of them. (888)

At this point your only option is to restore from backup and roll forward your after image logs.

proutil -C truncate bi session begin for single on /dev/ttyp12. (451)

1) Why? What do you hope to accomplish with this?

2) You used the -F option (shown in the message below) yet you are not reporting that you did so in your explanation of what you have done. This makes me deeply suspicious that we are not getting the full story.

** The FORCE option was given, database recovery will be skipped. (33)
** Your database was damaged. Dump its data and reload it. (37)
.bi file truncated. (123)
proutil -C truncate bi session end. (334)

Since you used -F you've tossed away all the data in the bi. You now have a damaged database with unreliable relational integrity.

I then increased the -L lock table to 20 000

Again... why? What reason did you have for increasing -L?

... and started the database with these parameters. All users are currently working, but when i start the database we get the following error: ** Your database was damaged. Dump its data and reload it.

is there a way to repair database without a dump and reload?

Dumping and loading will not repair the database.

It will clear the "tainted" flag and, possibly, expose any deeper problems that you may have. But you lost all hope of recovering all of the data when you used -F.

If you are very, very, very lucky you haven't lost much. Perhaps most of the bi file had already been flushed to disk. This might be true is the system was relatively quiet at the time.

But I wouldn't count on it. You'll probably be dealing with strange data errors for months. Maybe forever.

Obviously you should upgrade. If you have source and can compile code there is no technical reason why this application cannot be moved to a safe OS, such as Linux, and an up to date Progress release -- like OpenEdge 10.1C. Actually by the time you get around to it 10.2 will probably be perfectly stable for this purpose.

You should also learn about and implement after-imaging so that the next time something happens you can actually recover from it rather than living with an unknown amount of damage.
 

Renaldo

New Member
Hallo Tom.

1. Thank you for your response.
2. I shall answer all the question with as much detail as possible so you have a complete picture. The application are supported by a supplier. After the database went down we made contact with them, they logged in remotely and after about 30mins the database came up again. I was concerned about the error message I saw upon starting the database, thus I started reading through the log files. That is when I started raising the same questions, thus my post to obtain additional info / research. They proposed a dump and load, but this did not make sense, from my understanding the structure of the database is damaged (this could include data as well, i am unsure of the extent of damage, i have not picked anything up from the users)
3. Regarding the upgrade, we are busy with the negotiations, we should be on their latest DB and application by the end of this year, the current system have to last until then, so i need to rectify it.
Quote:
Was there a reason for doing something other than what the message indicated? I am not sure why prostrct was used, when i stared investigating i picked up the one DB had a -L 20000 when it started, the other did not. I asked the question and they said it might have a connection, this the change. It did however work for 4 years without the -L.
Quote:
You used the -F option (shown in the message below) yet you are not reporting that you did so in your explanation of what you have done. This makes me deeply suspicious that we are not getting the full story: I cannot see where in the log they forced it, i assume the ran the command and this was reported in the log file. I will attach the full log file to this post, that should highlight everything

All users are currently working on the system. (it seems without any problems, but my gut feeling is there is a huge risk attached to it, or all data entering the database might not be 100%) I am not comfortable with the current situation, but need to make the right decision in consultation with my supplier. I am on my way to the office and shall attach the logfile when I get there.

is there something i can do to remove the ** Your database was damaged. Dump its data and reload it. This message prohibits me from running our "end of day" script withou manuel intervention, requiring us to sit through the whole procedure for 7 hours.

all the best, thanks again, Renaldo
 

Attachments

  • automate.log.txt
    57.2 KB · Views: 20

TomBascom

Curmudgeon
You can clear the message with a mock index rebuild.

In a nutshell you run idxbuild, select "some" indexes and then tell it you're done selecting indexes without actually selecting any. The idxbuild will then clear the flag and exit.

Don't be fooled. Data was lost. You just have no way of knowing how much data or which data.

Apparently your vendor's support organization did this to you? I'd be very concerned about the quality of their advice and the competence of their staff.

You say that you are moving up to their latest and greatest -- that's good, presumably you'll be on a supported OS and a current release of Progress. Like many application partners they probably produce an excellent product for the business problem. But they don't seem to have a very firm grip on operational matters. IMHO, you should get outside help regarding the setup and configuration of the new system.
 

ePrasa4

New Member
proutil skybank.db -C truncate bi -F

this is a good wayyyyyyyyyyy OpenArc member

[Edit by casper]
Don't ever use this command! (it should be even forbidden to post it, a well known expert once told me...)
[/edit]
 

tamhas

ProgressTalk.com Sponsor
I almost said that, but I thought I would give the advice giver a chance to say why ... and then let you shoot holes in it!
 
Top