After image roll forward performance

[Shanmugam T]

Hello Experts,

After Image is enabled on Production database and Log switch happens once the AI extends becomes full. Archival of these full AI extends are taken care by our scripts and shipped to DR server automatically.

On DR server, AI files are applied to hot-spare database which takes significant amount of time to roll forward all the transactions. Ex: it takes approximately 10 minutes to roll forward 2000 transactions. Sometimes it takes more than one hour to roll forward single AI file if there were more than 20,000 transactions.

Is there any parameters to be passed with roll forward utility to improve the performance like -B, -bibuf and -aibuf etc., ?

I found few articles in KB suggest to upgrade the progress version which is not possible for me right now.
Also i found few old threads which is not helping me.

Is there any other way to improve the performance of AI roll forward? please suggest.

OS: AIX 6.1 & OE: 10.1B

 

[Cringer]

For all the effort involved you might be better off investing in OE Replication Plus licenses.

 

[cj_brandt]

Roll forward performance isn't much better in 10.2B, it will be a few months before I know what 11.5 roll forward performance is. One thing you can check is to make sure your DR server has the same disk performance as prod. Often companies deploy cheaper - slower - disk in DR.

 

[TheMadDBA]

1) Make sure the DR disks and memory are set up properly. On DR you would want the file system buffers (minperm/maxperm) set as high as possible. Just make sure they get set back to meaningful settings when you fail over.

2) Transfer and roll forward the AI files more frequently instead of waiting for them to fill. You probably want to do that anyways for better DR.

3) Cry and wait until Progress makes the AI apply multi threaded

 

[TomBascom]

Sometimes you get in a situation where each roll forward has to go through a very long startup phase.

You can sometimes cure this by shutting down the source, truncating the bi file and making a new backup to roll forward against.

More likely you just have a really crappy IO subsystem on your target.

 

[Rob Fitzpatrick]

10.1B did have a bug related to very long times for undo/redo. At the very least, install the latest service pack of 10.1B on both sides. Preferably, if you can't upgrade to v11.x, upgrade to at least 10.2B.

 

[Shanmugam T]

Hi All,

Yes on DR server we have cheaper and slower storage than the production. Not sure about the current version upgradation or purchasing license for OE replication.

On OS minperm is set to 3% and maxperm is set to 90%.

│ Memory ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────│
│ Physical PageSpace | pages/sec In Out | FileSystemCache │
│% Used 99.2% 0.6% | to Paging Space 0.0 0.0 | (numperm) 44.0% │
│% Free 0.8% 99.4% | to File System 24.0 2916.8 |.Process 41.4% │
│MB Used 39623.7MB 60.1MB | Page Scans 0.0 | System 13.8% │
│MB Free 312.3MB 10691.9MB | Page Cycles 0.0 | Free 0.8% │
│Total(MB) 39936.0MB 10752.0MB | Page Steals 0.0 | ------ │
│ | Page Faults 1403.9 | Total 100.0% │
│------------------------------------------------------------ | numclient 44.0% │
│Min/Maxperm 1160MB( 3%) 34795MB( 90%) <--% of RAM | maxclient 90.0% │
│Min/Maxfree 960 1088 Total Virtual 49.5GB | User 82.3% │
│Min/Maxpgahead 2 256 Accessed Virtual 21.3GB 43.1%| Pinned 14.6% │
│ | lruable pages 9897296.0


Current script have the below line of code to apply ai file.

$DLC/bin/_rfutil $DBNAME -C roll forward -B 8192 -r -a $AIFILE

Increasing the -B value or adding -bibuf or -aibuf will help?

I never used these parameters with rfutil utility before. Have anyone used?

Thanks
Rgs
-Shan

 

[Cringer]

I'm sure you've seen https://progresscustomersupport-sur...rticle/P65028?q=P65028&l=en_US&fs=Search&pn=1
This suggests that 10.1C has significant improvements. I doubt you'll manage to improve things much without a version upgrade. And to be fair, 10.1x is ancient. An upgrade to 10.2B08 should be a very high priority for you.

 

[Shanmugam T]

I'm sure you've seen https://progresscustomersupport-sur...rticle/P65028?q=P65028&l=en_US&fs=Search&pn=1
This suggests that 10.1C has significant improvements. I doubt you'll manage to improve things much without a version upgrade. And to be fair, 10.1x is ancient. An upgrade to 10.2B08 should be a very high priority for you.

Yes, i have already gone through that article. Thanks for your reply. Front-end is MFG/Pro application so not sure will i get approval for version upgrade.

 

[Cringer]

I hear this so often re MFG/Pro. Personally I think this is absolutely shocking for a product that is so widely used. Particularly when a Progress minor version upgrade is so simple. Major versions aren't that hard either really.

 

[TheMadDBA]

-B and most of the other command line options are going to make minimal (if any) improvements.

Since you are on AIX6 and nobody has set any of the strict parameters you are using a lot of memory for buffer cache already.

Truncating the BI file can help (depending on the size of your BI) but you really need to look into your disk setup. Things like the actual disk layout and the AIX related parameters for queue depth and adapter settings.

One thing I have had to do for some very large databases (2TB+) was put the BI files on their own filesystem with caching turned on and turn off the filesystem caching for the filesystems the DB/AI were on. This used most of the OS cache to keep the BI file in memory which reduced the BI portion to a few seconds on some large BI files.

BUT... you need to resolve your disk issues or when you actually do fail over it will most likely be a nightmare.