Parallel Computational Programming

[FocusIT]

Hi All

I have batch job written in ABL running against a Progress Enterprise RDBMS and was wondering if ABL was capable of parallel computation?

The database is running under 10.2B06 X64 on Windows 2008 R2 X64 Server. The batch jobs runs as a self service client on the server. The database holds customer accounts and the batch job reviews the accounts to push them down a work queue, accounts are unrelated to each other and can easily be grouped into portfolios. My normal approach would be to spawn multiple instances of the batch job each concerned with its own set of accounts e.g. one batch doing accounts A-G another doing H-R etc.. However there is another batch job that needs to start straight after the one I want to multi-thread and rather that write a watcher batch to wait for the last thread to complete I was wondering if there was something in ABL that supported parallel computations e.g. one program in ABL code that could start each of the threads and wait for the last to finish.

Parallel computation in my head would look something like this in code: -

/**/
/*Run three threads*/
RUN prThread (1) PROCESS-THREAD CONTINUE. /*spawns a thread running in its own server process*/
RUN prThread (2) PROCESS-THREAD CONTINUE. /*spawns a thread running in its own server process*/
RUN prThread (3) PROCESS-THREAD CONTINUE. /*spawns a thread running in its own server process*/

/*Wait for threads to complete then run next batch*/
RUN prNextbatch.

lots more code.
/**/

Does what I am asking for make sense, after all every server on the planet has multiple CPU cores and it would be good if Progress had support for parallel computational programming.

If 10.2B cannot do this then can OE11? Can any programming language?

Regards

Andrew Bremner

 

[TomBascom]

The 4gl is not multi-threaded. So, sorry, you need to code a "watcher". Which really isn't much different. You will just do something similar to:

os-command silent value( "mpro dbname -p program.p -param threadNum" )

for each thread that you want. The program that launches these then just loops and pauses waiting to see when they are all done. Methods for determining "all done" vary. Sometimes there is an obvious one based on what the tasks are doing, sometimes you need to invent something. Do you need ideas?

Other than that this will work just fine so long as your record locking and transaction scope are properly coded.

 

[Rob Fitzpatrick]

Other than that this will work just fine so long as your record locking and transaction scope are properly coded.

That's the magic sentence. Be careful if the program you are running multiple instances of is (or calls) an existing business object that was written without the notion that there might be several concurrent copies of it running (and corresponding error-handling).

If you can examine the code and determine that the different threads' updates are disjoint, you're fine. But if it's updating records in some control table common to all, or it's a black box, you may have contention between clients.

 

[MaximMonin]

We use the fllowing approuch:
1. There is bgparam.cfg file which containts some rules about bg routine params and num-treads values:
BGcount 3
uBGcount 10
APWcount 5
2. At the begining we run 1 thread and every minute this thread checks config file and decides what to do:
run 1 more thread or quit current thread. It looks like:

batch-count = 0.
for each _connect WHERE _connect._connect-usr <> ? AND
  _connect._connect-type = "self" and /* _connect._connect-batch = "yes" and */
  _connect._connect-name begins "batch" : 
  batch-count = batch-count + 1.
  if _connect._connect-usr = my-pr then
    my-pr-i = batch-count.
end.
/* ubatch */
ubatch-count = 0.
for each _connect WHERE _connect._connect-usr <> ? AND
  _connect._connect-type = "self" and /* _connect._connect-batch = "yes" and */
  _connect._connect-name begins "ubatch" : 
  ubatch-count = ubatch-count + 1.
end.

if search (file-param) <> ? then
do:
  input from value (file-param).
  repeat:
    import unformatted param-line.
    if substring(param-line, 1, 1) = "#" then NEXT.
    if index (param-line, "BGcount") > 0 then /* BGCount  */
    do:
      if index (param-line, "uBGcount") > 0 then /* uBGCount  */
      do:
        param-line = trim(substring ( param-line, index (param-line, "uBGcount") + 8 )).
        param-line = entry (1, param-line, " ").
      
        uplan-count = integer (param-line) NO-ERROR.
      end.
      else do:
        param-line = trim(substring ( param-line, index (param-line, "BGcount") + 7 )).
        param-line = entry (1, param-line, " ").
      
        plan-count = integer (param-line) NO-ERROR.
      end.
    end.
  end.
  input close.
end.




...
if plan-count < batch-count and user-id = "batch" then RETURN "STOP". /* stop processs */
if uplan-count < ubatch-count and user-id = "ubatch" then RETURN "STOP". /* stop process */





if plan-count > batch-count then /* run new process */
do:
&IF {&MultiTenant} = 'YES' &THEN
  os-command value("bin/account/bgrecalc batch@super &").
&ELSE
  os-command value("bin/account/bgrecalc &").
&ENDIF
end.
if uplan-count > ubatch-count then 
do:
&IF {&MultiTenant} = 'YES' &THEN
  os-command value("bin/account/ubg ubatch@super &").
&ELSE
  os-command value("bin/account/ubg &").
&ENDIF
end.

It is possible to change online number of threads online by changing values in cfg file