Do you have access to source? If so, you can compile, and therefore should in theory be able to use any version. Testing is key of course. And moving to 12 may be difficult because of the retirement of 32 bit server side stuff and the classic appserver.
If you don't have access to source, then you will be forced to use whatever version the vendor puts you on.
It is possible to run Syteline on OE11. There are several sites that I am aware of that are doing so.
You do need to have all of your source code and be able to compile it. SFAIK Syteline itself always shipped with source - every customer that I have ever dealt with had the source. I do know of at least one site where they somehow managed to lose the source for customizations that a consultant did for them. (And said consultant then shuffled off this mortal coil...)
Syteline does not use app servers so you are safe from that problem.
Server side changes from 32 to 64 are immaterial.
Client side usage of 32 bit OCX (or, worse, VBX) controls may be problematic as it is likely that the controls are no longer supported by the original authors. Which may force you to stay with 32 bit clients if you are unfortunate enough to be using that sort of thing.
You're right, of course, unless there is the necessity to run 32bit ocx dependencies on the server - for such things as printing. Bad design, but it happens a lot. 32bit and 64bit Progress do cohabit post v11, but it's a bit risky!
I have compiled sl2g with latest version of OE11/win32 for our client (main code + about 20 years of development client specific code).
SL2 has a little amout of OCX instead v6. Not all of them are working fine.
I have seen only one 4GL incompatibility issue with PROMPT-FOR statement on one screen (inside lib/acct.i - TAB not works correctly) after all regress tests.
The second problem for both SL2 and SL6 - if you using Trilogy/SyteSelect - you get stuck on ODBC queryes and ODBC-related OCX components. (Fields with dashes should be quoted). I see no solution yet.
v10.2B is retired this year, but works fine on win10 too.
Do you run application server or only client-server?