The second error you describe typically comes from running the database and the client with two different version. They need to be exactly the same version ( to the patch level ) when you access the database with a self-service client ( directly via shared memory as opposed to a remote client which access the database via TCP ) and with the database administration utilities like proutil for example. I suspect that you have two different version installed and you PROPATH setting for the client points into the "old" version.
Regarding the first error: Did you look into the log file of the database - it will tell you exactly as to why the database could not be started. The broker startup failed is way to generic to say something intelligent about it other than that: In 99% of the case where I have seen this either the path to the database was incorrect or the setting of the TCP port for the broker was incorrect. But that is just speculation. You need to look into the log file of the database - which is your best information source when something is wrong with the database.
Heavy Regards, RealHeavyDude.