1

TL;DR - How do I prevent -Embedding startup of a CLSCTX_LOCAL_SERVER server? Preferably so that the clients get a decent error code immediately.

Background

We have a native C++ interactive Desktop application that interacts with a COM object in another native C++ interactive Desktop application.

Basically, COM is used as an interprocess communication mechanism.

Now, when the "server" application is started interactively after the user has brought it into the correct state, it will prepare the COM interface: CoRegisterClassObject etc etc.

When the client application is used and then does it's CoCreateInstance for the coclass, it will communicate with the already running other Desktop application, which is what is intended.

However, when the "server" application is not running, starting the client will launch the server application interactively, which is NOT what we want, because it needs quite some processing and setting up before the client can meaningfully communicate with it.

Question

So, what would make more sense is for the client to just error out in case the server is not running, instead of having the COM infrastructure start an interactive application that can't service the request meaningfully anyways.

We've toyed around with the following ideas:

  • Unregister the server each time it is closed.
    • It seems this won't work, as we need administrative rights to register/unregister the server.
  • Use the CLSCTX_DISABLE_AAA flag on the client side.
    • This seems to work, if the client specifies this and the server ain't running, it will get 0x80070005 ERROR_ACCESS_DENIED, but we're rather unsure whether this is the correct approach.
  • Do an early check in the server application to detect the -Embedding switch and immediately exit the application on this.
    • The client application will run into a timeout (~ 2 min), which is not very user friendly.
  • from comments Do not write the info to the registry - CoRegisterClassObject should be enough.
    • Currently, I have:
      • HKCR\AppID (...)
      • HKCR\CLSID\{...} plus subkeys ProgID, VersionIndependentProgID, LocalServer32, Typelib
    • Clients currently resolve the CLSID from the VersionIndependentProgID
    • So I'm left wondering which parts of the registry are really optional.

Is there any "standard" way to prevent COM local server executable activation for a given registered COM class?

YosiFZ
  • 7,792
  • 21
  • 114
  • 221
Martin Ba
  • 37,187
  • 33
  • 183
  • 337
  • 1
    COM registration doesn't need admin rights, but you'll have to register everything under HKCU instead of HKLM (which seems to make sense in your context). Otherwise you don't have to use CoCreateInstance to get a reference, you can use the Running Object Table, from the client check your server has registered something in it and call IRunningObjectTable::GetObject – Simon Mourier Mar 05 '19 at 09:20
  • @SimonMourier - thanks. The IRunningObject approach sounds neat. I tried to quickly whip something together, but haven't managed to get it running so far `0x800401E3 (MK_E_UNAVAILABLE)` even while the server is running. – Martin Ba Mar 05 '19 at 11:20
  • You can use this tool to check the object is visible : http://alax.info/blog/1444 also make sure security is ok (UAC, etc.), client and server running in same Windows station. – Simon Mourier Mar 05 '19 at 13:10
  • 1
    When detecting the -Embedding arg, you may try to issue immediately the CoRegisterClassObject call and THEN exit. The client will get a CO_E_SERVER_EXEC_FAILURE without timeout... Yes, that's ugly. – manuell Mar 05 '19 at 16:07
  • 1
    For inproc com objects you don't need to have them in the registry - you can register their class factories at runtime. I'm wondering if this can be applied to out-of-process COM objects by registering the proxy objects... – Chris Becke Mar 06 '19 at 04:16
  • @SimonMourier - actually, my objects are not in the running object table at all. It seems CoRegisterClassObject doesn't put them there and that's all the server seems to do. – Martin Ba Mar 06 '19 at 09:44
  • 1
    CoRegisterClassObject is just to publish your object to COM (as an oop server), it does nothing with respect to the ROT. Once you've called CoRegisterClassObject, you must call IRunningObjectTable::Register from the server and IRunningObjectTable::GetObject from the client. – Simon Mourier Mar 06 '19 at 10:37
  • 1
    simply not register your object *clsid* in registry. if server yet not call `CoRegisterClassObject` client get error `REGDB_E_CLASSNOTREG`, otherwice call `CoRegisterClassObject` is enouth – RbMm Mar 13 '19 at 07:38
  • Please explain in details why the comments are not an answer. – Simon Mourier Mar 13 '19 at 08:38
  • @RbMm - well, that would actually be awesome. Must try that. Is that documented somewhere? – Martin Ba Mar 13 '19 at 09:41
  • @SimonMourier - your comments were really helpful, and I think I could go the ROT approach, but putting my object into the ROT would still not prevent a client from starting the server by calling `CoCreateInstance` – Martin Ba Mar 13 '19 at 09:44
  • you not need add it to *ROT* - the single call `CoRegisterClassObject` is enouth - the client can create instanse after this. no need any registration in registry - you easy can check this in test. you of course need for any remote interface entry in registry `Interface\{..}\ProxyStubClsid32` but not need this for `CLSID` – RbMm Mar 13 '19 at 09:51
  • @RbMm - I think you provided one missing puzzle piece, namely that CoRegisterClassObject does not need the registry at all. (Maybe [that one](https://stackoverflow.com/questions/36955791/can-i-create-and-use-a-com-class-without-registering-it-in-the-registry) is a bit of a dupe.) – Martin Ba Mar 13 '19 at 11:52
  • @RbMm - see, one problem still is what you mentioned "you of course need ... interface registry ..." - it is still clear as mud what parts of the `AppID`, `CLSID` and `TypeLib` entries in the registry I need or need not! (I should update the question with this.) – Martin Ba Mar 13 '19 at 11:54
  • @MartinBa - yes, `CoRegisterClassObject` does not need the registry at all - i say this based on test. but you need still marshall your interface(s). for this you need `Interface\{..}\ProxyStubClsid32` key . the `TypeLib` you need if you do this type of marshalling - set `{00020424-0000-0000-C000-000000000046}` here. the `CLSID` and `AppID` you not need. you can easy test this - say delete your `CLSID` key from registry. can even reboot for sure. and then call `CoRegisterClassObject` - it will be ok. and after this clients can create your instance – RbMm Mar 13 '19 at 12:13
  • `CLSID` and `APPID` need for start your app, if it already not started (or load dll). if you already run and call `CoRegisterClassObject` - this is enough for client call connect to you. but without `clsid` - client can not exec your app (simply unknown what ). if you not do custom marshalling - need for every interface have infor in registry - which dll `ProxyStubClsid32` do this marshaling. this can be or custom dll or if standard `{00020424-0000-0000-C000-000000000046}` - you need typelib for use by this oleaut32 – RbMm Mar 13 '19 at 12:22
  • 1
    https://blogs.msdn.microsoft.com/larryosterman/2005/10/18/activate-as-activator-activates-as-activator/ - so exist problem if client and server have different tokens (problem with impersonation), but *we didn't need a class registry for our COM objects, things just worked fine.* – RbMm Mar 13 '19 at 12:32
  • 1
    COM needs nothing but a vtable (contract between client and server) to work (for out-of-process, it does need proxy/stub for marshaling of course). You can also provide an export from a DLL that gives an interface directly (like Microsoft libraries do, for ex DXGI has a CreateDXGIFactory function); no need for CoCreateInstance at all. I still think the ROT is an easy way. – Simon Mourier Mar 13 '19 at 19:08
  • https://learn.microsoft.com/en-us/windows/desktop/learnwin32/creating-an-object-in-com – Martin Ba Mar 26 '19 at 21:54

2 Answers2

1

My idea is this ...

Just set a global flag in your EXE when you detect the server is started with -Embedding. I would probably create a special class factory only when you startup with the -Embedding flag. This class factory would return a fail code when IClassFactory::CreateInstance() is called. You would not register the standard class factory as running with CoRegisterClassObject(), but you'd register only your alternative factory that always returns the fail code.

Yes, there would still be a slight delay on starting up the EXE, but when the CreateInstance() is called, it would immediately return the fail code and so the caller would not have a long timeout...maybe 1-5 seconds only.

Joseph Willcoxson
  • 5,853
  • 1
  • 15
  • 29
  • Nice idea for a workaround. This would also enable a somewhat "custom" error code, which is a nice bonus. I have to say I still would very much prefer the executable not even being started, but as far as workarounds go, this seems good. – Martin Ba Mar 15 '19 at 09:33
  • You can put nearly everything you need to run in manifests. Manifests won't allow you to start up an EXE, but as you noted originally, once the server is running, you can connect to it's class factory. What makes things a little easier is if all your interfaces are dual or IDispatch derived. That way you just need to know the interfaces and put a entry in your manifest so it knows how to Marshal the interfaces. I'd put the typelib for the server in the same directory as your client as a TLB file and reference it in a manifest entry. – Joseph Willcoxson Mar 15 '19 at 14:21
-1

Write up of what I learned from comments so far:

You do not have to use CoCreateInstance, but you can use GetActiveObject, which

Retrieves a pointer to a running object that has been registered with OLE.

so there you have a way to get an object without activating it, but relying on it already being registered. (But this is not done by CoRegosterClassObject, instead you need to ...?)

Similar to the GetActiveObject approach, you can use the machinery around IRunningObjectTable - which may be what's used by GetActiveObject under the covers anyway. i git kind of lost there.

Another piece of info is that the registry is claimed to be kinda "optional" in all this: (paraphrasing)

CoRegisterClassObject is just to publish your object to COM (as an oop server), ...

simply not register your object CLSID in the registry. If the server has not yet called CoRegisterClassObject, client gets error REGDB_E_CLASSNOTREG, otherwise call CoRegisterClassObject is enough. ...

you not need add it to Running Object Table - the single call CoRegisterClassObject is enough - the client can create instance after this. No need for any registration in the registry - ...

... you of course need, for any remote interface, an entry in registry Interface\{..}\ProxyStubClsid32 but not need this for CLSID.

CoRegisterClassObject does not need the registry at all ... But you need still marshal your interface(s). For this you need ther Interface\{..}\ProxyStubClsid32 key.

The TypeLib you need if you do this type of marshaling - set {00020424-0000-0000-C000-000000000046} here. The CLSID and AppID you do not need.

CLSID and APPID you need for starting your app, if it is not started (or load dll).

If you already are running and call CoRegisterClassObject - this is enough for client call connect to you. But without CLSID, client can not exec your app (simply unknown what).

If you do not do custom marshaling - need for every interface have info in registry - which dll ProxyStubClsid32 do this marshaling. This can be or custom dll or if standard {00020424-0000-0000-C000-000000000046}, then you need a typelib for use by oleaut32.

In summary, it would appear the way to prevent having an app activated by Windows is to

  • NOT write the info needed for activation into the registry in the first place.
    • but only the info required for marshaling.
  • and/or NOT calling CoCreateInstance and retrieving the object via another route.
Martin Ba
  • 37,187
  • 33
  • 183
  • 337