Some initial words (You can skip this if you only want to know how it's done )
In this tutorial I want to create a Java Program that is able to access a remote Windows machines Service Registry and for example start and stop services on that.
Access will be done during DCE/RPC calls transported using SMB named pipes. In contrast to allmost all other techniques this has the smallest configuration impact on the targeted machine. All you have to have configured are the "File and Printer Sharing" and have the administrative Shares enabled. In most cases Windows is allready configured correctly (http://everydaynerd.com/how-to/fix/windows-7-enable-admin-share).
There seem to be several Java libraries that offer services in the DCE/RPC sector. The one I liked most was JCIFS, which seems to be the base of most more complex solutions and seems to the only not-dead DCE/RPC project. I could not find a single other library that had any update in the last 3-4 years. Unfortunately support I got on mailinglists, forums, etc. for all projects was allmost non-existent. At least with JCIFS I got a response on the list ... even if it was no real help. One thing that convinced me was that JInterop is based on JCIFS and this seems to be widely used.
While doing the basic stuff with JCIFS was pretty easy, finding out how to do DCE/RPC calls was allmost impossible. Unfortunately JCIFS allready had stuff for a lot of DCE/RPC services, but how to use them wasn't explained anywhere and nither Google nor the Mailinglist was able or willing to explain how to do it. So I had to look how JCIFS did stuff internally. So the source was with me, but it turned out it was more the dark side of the source ... and there were no Cookies (This was regarding the methodology of how to use it, but this seems to be more a DCE/RPC thing than a JCIFS thing).
The svcctl service I wanted to acces didn't have any client implementation in JCIFS so I had to create my own. Luckily this code can be generated automatically from so-called IDL files (http://en.wikipedia.org/wiki/Microsoft_Interface_Definition_Language). This interface definition was a well protected secret of Microsoft. The guys at the Samba project did quite a good job of reverse engineering this. Luckily Microsoft had to publish the specification in 2002 in order to avoid major trouble with the European Union (http://www.theregister.co.uk/2002/03/19/why_microsofts_eu_concession/).
Based upon an IDL it is possible to generate the code for communicating with MS services automatically. One implementation of such a compiler is MIDLC (http://jcifs.samba.org/src/) which is available from the JCIFS site.
Fortunately midlc allready comes with a set of idl files. Even with an idl for the svcctl service. From having a look at them the svcctl.idl seems to be implemented as far as the Samba guys managed to do this in the early stages (Not all service functions are available). Unfortunately the ones I was looking for were missing (CreateServiceW, DeleteServiceW, StopServiceW, ...).
So If you now think: "Hey ... so I'll get the official IDL file from Microsoft and use that" (http://msdn.microsoft.com/en-us/library/cc245860(v=prot.10).aspx) ... well what should I say ... the formats are slightly different and therefore you can't use them out of the box. Michal from the JCIFS Mailinglist explained that midlc understands a subset of the official Microsoft IDL format, but with the extension of the OpNums which provide the operation-code that has to be transmitted with every request (http://www.hsc.fr/ressources/articles/win_net_srv/msrpc_svcctl.html). So even if the official IDLs are not directly usable, they will certainly help extending the svcctl.idl shipped with midlc.
So far, so good ... Now I used midlc on a linux machine to generate the stubs for communicating with SVCCTL (I couln't get it to compile on Windows). I used the default files first because I had to learn how do do these calls first and I didn't want to get stuck, just because my IDL was wrong.
Generating the file was pretty easy but unfortunateley I wasted about 5 full days on trying to find out how to use the generated stub classes. All I got was "DCERPC_FAULT_PROTO_ERROR" without any explanation from Windows, what was actually going wrong. That was when I temporarily abandoned my quest. Thanks to Amazon I was able to purchase one used print of the Book "DCE/RPC over SMB - Samba and Windows NT Domain Internals" (With a neat stamp that this book is the property of the Syracuse University). This tought me a lot about how these DCE/RPC calls have to be performed in general and I found the book quite entertaining. Still I could not work out why I was getting these stupid errors. I even started to compare my WireShark dumps with those of PSExec (http://technet.microsoft.com/de-de/sysinternals/bb897553) as this is a tool that does something very similar. But it seems to be doing it completely different. So I was even more confused afterwards.
Beeing completely frustrated I even offered JCIFS professionals a bounty (not the chocolate bar) for providing me with a solution, but not a single "professional" even answered ... "Hail to the glory of open-source!!!!".
Luckily my employee Björn stumbled over Alfresco JLan (http://svn.alfresco.com/repos/alfresco-open-mirror/alfresco/HEAD/root/projects/alfresco-jlan/) and discovered that it could do a lot more than simply copy files from one server to another. It actually came with a full implementation of the SVCCTL Service. This allowed him to build the Software I was trying to build for so long. It took quite some time to figgure out how to build such an application with JLan, but he managed to get it going.
The general functionality was implemented pretty fast after that and he sarted implementing some logic that sent quite large amounts of data over Named Pipes. This is where he began having trouble. Errors complaning about the size of data packets keept on occuring, ruining the data-transfer. He avoided this by breaking everything down to really small pieces of data, but I was never really satisfied with this and it didn't look as if there was any support available from anywhere . So this was only a Proof-Of-Concept solution.
Then one day I had the Idea to use a stripped-down minimal implementation of a JLan program and to have a look at what it does in WireShark and to try to replicate what JLan did in JCIFS. With this approach it took another two days and I was actually able to start a Windows service on a remote machine using JCIFS.
The rest of this tutorial will explain in detail the steps needed to do this. Hopefully this document might save the sanity of the one or the other developer also going nuts on something similar
Generating the Stub
Ok ... so before we can start coding we have to generate the stub code, that we will use do encode/decode the DCE/RPC requests. This is done using the midlc tool (http://jcifs.samba.org/src/midlc-0.6.1.tar.gz). Download it to a linux machine and build it:
You can leave away the "make install" documented in the README.txt as this failled to execute and isn't really needed.
Now you have the "midlc" binary and can start generating code. So if you simply execute the "midlc" binary, it will present you the options:
Unfortunately I was complaining about missing documentation of the parameters here before and I am sorry for that because I was corrected by Michael on the mailinglist. It turned out that the explanation of how to use the tool simply came AFTER all of the change-history. I was assuming no more usage information was to be expected after the first "how to compile block". So just scroll down to the end of the midlcs README.txt and you will find this:
The good thing ist that it claims to only support "java" format, but there is the previously undocumented "jcifs" format, which works nicely. So stick to that.
The other important option was "-o". This controls the name of the output file. So if you set that option to "myCoolFile", then midlc will generate myCoolFile.java".
Inside the midlc package there is a directory "idl" which contains several IDL files for different Windows services. The one I was interested in was "idl/svcctl.idl", so I used that one to generate the initial stub:
The resulting "svcctl.java" was now the starting point of my JCIFS-based SVCCTL client.
Making DCE/RPC calls from Java using JCIFS
So if you generated the code as described in the previous chapter and you use this unmodified, all you will get are "DCERPC_FAULT_PROTO_ERROR" errors. As I had absolutely no clue on how to use the Stub classes and perform DCE/RPC requests, these Errors were driving me nuts. Expecially when comparing my WireShark dumps with those of PSExec. But when I started comparing them to my JLan POC I was able to reverse enginer what to do in JCIFS. I did this by looing at what packets were sent by JLan and then to search through the JCIFS source to find something that looked similar and somehow figgured out how to use them.
One thing I found relatively anoying was the wide usage of packet-level accessability. So I had to implement the code for accessing SVCCTL in a class in the "jcifs.smb" package to be able to access some methods of the SmbTransport class.
After setting up the initial handshake stuff identical to the way JLan did it, I wanted to actually perfom my first SVCCTL call. The result was my ever-so-beloved "DCERPC_FAULT_PROTO_ERROR". But this time comparing the bytes generated by JLan and JCIFS showed me that the packet-type "ptype" being sent was set to "0xff" (-1) instead of "0" (Request). So I modified the generated classes so manually set "ptype" to "0" and was actually able to perform my calls without any problems
So now comes the code for starting the "Remoteregistry" service on a remote Windows system using JCIFS:
For the numeric codes used in the example, please have a look at Microsofts specification document at: http://msdn.microsoft.com/en-us/library/cc245832(v=prot.13).aspx
As I mentioned, I simply replicated the calls done by JLan ... after having a running POC I had another look at the code and noticed that there seemed to be no relation at all between the smbTransport and the DCE/RPC requests. I assumed this "simply has to be done". But now I simply commented parts out, that I assumed to be obsolete. The following code conatins all code needed to perform the actions I wanted:
As now all references to NbtAddress were eliminated, my code is now also free to run in any package I like
Here comes the code that was generated by midlc from the default svcctl.idl (So you don't have to generate it, you can simply use it)
The only difference to the original was setting the "ptype" property to "0" in the constructor of every call.
Extending the default svcctl.idl
The Samba guys could only reverse engineer service calls that were actually used by official tools. I found the Book: "DCE/RPC over SMB - Samba and Windows NT Domain Internals" from Luke Kenneth Casson Leighton (Yes ... that's only one guy ) to describe this process of reverse-engineering the SMB protocol very nicely. It seems that there were no tools available that made use of the ability to remotely create or delete services therefore they could never get network captures of this and therefore never integrate these calls into their IDL.
But now we have the official IDL from Microsoft and we can use this to complete the idl-definition and use this to complete the Stub for the SVCCTL DCE/RPC stub.
I did the extension step by step by extending it with one call at a time. As soon as the call I was woring on worked fine, I took the next one. It turned out to be pretty simple.
I will now describe how I did the extension for one particular Command and will post my finished IDL as well as the generated svcctl.java after that.
In order to add a new command, I needed to find out the operation number for that call. I used this link to find that out. In case of the CreateService call this was 0x0c (Please note that there are Commands ending with "W" and ones ending with "A". As far as I understood the ones with "W" are the synchrounus ones and the ones with "A" are the asynchronous ones. I focussed on the synchronous "W" ones).
Next step was to get the corresponding IDL fragment from the original Microsoft IDL found here.
Comparing this to commands that were allready implemented I saw that I have to remove the "range" stuff first. Which type in this IDL I had to replace with which type of the midlc IDL was pretty simply by consequently looking up the types. The resulting IDL fragment for midlc was this:
Midlc generated the following code from this:
The problem was, that all generated classes were missing the initialization of the "ptype" property, so I had to extend the Constructor manually to this (Only the last assignment had to be changed):
How to use the commands
In order to use the SVCCTL interface, I have to tell JCIFS about it. Without this, all I will get will be "Bad endpoint: \pipe\svcctl" errors. This is done by adding a static initializer bock in your code:
After this you will be able to get a DCE/RPC handle, which will be used for all interactions with the remote system:
In order to be able to perform tasks on the remote registry, you have to open the service manager and get a handle to that:
If the return value was 0 then scManagerHandle will contain a valid handle for the service manager.
You can now use this handle to perform tasks on the remote service registry ... for example get a handle for a service.
So if we were able to get a handle to the "Remoteregistry" service, we can query it's state:
So I hope you get an idea of how these calls are done.
Get the complete IDL from here.
Get the complete svcctl.java class here.
Get the complete example code here.