Tools

IDA, SoftICE, Microsoft VC++ v6.0, hex editor, DvdMaestro v2.5 and dongle.exe.

Introduction

Sentinel Shell is Rainbow Technologies answer to the HASP envelope. While the HASP envelope has been/is being widely used, Sentinel Shell is rather unknown. Looking at Rainbow Techs' protection solutions we will see that their two most widely used products are the SuperPro dongle and their software based SentinelLM. Sentinel Shell exists in both variants, called Sentinel Shell and SentinelLM shell. If you've read my past work I have been describing how to bypass the SentinelLM Shell by making real license keys which are similar to FLEXlm. In that case studying the shell was not needed as it could be bypassed easily.

As we will see soon the Sentinel Shell is heavily dependent on the SuperPro dongle used. The only way to remove this is to understand how it is built up, what it wants, and solutions to the associated problems. This essay will have parts left out as it's meant to be a short overview. If you are interested in fully learning this then spend most your time with your reversing tools instead of reading this. One of the things I left out on intention is the study of the SuperPro dongle. I am assuming you know it's use of the API set, memory cells and how to defeat it.

Overview of the shell

The shell is made like a onion skin envelope. To open it up you will need the real dongle which a program was protected with. At least that is what Rainbow Tech wants you to believe. The flow of the shell is like a big switch. It uses "messages" to determine the flow (success or failure). Opening a program in SoftICE using this scheme is meant to confuse people. The messages change whether a check for the dongle was successful or not. What's typical for this scheme is delayed actions, copying of important variables several times, and non-static use of various data.

The example below shows the main switch of dvdmaestro.exe :-

0098E465 mov ax, ds:word_98C4B8 ; Get message
0098E46B cmp eax, 342h ; Go through the switch
0098E470 jg loc_98E485
0098E472 jz loc_98E63E
0098E478 cmp eax, 0E4h
0098E47D jz loc_98E604
0098E483 jmp loc_98E436
0098E485 cmp eax, 704h
0098E48A jg loc_98E49F
0098E48C jz loc_98E6AB
0098E492 cmp eax, 611h
0098E497 jz loc_98E65A
0098E49D jmp loc_98E436
0098E49F cmp eax, 1092h
0098E4A4 jg loc_98E4BC
0098E4A6 jz loc_98E6DF
0098E4AC cmp eax, 96Fh
0098E4B1 jz loc_98EA89
0098E4B7 jmp loc_98E436

The code shows only a little bit of the main loop. As you might guess every message has a meaning. If you start a protected executable the message EAX = 96Fh is the one forcing either failure or success. This will be the last one which leads to a MessageBox error or success. If you need a real entry point of the file this will be the one to follow at the end. When protecting your program with Sentinel Shell it gives access to use counters, expire control and etc. Depending on how you want the program to run, the shell will be changed in a way to suit your needs.

Initial analysis

First thing which is important to know is how it calls the dongle. Unsurprisingly the initial check is RNBOsproFindFirstUnit(). Forcing this good will result in a few calls to to RNBOsproRead(). All those initial calls read cell 0 which we know is the dongle serial and therefore isn't verified later on. After this RNBOsproQuery() is called. This will be the main exercise of this project. You will see that it's not as easy to defeat as the first 2 API's.

Looking at DvdMaestro

We will take this as our first target. First we do is to load it into IDA, apply the needed Sentinel SuperPro signature, make a map file, and force SP_SUCCESS by patching RNBOsproFindFirstUnit() and RNBOsproRead(). Analyzing the use of RNBOsproQuery(). Below is posted the header of sproQuery. We will need to clearly identify what we are working on to get anywhere :-

SP_STATUS SP_API RNBOsproQuery( RBP_SPRO_APIPACKET packet,
                                RB_WORD            address,
                                RBP_VOID           queryData,
                                RBP_VOID           response,
                                RBP_DWORD          response32,
                                RB_WORD            length );

Implementation in our file is :-

0098E604 mov ebp, ds:dword_98C4A8
0098E60A mov eax, ds:dword_98C4A8
0098E60F add ebp, 14h
0098E612 mov ecx, [eax]
0098E614 push ecx ; length
0098E615 mov edx, [eax+4]
0098E618 push edx ; response32
0098E619 mov ecx, [eax+8]
0098E61C push ecx ; response
0098E61D mov edx, [eax+0Ch]
0098E620 push edx ; queryData
0098E621 mov ecx, [eax+10h]
0098E624 push ecx ; address
0098E625 mov edx, [ebp+0]
0098E628 push edx ; packet
0098E629 call ds:dword_98C4B0 ; RNBOsproQuery()
0098E62F mov [ebp+0], eax ; save return code
0098E632 add ds:dword_98C4A8, 14h
0098E639 jmp loc_98E436 ; continue for delayed action

As we see it's not compiled statically. First time Query() is called we see that the query value differs. Backtracing can be a bit tricky but you will notice there is 1 table and 1 static xor operand used. After using the bpm feature in SoftICE we will find this table and value. First we find where the code is called and saved. To clear up the meaning I will call the this table call GetQuery.

0098E754 add ds:dword_98C4A8, edi
0098E75A call ds:dword_98C4B0 ; call GetQuery
0098E760 mov ecx, ds:dword_98C4A8 ; get where to save
0098E766 mov [ecx], eax ; save
0098E768 jmp loc_98E436 ; continue

Again we notice indirect calling. The GetQuery call looks like this :-

0098E230 mov eax, [dword_98B9C8] ; get randomly defined number
0098E235 mov eax, [eax*4+0098b010] ; use displacement to get a query value
0098E23C retn

We see that the size of the query is a DWORD meaning 4 bytes. Continuing tracing ECX where the query got saved before we get here :-

0098E8F2 mov ax, ds:word_98C4BC
0098E8F8 mov ecx, ds:dword_98C4B0
0098E8FE push eax
0098E8FF push ecx
0098E900 call sub_98E360 ; get xor value
0098E905 add esp, 8
0098E908 mov ecx, ds:dword_98C4A8 ; get the saved query value
0098E90E xor [ecx], eax ; perform xor on the value
0098E910 jmp loc_98E436 ; continue

We will notice this xor value is static so lets save it and reuse it for our emulation. In this specific case the value is 0xFD0699D6. Now the result value will be the one which gets passed to sproQuery(). We now have the start to our emulation and need to find our response. Making sproQuery return SP_SUCCESS and bpm'ing on the response leads us in the right direction. We will see that our response gets xor'ed with another value :-

0098E900 call sub_98E360 ; get our response
0098E905 add esp, 8
0098E908 mov ecx, ds:dword_98C4A8 ; get address of a 2nd value
0098E90E xor [ecx], eax ; perform xor on the value
0098E910 jmp loc_98E436 ; continue

Monitoring the memory address in ECX gets us here :-

0098E688 mov eax, ds:dword_98C4A8 ; get xor'ed value
0098E68D add ds:dword_98C4A8, esi
0098E693 cmp dword ptr [eax], 0 ; was it zero?
0098E696 jz loc_98E436

Making the jump gets us to the next query and not making the jump leads to an error. We can now conclude the jump must be taken and therefore the value that our response got xor'ed with must be the right one. So now we must trace the source of that real value. Again we encounter 1 table and 1 xor. The table call I will call GetResponse. We reach the GetResponse table :-

0098E240 mov eax, [dword_98B9C8] ; get same random number as with GetQuery
0098E245 mov eax, [eax*4+dword_98B090] ; use displacement to get a return value
0098E24C retn

This value gets saved and continuing it gets reused :-

0098E900 call sub_98E360 ; get xor value
0098E905 add esp, 8
0098E908 mov ecx, ds:dword_98C4A8 ; get the saved response value
0098E90E xor [ecx], eax ; perform xor on the value
0098E910 jmp loc_98E436 ; continue

In this case the value is 0x7B2309A7. Now we have all the needed info for the entry emulation of sproQuery. A example of an emulation could be :-

call GetQuery
xor eax, 0xFD0699D6
mov edx, [inputed query value from stack]
cmp eax, [edx]
jne next_query_check
call GetResponse
xor eax, 0x7B2309A7
mov [response on stack], eax
xor eax, eax
retn (displacement)

This should take care of it. If we deal with the first Query, the first compare value should match with the input parameter and the matching response gets saved.

Getting distracted

Implementing this makes the first 2 query checks go correctly and a new error gets shown. We see that there is no 3rd query call so something else must block it. Reviewing sproRead again shows it will make a call with the cell 0Bh to read. To recover the WORD which should be read we bpm on the response address and get here :-

0098E79E call sub_98E360 ; get our response
0098E7A3 add esp, 8
0098E7A6 mov ecx, ds:dword_98C4A8 ; get constant
0098E7AC sub [ecx], eax ; subtract
0098E7AE jmp loc_98E436 ; continue

In my case, this constant is 0x00002A58. Continuing tracing the data in memory we will see first a and operation with 0xFFFF0000.

0098E7C1 call sub_98E360
0098E7C6 add esp, 8
0098E7C9 mov ecx, ds:dword_98C4A8
0098E7CF and [ecx], eax ; logical AND operation
0098E7D1 jmp loc_98E436 ; continue

....and at the end a compare :-

0098E688 mov eax, ds:dword_98C4A8
0098E68D add ds:dword_98C4A8, esi
0098E693 cmp dword ptr [eax], 0
0098E696 jz loc_98E436 ; was the result zero?
0098E69C mov eax, ds:dword_98C4B0
0098E6A1 mov ds:dword_98C4AC, eax
0098E6A6 jmp loc_98E436

By default the result will be zero and the jump will occur. Changing the flag and taking the unconditional jump will get us to the next sproQuery check. So now we have enough data about the WORD read from the dongle. First the constant which is also in WORD size and it will be subtracted with our WORD. Later it gets AND'ed with 0xFFFF0000 which only lets the upper bits pass. So we can now conclude that the only response which will pass is one which makes the subtraction give a negative result and thereby setting the sign bits. Therefore any value bigger than the constant will let us pass.

Extending the sproRead is easily done :-

mov edx, [Read address from stack]
cmp edx, 00B
jne Not_B
mov edx, [response address from stack]
mov [edx], 0xFFFF
Not_B:
xor eax, eax
retn (displacement)

Getting to the decryption stage

We now got rid of the 2 first query calls and the read operation and will get to the 3rd query. First thing we will notice is that the query value does not match with the tables used before. Running the program a few times will show that the value is static too, 0x022A4B528. Like before we initially study what will become of our response. First it will be copied a bit around and then it will be input to a call below.

0098E9E7 mov eax, ds:dword_98C4A8 ; response address
0098E9EC add ebp, 8
0098E9EF mov ecx, [eax] ; our response
0098E9F1 push ecx
0098E9F2 mov edx, [eax+4]
0098E9F5 push edx
0098E9F6 mov ecx, [ebp+0]
0098E9F9 push ecx
0098E9FA call ds:dword_98C4B0
0098EA00 mov [ebp+0], eax ; save status
0098EA03 add ds:dword_98C4A8, ebx
0098EA09 jmp loc_98E436 ; continue

We now trace the status getting returned from the call and see it gets xor'ed with 0x5C95209D :-

0098E900 call sub_98E360 ; get operand
0098E905 add esp, 8
0098E908 mov ecx, ds:dword_98C4A8 ; the address of our status
0098E90E xor [ecx], eax ; logical XOR operation
0098E910 jmp loc_98E436 ; continue

Tracing further and we will see a compare :-

0098E693 cmp dword ptr [eax], 0 ; was the value zero?
0098E696 jz loc_98E436

The only way to make that compare get happy is to make the status of the call give the same value as the other xor operand. Now lets see what the call hides. The first instruction address inside that call is 0x98EB30. In IDA this is interpreted as an array of bytes, so using the undefine and make code feature gets us what we want to see, the decryption code. In the code below I already named the inputs so it will be easier to understand :-

0098EB30 push ebx
0098EB31 xor eax, eax
0098EB33 push esi
0098EB34 push edi
0098EB35 mov esi, counter
0098EB39 shr esi, 2
0098EB3C mov ecx, esi
0098EB3E dec esi
0098EB3F test ecx, ecx
0098EB41 jz loc_98EB74
0098EB43 mov edx, decryption_start_address
0098EB47 mov edi, our_response
0098EB4B mov ecx, edi
0098EB4D mov ebx, edi
0098EB4F shl ecx, 4
0098EB52 add edx, 4
0098EB55 shl ebx, 5
0098EB58 add ecx, edi
0098EB5A shr ecx, 9
0098EB5D xor ecx, ebx
0098EB5F add edi, ecx
0098EB61 mov ecx, [edx-4]
0098EB64 xor ecx, edi
0098EB66 add eax, ecx
0098EB68 mov [edx-4], ecx
0098EB6B xor edi, eax
0098EB6D mov ecx, esi
0098EB6F dec esi
0098EB70 test ecx, ecx
0098EB72 jnz loc_98EB4B
0098EB74 pop edi
0098EB75 pop esi
0098EB76 pop ebx
0098EB77 retn 0Ch

This code decrypts the memory depending on our response. To make things easier below is an example of a C implementation of this code :-

do
{
	tResponse += (((tResponse << 4) + tResponse) >> 9) ^ (tResponse << 5);
	*(lpUnCryptedBuffer) = *(lpCryptedBuffer) ^ tResponse;
	Sum += *(lpUnCryptedBuffer);
	tResponse ^= Sum;
	++lpCryptedBuffer;
	++lpUnCryptedBuffer;
	--iCounter;
} while ((iCounter+1) != 0);

Sum will be the variable holding the return code so as soon as we know the correct response we will get the correct end value of Sum. One way to get the correct response would be to bruteforce the entire array of possible response codes while checking the return code. This array is all from 0x00000000 - 0xFFFFFFFF. For DvdMaestro.exe it works like this (using the query response 0x12345678 as a test) :-

First we get tResponse from tResponse += (((tResponse << 4) + tResponse) >> 9) ^ (tResponse << 5);.

This gets xor'ed with the first DWORD to decrypt which is stored in *(lpCryptedBuffer). In *(lpUnCryptedBuffer) we have the first decrypted DWORD 0xACD6CE25 which in this example is wrong. Because Sum starts with 0, Sum will become the same value as the decrypted DWORD. Next when xor'ed with tResponse, tResponse will hold the first encrypted DWORD from the file 0xF4120473.

In the file this is represented as 0x73, 0x04, 0x12, 0xF4. Next round the tResponse will become 0x766C1BB8 and xor'ing this with *(lpCryptedBuffer) gives us the bytes 0x9090C300 in *(lpUnCryptedBuffer). In bytes it would be treated as 0x00, 0xC3, 0x90, 0x90. If we think of this as assembler instructions it would look like this :-

??
RET
NOP
NOP

Now our guessing skills are needed. If we assume that VC++ was the compiler used it aligns code using NOP instructions.
If we assume the next instructions also are NOP's (0x90) we can modify the decryption code above to only decrypt 3 times where the 3rd time the result in *(lpUnCryptedBuffer) should be the DWORD 0x90909090. Once this is implemented just wait for your computer to do the rest.

The shortcut

The above mentioned way can be very time consuming with the computer speeds of today. However there exists a shortcut.
When analyzing this you will see that the 2nd DWORD getting decrypted will always contain the correct decrypted bytes. Using this powerful info you can start looking at what code there could be and build a much faster bruteforcer which in my case took a little under 2 minutes to find the correct info.

The big bad bug

With a PE editor you will see that the first section 00000001 with the data which should be decrypted starts at offset 0x2000 in the file. This corresponds with the address 401000 which gets pushed as a parameter to the decryption routine. This data has been reallocated with 1000h bytes. Looking at 0x1000 and using the info above you will now see that the data is the first part of the decrypted data which we need. Now no more guessing is needed since it's all there ready to be abused. Whether this is a bug belonging to the version of the shell being used or belonging to DvdMaestro is unknown to me.

We now find that the query response should be 0x342A3C33. Implementing the emulation is done easily :-

mov edx, [inputed query value from stack]
cmp [edx], 022A4B528
jne next_query_check
mov [response on stack],0342A3C33
xor eax, eax
retn (displacement)

Now the decryption return status matches the xor operand and we continue towards the next query check.

Final Stage

4th sproQuery call is another decryption call like the last one. Same routines are used and same checking method. The start point of decryption we find is address 518000. While before the .text section got decrypted this matches with the .data. We assume that the compiler used is MS VC++ and checking various executables for interesting stuff we will discover that the first DWORD always contains zero's in the .data section. Again we have a huge shortcut to follow making the discovery of the response take a lot less time. The response should be 0x1A610596. Implementing this in a similar way as above should make this check go in the right direction.

Success

We've now defeated the shell and the program starts even though we get another dongle error. This is of course related to the executable which was encrypted. Since we now have the needed data to decrypt it ourselves it's possible to recover the real executable and continue the emulation. However I won't go into details with this as we'll only concentrate on the shell. At the bottom I posted the code to decrypt the sections in the file.

Starting over again

Lets take another target to not spoil the fun so soon. This is a simple MessageBox which I encrypted using the Sentinel SuperPro dongle SDK. The source code is posted at the bottom of the document. The program was compiled in release mode and the Sentinel protection program was in v6.0 (at the time of writing the most recent version). Under the protection part of the program I chose cell 0x8 as first half of the algorithm descriptor and no read cells used.

Again we disassemble the file, apply a superpro sig and make a map file. First we deal with sproFindFirstUnit. Next comes sproRead. Already here you should see that the versions of the shell are different :-

00416AF0 sub esp, 8
00416AF3 lea eax, [esp+0]
00416AF7 push esi
00416AF8 xor esi, esi
00416AFA push eax
00416AFB mov [esp+8], si
00416B00 push 30h ; cell 0x30?
00416B02 mov [esp+0Eh], si
00416B07 push offset unk_40ACC8
00416B0C mov [esp+14h], si
00416B11 mov [esp+16h], si
00416B16 call sproRead
00416B1B test ax, ax
00416B1E jnz loc_416B8B
00416B20 lea eax, [esp+6]
00416B24 push eax
00416B25 push 34h ; cell 0x34?
00416B27 push offset unk_40ACC8
00416B2C call sproRead
00416B31 test ax, ax
00416B34 jnz loc_416B8B
00416B36 lea eax, [esp+8]
00416B3A push eax
00416B3B push 38h ; cell 0x38?
00416B3D push offset unk_40ACC8
00416B42 call sproRead
00416B47 test ax, ax
00416B4A jnz loc_416B8B
00416B4C lea eax, [esp+0Ah]
00416B50 push eax
00416B51 push 3Ch ; cell 0x3c?
00416B53 push offset unk_40ACC8
00416B58 call sproRead
00416B5D test ax, ax
00416B60 jnz loc_416B8B
00416B62 cmp word ptr [esp+4], 0DE9Bh
00416B69 jnz loc_416B8B
00416B6B cmp word ptr [esp+6], 0A17Ch
00416B72 jnz loc_416B8B
00416B74 cmp word ptr [esp+8], 9A8Fh
00416B7B jnz loc_416B8B
00416B7D cmp word ptr [esp+0Ah], 74BEh
00416B84 jnz short loc_416B8B
00416B86 mov esi, 1
.....
00416B8B mov eax, esi
00416B8D pop esi
00416B8E add esp, 8
00416B91 retn

Very interesting. If you force the compares to get good the program will give you an error and stop running. Besides as I stated above the only thing which got written to my dongle was the algorithm descriptor. By now we see that the version of the shell must be newer then the one used with DvdMaestro. To do some research I checked it with my dongle plugged in and it showed also the compares failed while the program got decrypted anyway. Trivial emulation needed and we'll continue. After this another call to sproRead occurs but it's cell 0 getting read so not of any importance.

Getting confused

As you probably realised by now they did some additional work to confuse people. We now get to the first sproQuery call and should be expecting the two tables. But they never come!? After alot of memory copying we will find our queries :-

004157E8 call ds:__imp_GetTickCount
004157EE add ds:dword_41391C, eax
004157F4 call ds:__imp_GetCurrentProcessId
004157FA xor ds:dword_41391C, eax
00415800 xor eax, eax
00415802 or ds:dword_41391C, 200001h
0041580C and ds:dword_41391C, 3FFFFFFFh

We see another attempt to get us confused. Next thing we do is to follow the response from the query. Soon we will see the same type of xor/zero compare. Now we check the xor operand and realize it's the same as our query. Of course there is a logical explanation for this. Checking the input parameters for sproQuery we see that 0 is the address to do query on. Knowing the SuperPro theory well we know that a algorithm descriptor must be 2 WORD cells next to each other starting on a even adress (8,10,12,...). Also querying a non-query cell always gives us the query back as response.

From above we see that the data we need is stored in dword_41391C. We can now implement our emulation :-

mov eax, [dword_41391C]
mov edx, (query from stack)
cmp eax,[edx]
jne next_query_check
mov edx, (response from stack)
mov [edx],eax
xor eax,eax
retn (displacement)

First query check down. We now continue and will find one of the table calls. From dvdmaestro we saw that those two table calls were placed next to each other. This fact we of course reuse to save time.

GetQuery:
004155F0 mov eax, ds:dword_40AC70
004155F5 mov eax, ds:dword_409010[eax*4]
004155FC retn

GetResponse:
00415600 mov eax, ds:dword_40AC70
00415605 mov eax, ds:dword_409090[eax*4]
0041560C retn

Breaking on both of these leads us towards the two xor operands. We will soon see they are the same as with DvdMaestro. 0xFD0699D6 and 0x7B2309A7. I already posted the emulation of this before so review DvdMaestro for it. The rest is just the 2 decryptions of .text and .data which is the same as before.

Conclusion

We clearly saw the two versions of Sentinel Shell were not the same. The last one v6.0 (newest version at the time of writing) has a bit more fooling around and fake checking. The decryption can be a little hard to overcome but not that bad. The only way to make things go faster is to make attack yourself instead of bruteforcing the entire array of DWORDs. The shell uses alot of memory copying and indirect addressing to hide away all the good stuff. However by following the data we saw it easily recovered. Another ready made protection is now broken.

Content of decrypt.cpp
-----------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#define Bytesize 0xd87e2				//1: 0xd87e2 2:0xE000

unsigned char szCryptedBuffer[Bytesize+1];
unsigned char szUnCryptedBuffer[Bytesize+1];
FILE *pF;
fpos_t pos = 0x2000;					//1: 0x2000 2:0x119000

void main()
{
	int iCounter= Bytesize/4;
	unsigned long tResponse = 0x342a3c33, Sum = 0;	//1: 0x342a3c33 2:0x1A610596
	unsigned long *lpUnCryptedBuffer = (unsigned long*)&szUnCryptedBuffer;
	unsigned long *lpCryptedBuffer = (unsigned long*)&szCryptedBuffer;
	memset(szUnCryptedBuffer, 0, Bytesize);
	memset(szCryptedBuffer, 0, Bytesize);
	if ((pF = fopen("dvdmaestro.exe", "r+b")) == NULL)
	{
		printf( "The file 'dvdmaestro.exe' was not opened\n" );
		return;
	}
	fsetpos(pF, &pos);
	if (fread(&szCryptedBuffer, 1, Bytesize, pF) != Bytesize)
	{
		printf("Error Bytesize bytes not read\n");
	}
	do
	{
		tResponse += (((tResponse << 4) + tResponse) >> 9) ^ (tResponse << 5);
		*(lpUnCryptedBuffer) = *(lpCryptedBuffer) ^ tResponse;
		Sum += *(lpUnCryptedBuffer);
		tResponse ^= Sum;
		++lpCryptedBuffer;
		++lpUnCryptedBuffer;
		--iCounter;
	} while ((iCounter+1) != 0);
	fsetpos(pF, &pos);
	if (fwrite(&szUnCryptedBuffer, 1, Bytesize, pF) != Bytesize)
	{
		printf("Error Bytesize bytes not written\n");
	}
	fclose(pF);
}
-----------------------------------------------------------------------
Content of dongle.cpp
-----------------------------------------------------------------------
#include <windows.h>
int __stdcall WinMain(HINSTANCE hInstance,HINSTANCE hPrevInstance,LPSTR 
lpCmdLine,int nCmdShow)
{
	MessageBox(0, "Dongle Found!", "Dongle Test...", MB_OK);
	return (0);
}
-----------------------------------------------------------------------

Return to Dongles Return to Main Index