Stepler
07-18-2009, 03:25 PM
Tibia Internals
Hello. It would be good to start with analysing what Tibia provides to us. Below is a code of Tibia packet receiver subsystem. It's a bit simplified but it shows all major parts:
void OnDataAvailable(...) //0x4F63C0
{
recv(...);
while(IsFullPacket)
{
Parser();
}
}
void Parser() //0x45B810
{
while(true)
{
int cmd = GetNextPacket();//0x45B845
if(cmd==-1) break;
switch(cmd)
{
//a big switch to process all packets...
}
}
}
The loop starts when a socket sink window receives an event that some data is available on the socket and OnDataAvailable(...) handler is called. It reads data by recv call and if a received bytes comtain at least one complete packet it starts calling Tibia parser for every packet. Remaining bytes are hold for the next recv event.
The next step in the line is Tibia Parser. It's just a loop with a huge switch with many cases to process every single logical command from the packet. At the beginning it calls a helper function to get next command
to process GetNextPacket(). The purpose of that helper is to manage packets and commands to provide commands one by one as long as one or more unparced packets are available.
GetNextPacket() is the main part of the puzzle. It is that function that can alone provide everything you need to use Tibia receiver subsystem. Internally it checks if we have a complete packet in the buffer, decodes it, sets internal pointer at the next logical command and return it's code (the first byte).
The next time Tibia calls it the next command will be returned. As soon as the packet is processed in full it will shift out processed packet from the buffer, update pointers, decrypt next available packet and so on...
It's clear that if we take GetNextPacket() under control then we can call Tibia parser as our own function and force it to get anything we want. And that's what we will do today.
You can also notice the name GetNextPacket and not GetNextCommand. Don't be cofused. It's because of historical reason, because the function works with both commands and complete packets.
Although before we start I would like to note that Tibia works with buffers in some kind of standardized way, by using structures to store all related info. Therefore it's good to use the same way in our code. Here is the structure,
I call it TPacketStream.
struct TPacketStream
{
LPVOID pBuffer; //pointer to the memory buffer
DWORD dwSize; //size of the buffer
DWORD dwPos; //current position to work with the buffer
};
All function in both subsystems Receiver and Sender in Tibia work using those streams. So it's useful to have it in our Tibia programms
whatever we code, be it a bot or not :)
The stream we need today is a Tibia parser stream. It's located at this address: 0x78BF24.
Let's start.
Sending plain decrypted packets to client
At first we need to init all data we need in our work. It's addresses of Tibia functions, stream and additional flag.
//addresses
#define FUNC_PARSER 0x45B810 //8.50
#define CALL_GET_NEXT_PACKET 0x45B845 //8.50
#define ADDR_RECV_STREAM 0x78BF24 //8.50
//structure Tibia uses to work with buffers
struct TPacketStream
{
LPVOID pBuffer;
DWORD dwSize;
DWORD dwPos;
};
//define types of functions we want to use or hook
typedef void TF_PARSER();
typedef int TF_GETNEXTPACKET();
//pointers to call original function
TF_PARSER *TfParser = (TF_PARSER*)FUNC_PARSER;
TF_GETNEXTPACKET *TfGetNextPacket = NULL;
//pointer to Tibia packet stream
TPacketStream * pRecvStream = (TPacketStream*)ADDR_RECV_STREAM;
//flag indicates that we are sending to client
BOOL fSendingToClient = FALSE;
The next usual thing we need is a function to hook call instruction in memory.
It injects a new address as operand and stores the old one to call original function later. Plus below is a line to actually to actually do injection.
void HookCall(DWORD dwCallAddress, DWORD dwNewAddress, LPDWORD pOldAddress)
{
DWORD dwOldProtect, dwNewProtect, dwOldCall, dwNewCall;
BYTE call[5] = {0xE8, 0x00, 0x00, 0x00, 0x00};
dwNewCall = dwNewAddress - dwCallAddress - 5;
memcpy(&call[1], &dwNewCall, 4);
VirtualProtectEx(GetCurrentProcess(), (LPVOID)(dwCallAddress), 5, PAGE_READWRITE, &dwOldProtect);
if(pOldAddress)
{
memcpy(&dwOldCall, (LPVOID)(dwCallAddress+1), 4);
*pOldAddress = dwCallAddress + dwOldCall + 5;
}
memcpy((LPVOID)(dwCallAddress), &call, 5);
VirtualProtectEx(GetCurrentProcess(), (LPVOID)(dwCallAddress), 5, dwOldProtect, &dwNewProtect);
}
//we hook a single CALL instruction in packet parser
//that calls GetNextPacket function
HookCall(CALL_GET_NEXT_PACKET, (DWORD)&OnGetNextPacket, (LPDWORD)&TfGetNextPacket);
Now let's write a new functionality for hooked GetNextPacket. Here we check if we are sending to client then we skip real body and just follow the original behavior by adjusting position in the stream and returning a first byte of the command. Otherwise we call original function unchanged.
int OnGetNextPacket()
{
if(fSendingToClient)
{
if(pRecvStream->dwPos < pRecvStream->dwSize)
{
//read the first byte of command to return from the function
BYTE bNextCmd = *((LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos);
//increase stream pointer since we've read the first byte
pRecvStream->dwPos++;
return (int)bNextCmd;
} else return -1;
}
return TfGetNextPacket();
}
Once GetNextPacket is under our control. We are free to play with parser.
Let's change the current packet stream for the stream with our own packet, call Tibia Parser function directly and then return back the old stream.
The result - a wonderful SendToClient.
void SendToClient(LPBYTE pBuffer, DWORD dwSize)
{
//turn on new behavior for GetNextPacket
fSendingToClient = TRUE;
//store Tibia recv stream
TPacketStream StreamHolder = *pRecvStream;
//point the stream to our buffer with packet
pRecvStream->pBuffer = pBuffer;
pRecvStream->dwSize = dwSize;
pRecvStream->dwPos = 0;
//call Tibia packet parser directly
TfParser();
//restore Tibia recv stream
*pRecvStream = StreamHolder;
//turn off new behavior for GetNextPacket
fSendingToClient = FALSE;
}
As you can see the complete code is very simple. It consists of two really small functions and involves only three addresses that are very easy to find and update. As a bonus I have looked into old Tibia and can confirm that the way it works wasn't changed within many years. So it is the most prefered
way to work with Tibia.
Receving separate packet commands in your app
Most of the projects for Tibia requires the ability to receive and process packets. It is very complex task to create a complete and working Parser for all possible packets. So instead of processing everything we can use Tibia internals to receive all commands separately, process only those we need and skip a complex map packet for example (and still be able to not miss anything).
I believe it's the best way to create an ideal bot core, so far I haven't seen anything more comfortable. Let's modify our function by a few lines to get the desired result.
int OnGetNextPacket()
{
int iCmd = TfGetNextPacket();
if(iCmd != -1)
{
LPBYTE pCmdBuffer = (LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos - 1;
//here you can process every single command from the packets
//pCmdBuffer is a pointer to the buffer with the command
//example: MyProcessCommands(pCmdBuffer);
}
return iCmd;
}
Very easy but solves a lot. Unfortunately by design you can't get the length of the current Command, since only Parser knows about that, but it is still possible to add some logic to process the commands with a single step lag
and calculate sizes. (I leave this for advanced programmers, there is no special magic about that)
Receving complete decrypted packets in your app
Ok, if you create a different kind of application (TibiaCam?) and don't need to process every single command but have to record complete packets
then you can do that as well. Let's modify our function a bit more.
int OnGetNextPacket()
{
int iCmd = TfGetNextPacket();
if(iCmd != -1)
{
if((pRecvStream->dwPos-1) == 8)
{
LPBYTE pWholePacket = (LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos - 1;
DWORD dwPacketSize = pRecvStream->dwSize - pRecvStream->dwPos + 1;
//now you can process complete packet
//example: MyParsePacket(pWholePacket, dwPacketSize);
}
}
return iCmd;
}
The things we have added is a check for current position in the stream. Since there are additional data at the beginning of the packet (totalsize, adler, size) with a constant size of 8 bytes, then every time the position is equal to 8 we have a new packet. Please also note that we should go back 1 byte before we check because original function has already read one byte.
Synchronization
The only thing that I moved out from this tutorial and a demo is thread synchronization part. Although the packets are synchronized between each others perfectly now, you also have to Choose a correct place to call SendToClient(LPBYTE pBuffer, DWORD dwSize). I advice to use socket
sink window procedure to do all work. The sequence to achive this could be:
1 - FindWindow OR hook for WSAAsyncSelect OR sink handle from memory
2 - SetWindowLong(hSink, GWL_WNDPROC...)
3 - New handler for events, SendToClient inside
Demo Application
I have attached a demo application. It consists of Injector.exe and Hook.dll and source code. It you run it in Game you will see a test message sent back to client every 2 seconds. The source code is very simplified, class-free and without thread synchronization.
The project is for Visual C++ 2008.
Improving
There is a way to make the technique more smart and able to replace and delete packets, Tibia provides everything we need for that. It is not trivial though, therefore I won't mess with it in this article. Also Tibia sender works in the similar way and can be hooked as well.
Conclusion
As you see the code I have described has one page in size with three addresses involved but it does everything we ever need from Tibia receiver. This technique was used with a success in a bot and tibiacam development and was tested very well.
History
Below are some topics you can read to know more about Tibia internals.
http://www.tpforums.org/forum/showthread.php?t=3327
http://www.tpforums.org/forum/showthread.php?t=3024
http://www.tpforums.org/forum/showthread.php?t=2399
We have much more, but I could not find those I wanted.
Thank you.
Stepler.
And sorry about my english. I will re-read my article and try to fix mistakes later.
Hello. It would be good to start with analysing what Tibia provides to us. Below is a code of Tibia packet receiver subsystem. It's a bit simplified but it shows all major parts:
void OnDataAvailable(...) //0x4F63C0
{
recv(...);
while(IsFullPacket)
{
Parser();
}
}
void Parser() //0x45B810
{
while(true)
{
int cmd = GetNextPacket();//0x45B845
if(cmd==-1) break;
switch(cmd)
{
//a big switch to process all packets...
}
}
}
The loop starts when a socket sink window receives an event that some data is available on the socket and OnDataAvailable(...) handler is called. It reads data by recv call and if a received bytes comtain at least one complete packet it starts calling Tibia parser for every packet. Remaining bytes are hold for the next recv event.
The next step in the line is Tibia Parser. It's just a loop with a huge switch with many cases to process every single logical command from the packet. At the beginning it calls a helper function to get next command
to process GetNextPacket(). The purpose of that helper is to manage packets and commands to provide commands one by one as long as one or more unparced packets are available.
GetNextPacket() is the main part of the puzzle. It is that function that can alone provide everything you need to use Tibia receiver subsystem. Internally it checks if we have a complete packet in the buffer, decodes it, sets internal pointer at the next logical command and return it's code (the first byte).
The next time Tibia calls it the next command will be returned. As soon as the packet is processed in full it will shift out processed packet from the buffer, update pointers, decrypt next available packet and so on...
It's clear that if we take GetNextPacket() under control then we can call Tibia parser as our own function and force it to get anything we want. And that's what we will do today.
You can also notice the name GetNextPacket and not GetNextCommand. Don't be cofused. It's because of historical reason, because the function works with both commands and complete packets.
Although before we start I would like to note that Tibia works with buffers in some kind of standardized way, by using structures to store all related info. Therefore it's good to use the same way in our code. Here is the structure,
I call it TPacketStream.
struct TPacketStream
{
LPVOID pBuffer; //pointer to the memory buffer
DWORD dwSize; //size of the buffer
DWORD dwPos; //current position to work with the buffer
};
All function in both subsystems Receiver and Sender in Tibia work using those streams. So it's useful to have it in our Tibia programms
whatever we code, be it a bot or not :)
The stream we need today is a Tibia parser stream. It's located at this address: 0x78BF24.
Let's start.
Sending plain decrypted packets to client
At first we need to init all data we need in our work. It's addresses of Tibia functions, stream and additional flag.
//addresses
#define FUNC_PARSER 0x45B810 //8.50
#define CALL_GET_NEXT_PACKET 0x45B845 //8.50
#define ADDR_RECV_STREAM 0x78BF24 //8.50
//structure Tibia uses to work with buffers
struct TPacketStream
{
LPVOID pBuffer;
DWORD dwSize;
DWORD dwPos;
};
//define types of functions we want to use or hook
typedef void TF_PARSER();
typedef int TF_GETNEXTPACKET();
//pointers to call original function
TF_PARSER *TfParser = (TF_PARSER*)FUNC_PARSER;
TF_GETNEXTPACKET *TfGetNextPacket = NULL;
//pointer to Tibia packet stream
TPacketStream * pRecvStream = (TPacketStream*)ADDR_RECV_STREAM;
//flag indicates that we are sending to client
BOOL fSendingToClient = FALSE;
The next usual thing we need is a function to hook call instruction in memory.
It injects a new address as operand and stores the old one to call original function later. Plus below is a line to actually to actually do injection.
void HookCall(DWORD dwCallAddress, DWORD dwNewAddress, LPDWORD pOldAddress)
{
DWORD dwOldProtect, dwNewProtect, dwOldCall, dwNewCall;
BYTE call[5] = {0xE8, 0x00, 0x00, 0x00, 0x00};
dwNewCall = dwNewAddress - dwCallAddress - 5;
memcpy(&call[1], &dwNewCall, 4);
VirtualProtectEx(GetCurrentProcess(), (LPVOID)(dwCallAddress), 5, PAGE_READWRITE, &dwOldProtect);
if(pOldAddress)
{
memcpy(&dwOldCall, (LPVOID)(dwCallAddress+1), 4);
*pOldAddress = dwCallAddress + dwOldCall + 5;
}
memcpy((LPVOID)(dwCallAddress), &call, 5);
VirtualProtectEx(GetCurrentProcess(), (LPVOID)(dwCallAddress), 5, dwOldProtect, &dwNewProtect);
}
//we hook a single CALL instruction in packet parser
//that calls GetNextPacket function
HookCall(CALL_GET_NEXT_PACKET, (DWORD)&OnGetNextPacket, (LPDWORD)&TfGetNextPacket);
Now let's write a new functionality for hooked GetNextPacket. Here we check if we are sending to client then we skip real body and just follow the original behavior by adjusting position in the stream and returning a first byte of the command. Otherwise we call original function unchanged.
int OnGetNextPacket()
{
if(fSendingToClient)
{
if(pRecvStream->dwPos < pRecvStream->dwSize)
{
//read the first byte of command to return from the function
BYTE bNextCmd = *((LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos);
//increase stream pointer since we've read the first byte
pRecvStream->dwPos++;
return (int)bNextCmd;
} else return -1;
}
return TfGetNextPacket();
}
Once GetNextPacket is under our control. We are free to play with parser.
Let's change the current packet stream for the stream with our own packet, call Tibia Parser function directly and then return back the old stream.
The result - a wonderful SendToClient.
void SendToClient(LPBYTE pBuffer, DWORD dwSize)
{
//turn on new behavior for GetNextPacket
fSendingToClient = TRUE;
//store Tibia recv stream
TPacketStream StreamHolder = *pRecvStream;
//point the stream to our buffer with packet
pRecvStream->pBuffer = pBuffer;
pRecvStream->dwSize = dwSize;
pRecvStream->dwPos = 0;
//call Tibia packet parser directly
TfParser();
//restore Tibia recv stream
*pRecvStream = StreamHolder;
//turn off new behavior for GetNextPacket
fSendingToClient = FALSE;
}
As you can see the complete code is very simple. It consists of two really small functions and involves only three addresses that are very easy to find and update. As a bonus I have looked into old Tibia and can confirm that the way it works wasn't changed within many years. So it is the most prefered
way to work with Tibia.
Receving separate packet commands in your app
Most of the projects for Tibia requires the ability to receive and process packets. It is very complex task to create a complete and working Parser for all possible packets. So instead of processing everything we can use Tibia internals to receive all commands separately, process only those we need and skip a complex map packet for example (and still be able to not miss anything).
I believe it's the best way to create an ideal bot core, so far I haven't seen anything more comfortable. Let's modify our function by a few lines to get the desired result.
int OnGetNextPacket()
{
int iCmd = TfGetNextPacket();
if(iCmd != -1)
{
LPBYTE pCmdBuffer = (LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos - 1;
//here you can process every single command from the packets
//pCmdBuffer is a pointer to the buffer with the command
//example: MyProcessCommands(pCmdBuffer);
}
return iCmd;
}
Very easy but solves a lot. Unfortunately by design you can't get the length of the current Command, since only Parser knows about that, but it is still possible to add some logic to process the commands with a single step lag
and calculate sizes. (I leave this for advanced programmers, there is no special magic about that)
Receving complete decrypted packets in your app
Ok, if you create a different kind of application (TibiaCam?) and don't need to process every single command but have to record complete packets
then you can do that as well. Let's modify our function a bit more.
int OnGetNextPacket()
{
int iCmd = TfGetNextPacket();
if(iCmd != -1)
{
if((pRecvStream->dwPos-1) == 8)
{
LPBYTE pWholePacket = (LPBYTE)pRecvStream->pBuffer + pRecvStream->dwPos - 1;
DWORD dwPacketSize = pRecvStream->dwSize - pRecvStream->dwPos + 1;
//now you can process complete packet
//example: MyParsePacket(pWholePacket, dwPacketSize);
}
}
return iCmd;
}
The things we have added is a check for current position in the stream. Since there are additional data at the beginning of the packet (totalsize, adler, size) with a constant size of 8 bytes, then every time the position is equal to 8 we have a new packet. Please also note that we should go back 1 byte before we check because original function has already read one byte.
Synchronization
The only thing that I moved out from this tutorial and a demo is thread synchronization part. Although the packets are synchronized between each others perfectly now, you also have to Choose a correct place to call SendToClient(LPBYTE pBuffer, DWORD dwSize). I advice to use socket
sink window procedure to do all work. The sequence to achive this could be:
1 - FindWindow OR hook for WSAAsyncSelect OR sink handle from memory
2 - SetWindowLong(hSink, GWL_WNDPROC...)
3 - New handler for events, SendToClient inside
Demo Application
I have attached a demo application. It consists of Injector.exe and Hook.dll and source code. It you run it in Game you will see a test message sent back to client every 2 seconds. The source code is very simplified, class-free and without thread synchronization.
The project is for Visual C++ 2008.
Improving
There is a way to make the technique more smart and able to replace and delete packets, Tibia provides everything we need for that. It is not trivial though, therefore I won't mess with it in this article. Also Tibia sender works in the similar way and can be hooked as well.
Conclusion
As you see the code I have described has one page in size with three addresses involved but it does everything we ever need from Tibia receiver. This technique was used with a success in a bot and tibiacam development and was tested very well.
History
Below are some topics you can read to know more about Tibia internals.
http://www.tpforums.org/forum/showthread.php?t=3327
http://www.tpforums.org/forum/showthread.php?t=3024
http://www.tpforums.org/forum/showthread.php?t=2399
We have much more, but I could not find those I wanted.
Thank you.
Stepler.
And sorry about my english. I will re-read my article and try to fix mistakes later.