5

I am using ollydbg and a Hex editor. I confirmed that once the application is edited in any way it behaves different than normal.

My first thought was that the file is checking the checksum value so I looked at the intermodular calls in olly and did not see anything about checksum. I was specifically looking for MapFileAndCheckSum

But I am trying to reason this out, I am thinking that a checksum value has to be hard coded in the file so it can be compared to the actual checksum. So I am wondering from the developers point of view how is it possible to get the checksum value to be hard code when the application is not complete/compiled

Which brings me to the question. What ways is there for an application to detect that it has been modified?

------- EDIT ------- Additional Information ------

I have been doing some testing and I have to say I'm baffled as to where the checksum value is being stored.

  1. There are no connections to the internet.
  2. Only one dll comes with the application (I extracted the installer files manually) the dll file is old and was last modified prior to the application. I even compared it to an earlier version of the application that did not have this checksum check and the dll is identical.
  3. I taught that maybe the checksum value would be entered in the registry by the installer so I extracted the .exe and .dll to a separate computer that has never used the installer. Changes are still being detected!
  4. It is definitely a checksum test, as I have changed a single byte of padding by from 00 to 20 and the change is detected. If I edit back to 00 to application performs normally.

So now I am wondering would it be possible to calculate what the checksum is going to be before entering the hard coded checksum value? I do realize that the actual checksum value will change when changing the hard coded checksum value. I want to know if there is any method to predetermine a checksum value when hard coding and finding a match. Seems impossible but I cannot think of any other means considering the situation.

GuYY
  • 65
  • 1
  • 6

3 Answers3

4

The application can check only a part of itself, excluding the checksum part. You can also have a runtime decryption, that will be wrongly decrypted if tampered with. There are several research projects in this area.

jvoisin
  • 2,516
  • 16
  • 23
2

I'm assuming you've hex edited the actual file. In that case, the application may be reading itself from disk and manually calculating a checksum. Most likely it uses one of the following to load the file into memory:

  • CreateFile/OpenFile with ReadFile (and all their suffixed forms)
  • CreateFileMapping and MapViewOfFile (and all their suffixed forms)

You can use Process Monitor to conveniently monitor file-related API calls and filter by process name, or just bp everything.

Also remember that a checksum could be stored in dll's/any other files that come with the application, or downloaded from the internet.

higaki
  • 473
  • 2
  • 8
  • yes I hex edited the file. I am not that familiar with Process Monitor, I have found CreateFile and ReadFile on the application. Would the calculated checksum value be visible in process monitor? If so where do I have to look? – GuYY Feb 11 '14 at 06:25
  • Did you change the original file on disk? If so, then practically, checking on byte basis memory with disk version should not show any difference. – PhoeniX Feb 11 '14 at 07:31
  • not 100% I understand "on disk" I tried to permanently hex edit with hex application and change is detected, when I do the hex edit within olly change is not detected. However I need to make the hex change permanent – GuYY Feb 11 '14 at 07:38
  • so, if you only change in memory, during application execution, the change is not detected? On disk, I mean that you edit the exe with hex editor and then execute it. – PhoeniX Feb 11 '14 at 09:00
  • yes that is correct if I change hex when opened in olly change is not detected. I am doing the change right after I open the application in olly before running. Is it possible that the checksum check is being done before running? initializing? – GuYY Feb 11 '14 at 09:34
  • break on entry point of an application (and check that there are no TLS calls - CFF explorer) and make your change. After the change, continue the execution and see whether the change is detected. – PhoeniX Feb 11 '14 at 09:41
  • did as you said, change not detected. was editing my comment while you were answering. not sure what TLS calls are, I downloaded CFF explorer now, what am I looking for here? – GuYY Feb 11 '14 at 10:04
  • It looks, that the check is done for the application executable on disk. As user3262342 wrote, try to check whether the application is reading its own file from disk, break on CreateFileA(W) or NtCreateFile and continue from there. – PhoeniX Feb 11 '14 at 13:35
  • please check the question as I have added some additional information – GuYY Feb 12 '14 at 08:17
0

MapFileChecksum is used to calculate the checksum of the executable that's stored in the IMAGE_OPTIONAL_HEADER structure using Microsoft's custom checksum. It's usually used to re-calculate the checksum value once the executable has been modified, since the windows kernel checks the checksum before it loads drivers and system files. But in case of your application, the application might be using any kind of checksum (crc32, adler, e.t.c.) or might even be using hash functions. So just breaking on MapFileChecksum might not be enough. Since it has to read it's own binary to calculate the checksum/hash, break on the file manipulation functions and go from there.

shebaw
  • 685
  • 6
  • 13