Utf8

13 Sep 2014

Quite some time since I last wrote something. I recently started attending a University, so that took all of my time past weeks :)

Anyway, during a very easy lecture about programming I decided to actually program something myself, which turned out to be Unicode support for x64dbg!

Theory

At first I thought I would have to rewrite the command parser and whatnot, but that turned out to be not needed at all…

At first I wanted to convert every char pointer and constant to the Windows-supported wchar_t type, but this would take far too long to execute + it would break plugin compatibility and the complete internal API, which just sucks. Various discussions and a blogpost later I decided to use UTF-8 internally and call WinAPI with conversion functions.

Basically it required three things: 1) A C++ class like QString that allows string operations on UTF-8 string; 2) Conversion functions from UTF-8 to UTF-16 and the other way around; 3) ‘Converting’ all external ASCII calls to their UNICODE variant (WinAPI, TitanEngine, dbghelp, etc).

Practice

The first step was quite easy, I grabbed this GPLv3 UTF::String class and changed it to my needs. This became the UString class.

The second step was also very easy, the blogpost I mentioned earlier had two ready-to-use functions called ConvertFromUtf16ToUtf8 and ConvertFromUtf8ToUtf16. Those worked great, except that they would crash when fed with null as argument. Wrapping them in UString solved that issue without having to think :)

The third step seemed easy at first, I could debug a random application with a Chinese path within minutes. After that however, came a small moment of confusion, because Qt appears to be interpreting const char* strings as Latin1 per default. The following code solved this and after that the log etc. were working correctly:

// Set QString codec to UTF-8
QTextCodec::setCodecForLocale(QTextCodec::codecForName("UTF-8"));
QTextCodec::setCodecForCStrings(QTextCodec::codecForName("UTF-8"));
QTextCodec::setCodecForTr(QTextCodec::codecForName("UTF-8"));

Now all that is left is the tedious task of snooping through the code looking for incompatible GetModuleFileNameExA functions calls and convert them.

Concerns

The main concern will be that plugins will need to support UTF-8 and that new developers for x64dbg will have to adapt their coding a little. For plugin coders there will be conversion functions in the Bridge, but the conversion functions from the blogpost are really easy to copy-paste.

Conclusion

In overall adding UTF-8 support turned out to be quite easy and the work involved is just tedious, not really hard or very annoying. It can be done in little free time by almost anyone, so feel free to submit pull-requests :)

Leave a comment

Coverity

02 Sep 2014

Recently jvoisin contacted me on IRC (#x64dbg on Freenode) about Coverity:

Find defects in your Java, C# or C/C++ open source project for free.

Test every line of code and potential execution path.

The root cause of each quality or security defect is clearly explained, making it easy to fix bugs

Integrated with Github and Travis Ci

The best thing about all of this: It’s free for open source software. Simple register an account and then your open source project and you’re good to go. Before you can see the scan results they have to approve your project though.

For me the tricky part was building x64dbg with the command line. I never did this before and the documentation wasn’t very clear to me. Basically you run the following commands:

>Configure cov-build for MSVC
cov-configure --msvc
>Build into the required databases
cov-build --dir cov-int --instrument [command that builds here]

What was recommended on the internet was creating a script that fully builds your project. This is coverity_build.bat:

@echo off

echo Building DBG...
devenv /Rebuild "Release|x64" x64dbg.sln

echo Building GUI...
rmdir /S /Q build
mkdir build
cd build
qmake ..\src\gui\x64dbg.pro CONFIG+=release
jom

Notice that you have to be in the Visual Studio Command Prompt (+ Qt paths configured) for this to work. Just run coverity_setenv:

@echo off
echo Setting Qt in PATH
set PATH=%PATH%;c:\Qt\4.8.6-x64\bin
set PATH=%PATH%;c:\Qt\qtcreator-3.1.1\bin

call %comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat"" amd64

This will launch a new console, from there run coverity.bat:

@echo off
cov-configure --msvc
cov-build --dir cov-int --instrument coverity_build.bat

exit

After a long time (I have 6 cores, it still took me 5-10 minutes to build with cov-build), the building is finished and you have to ZIP the cov-int directory (not the files inside, the whole directory has to be in the ZIP).

When zipped, simply submit the build to your Coverity project and start analyzing errors.

Here is a screenshot of what the Coverity interface looks like: Problems in the GUI

See you all later,

mrexodia

Leave a comment

A New Blog

31 Aug 2014

Hey everyone, this is my first blogpost using Jekyll. My previous blog used WordPress and I never posted anything there. Hopefully it will be different this time :)

mrexodia

Leave a comment