GeistHaus
log in · sign up

Hooked on Mnemonics Worked for Me

Part of blogger.com

stories
Frost64
Show full content

At my day job, I run a RE mentorship. It’s a simple process for me. I have a link to the task, and the user emails me the answers to the questions in the task. If their answers are correct, I send them the next task. If the answers are incorrect, I let them know their mistakes and send them references to help. Sometimes people get tripped up with learning Assembly, or they think it’s boring (which it kind of is). One tool I have always wanted was a way to gamify the learning of Assembly.  So, my vibe-coding weekend project was a SoftIce-like interface that teaches the basics of Assembly. It uses the Godot gaming engine, Unicorn-Engine for emulation, and iced. I hit my quota for the next couple of hours I figured I’d post a screenshot:



Overall I’m very happy with it. It’s a complete emulator, has correct addresses, bytes, validation and reminds me of SoftIce. I still need to audit the text in the lesson notes because they are duplicated in the command window text. Once it’s complete I’ll post the code to GitHub or maybe post the game on Steam.  

I’m still working on my Stressing LLMs project. Some local models that I tested failed miserably on writing basic python decryptors. I have been digging into proper prompts, skills.md and settings for Ghidra. That project should be out in a couple of weeks. 


tag:blogger.com,1999:blog-4093139800580227296.post-1972157180297157573
Extensions
Stressing LLMs - Triage Stage
Show full content

Packers, cryptors, and code obfuscation are all methods used to bypass signature-based scanners in AV/EDR or to slow down the reverse engineering process. Many people are now using Large Language Models (LLMs) to reverse engineer or thwart these protections. It is increasingly common to see examples of frontier models solving CTF challenges or being used to port old video games to modern code. It is somewhat morbidly fascinating to consider how LLMs could drive an arms race with DRM systems.

When thinking about LLMs for reverse engineering, I keep asking: at what point does randomization degrade tokenization or code comprehension? This is a reasonable question in the context of compiled executables.

In my view, there are two types of potential attacks against LLMs in the context of static analysis of compiled binaries. The first is making the code so complex that the context size and token cost are no longer practical. The second, which I call “Tokenization Inflation,” attempts to inflate or fragment tokens to increase processing cost or reduce coherence. These “attacks” may not even be effective against LLMs, especially for trivial tasks, but they are still worth exploring. This is the first of two posts: this one outlines the approaches and code; the second tests the hypothesis.

Complexity Attack

The complexity attack increases computational complexity by generating binaries with a large number of interdependent functions. Instead of hiding the logic, the goal is to make the amount of state and code too large for practical static reasoning. The executable contains a toy XOR cipher with a keystream derived from a set of N functions, where N is the number of generated rounds. A Python script generates C source code with an embedded encrypted string and decryption loop. GCC is then used to compile the C source into an executable. At runtime, the decrypted string is printed to the console. To make this concrete, we can walk through generating the code and compiling it.

python gen_fixture.py generate --seed 0xdeadbeef --rounds 5 --symbol-len 16 --symbol-pad 0 --message "Hello, World" --out fixture.c
Wrote fixture.c
Generated symbol prefix length: 16
Compile with:
gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture.c -o fixture.exe

Here is fixture.c. There are 5 functions named TokenizerBench, which matches the number of rounds specified on the command line. If this were increased to 16,397 rounds, the generated binary would contain 16,397 functions.

// Generated CTF-style static-analysis fixture
//
// Suggested build:
//   gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture.c -o fixture.exe
//
// seed=0xdeadbeef
// rounds=5
// const_mode=rand
// const_seed=0xc001d00d
// symbol_len=16
// symbol_pad=0
// generated_prefix_length=16
//
// Notes:
// - Per-function constants are baked into each generated function.
// - Function bodies vary by generated round variant.
// - The plaintext is stored encrypted in the binary and decrypted at runtime.

#include <stdint.h>
#include <stdio.h>
#include <stddef.h>

typedef struct TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens {
    uint64_t a;
    uint64_t b;
    uint64_t c;
} TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens;

static uint32_t xorshift32(uint32_t x) {
    x ^= x << 13;
    x ^= x >> 17;
    x ^= x << 5;
    return x;
}

__attribute__((used, noinline))
uint32_t TokenizerBench___R0(TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens *p) {
    uint32_t m1 = xorshift32(0x9336956du ^ 0x31bbf978u ^ (uint32_t)p->a);
    uint32_t m2 = xorshift32(0xcd6f55fcu ^ (uint32_t)p->b);
    p->a ^= ((uint64_t)m1 << 32) | (uint64_t)m2;
    p->b += (uint64_t)(0x9336956du ^ m2);
    p->b = (p->b << 10) | (p->b >> 54);
    p->c = (p->c + p->a) ^ (uint64_t)(0x31bbf978u ^ 0xcd6f55fcu);
    uint64_t r = p->a ^ p->b ^ p->c ^ (uint64_t)0x9336956du ^ (uint64_t)0x31bbf978u ^ (uint64_t)0xcd6f55fcu;
    return (uint32_t)(r ^ (r >> 32));
}

__attribute__((used, noinline))
uint32_t TokenizerBench___R1(TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens *p) {
    uint32_t m = xorshift32(0x366856bbu ^ (uint32_t)p->a);
    p->a ^= ((uint64_t)0x366856bbu << 32) | (uint64_t)m;
    p->b += p->a ^ (p->c + (uint64_t)0x72fcd409u);
    p->c = ((p->c ^ (uint64_t)0x3afd4cabu) << 24) | ((p->c ^ (uint64_t)0x3afd4cabu) >> 40);
    uint64_t r = p->a ^ p->b ^ p->c ^ (uint64_t)0x366856bbu ^ (uint64_t)0x72fcd409u ^ (uint64_t)0x3afd4cabu;
    return (uint32_t)(r ^ (r >> 32));
}

__attribute__((used, noinline))
uint32_t TokenizerBench___R2(TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens *p) {
    uint32_t m = xorshift32(0x046d6ad2u ^ (uint32_t)p->b);
    p->b ^= ((uint64_t)m << 32) | (uint64_t)0xc719f452u;
    p->c += p->b ^ (uint64_t)0x0fc1bdd9u;
    p->a = (p->a + (uint64_t)0x046d6ad2u);
    p->a = (p->a >> 21) | (p->a << 43);
    uint64_t r = p->a ^ p->b ^ p->c ^ (uint64_t)0xc719f452u ^ (uint64_t)0x046d6ad2u ^ (uint64_t)0x0fc1bdd9u;
    return (uint32_t)(r ^ (r >> 32));
}

__attribute__((used, noinline))
uint32_t TokenizerBench___R3(TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens *p) {
    uint32_t m = xorshift32(0xc55b15eeu + (uint32_t)p->c);
    p->a += ((uint64_t)m << 32) | (uint64_t)0x0d11e683u;
    p->c ^= p->a;
    p->c = (p->c >> 16) | (p->c << 48);
    p->b ^= (uint64_t)(0xc8e57b40u + m);
    uint64_t r = p->a ^ p->b ^ p->c ^ (uint64_t)0xc8e57b40u ^ (uint64_t)0x0d11e683u ^ (uint64_t)0xc55b15eeu;
    return (uint32_t)(r ^ (r >> 32));
}

__attribute__((used, noinline))
uint32_t TokenizerBench___R4(TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens *p) {
    uint32_t m1 = xorshift32(0xdaf09eaeu ^ 0xf6f1f787u ^ (uint32_t)p->a);
    uint32_t m2 = xorshift32(0xe0cf500du ^ (uint32_t)p->b);
    p->a ^= ((uint64_t)m1 << 32) | (uint64_t)m2;
    p->b += (uint64_t)(0xdaf09eaeu ^ m2);
    p->b = (p->b << 21) | (p->b >> 43);
    p->c = (p->c + p->a) ^ (uint64_t)(0xf6f1f787u ^ 0xe0cf500du);
    uint64_t r = p->a ^ p->b ^ p->c ^ (uint64_t)0xdaf09eaeu ^ (uint64_t)0xf6f1f787u ^ (uint64_t)0xe0cf500du;
    return (uint32_t)(r ^ (r >> 32));
}

__attribute__((used, noinline))
uint32_t derive_state(uint32_t seed) {
    TokenizerBench___Type__LongRecord__With__Lots__Of__Nested__Like__Tokens x = {
        seed,
        seed ^ 0x12345678ULL,
        seed + 0x9ULL
    };

    uint32_t s = seed;
    s ^= TokenizerBench___R0(&x);
    s ^= TokenizerBench___R1(&x);
    s ^= TokenizerBench___R2(&x);
    s ^= TokenizerBench___R3(&x);
    s ^= TokenizerBench___R4(&x);

    s = xorshift32(s);
    return s;
}

int main(void) {
    uint8_t encrypted[] = { 0xcf, 0x7a, 0xe5, 0x10, 0x3c, 0x49, 0xe6, 0x0b, 0x79, 0xcb, 0xf9, 0x3d, 0x00 };
    uint32_t s = derive_state(0xdeadbeef);

    for (size_t i = 0; i < sizeof(encrypted) - 1; i++) {
        s = xorshift32(s + 0xA5A5A5A5u);
        encrypted[i] ^= (uint8_t)(s & 0xffu);
    }

    puts((const char *)encrypted);
    return 0;
}

Below is the creation and execution of a 100,000-round binary.

python gen_fixture.py generate --seed 0xdeadbeef --rounds 100000 --message "Hello, World" --out fixture-100k.c --symbol-len 16 --symbol-pad 0
Wrote fixture-100k.c
Generated symbol prefix length: 16
Compile with:
gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture-100k.c -o fixture-100k.exe

gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture-100k.c -o fixture-100k.exe
.\fixture-100k.exe
Hello, World

The 100k-function binary was over 55 MB. Dynamic analysis could bypass this obfuscation with a single breakpoint, but the focus here is static analysis. The interesting part is that the number of functions scales easily for testing, and each function contributes to the final state. If the analysis is incomplete or incorrect, the derived decryption key will also be incorrect.

Tokenization Inflation

Once a prompt is sent to an LLM, it is tokenized into integers. A simple way to think about this is mapping chunks of text to IDs. These IDs are then used to index into the model’s embedding table. This may seem similar to compression algorithms, since both map variable-length sequences to codes. The difference is that tokenization uses a fixed vocabulary optimized for model performance, while compression builds or applies dictionaries to reduce size by exploiting repetition in the data.

A potential weakness in both compression and tokenization is that long inputs increase computational cost. Repetitive or structured data can also affect how efficiently it is represented as tokens. Most modern implementations handle this reasonably well, but there is still a cost.

In an executable, one of the most common ways to introduce large amounts of data is through strings. However, not all strings are surfaced or prioritized during analysis. One type of string that is often preserved and exposed is debug information. With GCC, DWARF debug metadata can be used to store extremely long function names. We can generate function names of arbitrary length using the Python script. By passing -g3 -gdwarf-5, GCC emits DWARF metadata. Disassemblers such as Binary Ninja, Ghidra, and IDA can read this metadata, recover the names, and in some workflows pass them along to an LLM, which then tokenizes the text. The following command generates 5 rounds with a function name length of 7,331 characters.

python gen_fixture.py generate --seed 0xdeadbeef --rounds 5 --message "Hello, World" --out fixture-p.c --symbol-len 7331 --symbol-pad 1337
Wrote fixture-p.c
Generated symbol prefix length: 8672
Compile with:
gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture-p.c -o fixture-p.exe

gcc -O0 -g3 -gdwarf-5 -fno-omit-frame-pointer -fno-inline -std=c11 fixture-p.c -o fixture-p.exe

Below is a screenshot of a graph view in IDA. It gives a sense of how long the function names are, although they are truncated after 1024 characters in IDA.

Here is an example of a complete function name.

uint32_t __cdecl TokenizerBench__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char___0(TokenizerBench__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__DemangleLike__std__basic_string__char__std__char_traits__char__std__allocator__char__vector__pair__basic_string__int__PadXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_Type__LongRecord__With__Lots__Of__Nested__Like__Tokens_0 *p) 
Combining the code complexity option with the large function names makes it so a large corpus of similar strings would be generated in a single function which might be taxing on a tokenizer. This attack can be easily defeated by not loading the debug/dwarf strings in the disassembler.Summary

The goal is not to make binaries impossible to reverse, but to push LLM-based analysis into inefficient paths. One approach scales interdependent functions to force complexity. The other inflates token-heavy inputs through debug metadata to stress context limits and attention costs. These are better understood as attempts to trigger worst-case behavior in the analysis pipeline, not attacks on tokenization itself. This highlights a shift in where the pressure points are within LLMs. Context windows, token budgets, and attention scaling become part of the attack surface. If LLMs are used in reverse engineering workflows, understanding where they degrade may matter as much as improving their capability.

The next step is validating whether these ideas actually hold up in practice. That means testing them in a way in which I don't go broke with token cost and/or get banned by Anthropic or OpenAI. Odds are my first attempts will be locally using resources referenced in this gist.

Feel free to send me an email if you have any ideas at alexander dot hanel at gmail dot com.

Here is the source code: https://github.com/alexander-hanel/StressingLLMs

tag:blogger.com,1999:blog-4093139800580227296.post-1735014638427111153
Extensions
Codex’s Model Interaction & Inter-process Communication
Show full content
Over the weekend I explored OpenAI’s Codex source code using Codex with the goal of understanding how it sends, receives, and processes responses from the API. Here is a link to the report. While going through it, I started thinking about inter-process communication (IPC) between Codex and other processes. In the coding agents I’m familiar with (Anthropic’s Claude Code and OpenAI’s Codex), there isn’t much support for receiving input from external processes, which raises a question I’ve had for a while: how does a third-party process communicate with the agent in a meaningful way? For example, if a command is blocked by a security provider like an EDR, the agent could simply generate a slightly modified version of that command and try again. But how would it know the block was due to a security event and shouldn’t be retried?
To explore this, I had Cursor modify its source to add IPC. The forked vibe coded version compiles and connects to OpenAI’s API just like a normal Codex install. It has a local file-based IPC surface so secondary processes can discover active sessions and submit feedback which gets added to the models context. While testing with multiple Codex instances, I accidentally ran a command to update README.md in the wrong terminal. Earlier, I had injected an “external security control” signal via a Python script, and the session responded with: 
“No. README.md was not updated. The edit attempt on README.md was blocked by an external security control, and the runtime indicated not to retry until that condition is cleared.” 
At first it didn’t make sense, then it registered that the previous tests had worked and the session context had been updated through the IPC channel. I thought it was fascinating because it opened a whole almost philosophical question: how would AI-Agents safely evaluate prompts from remote processes in the context of its current task? It shows how much trust matters in the context of AI-Agents. 
The README.md of the project is constructed as a learning guide. Enjoy. 
tag:blogger.com,1999:blog-4093139800580227296.post-8564756423056052447
Extensions
Agentic AI Security: Reviewing the Past to Predict the Future
Show full content

OpenAI recently posted a role for a Cybersecurity Landscape Analyst within their Intelligence and Investigation team. One line stood out:

“Develop forward-looking assessments of how cyber threats may evolve over 6–24 months.”

To predict the future of Agentic AI, we only need to look to the past. Agentic AI security is not emerging from nothing. It is replaying the same history as traditional computing security, but within a compressed timeline.

As of this writing, prompt injection is a commonly discussed attack vector against LLM-based systems. At its core, prompt injection exists because LLMs are sequence predictors with no native separation between trusted control instructions (system prompts) and untrusted input (user data). This is not a new problem. This is basically Intel x86 in Real Mode.

In Real Mode, code, data, the stack, and even the interrupt vector table all share the same memory space. There is no privilege separation. Any instruction can jump anywhere, overwrite anything, and execute without restriction. The fundamental issue is identical: no boundary between control and data. Detection strategies in that era relied on pattern matching, heuristics, checksums, and runtime hooking. Modern defenses against prompt injection, such as guardrails, input filtering, and heuristic detection, are not that different. They are variations of the same reactive strategies used before architectural fixes existed.

What about forward-looking cyber threats like the first Agentic-AI worm? For this example, we could  consider the Morris Worm in 1988. Its success was not due to a single vulnerability, but an environment characterized by high trust between systems, widespread exposure of network services, weak authentication mechanisms, and a highly connected user base.

Now map this to Agentic AI. Instead of network services like sendmail, finger, or rsh, we have tool-enabled agents such as OpenClaw. Instead of academic researchers, we have early adopters rapidly integrating these systems into real workflows. Instead of BSD Unix systems in academic environments, we have Mac Minis showing up in homes and offices because people want to run OpenClaw locally. Instead of executable payloads, we have prompts. The conditions for a worm are the same: trust, connectivity, and execution capability. What is currently missing is density. There are not yet enough interconnected, tool-enabled systems for large-scale, worm-like propagation comparable to the Morris Worm or Slammer

My theory is that the same threats, along with the security mitigations developed to address them since the 60s and 70s, will replay themselves within the microcosm of Agentic AI. We are currently in DOS Mode for Agentic AI. 

Update: A colleague shared the following link 

https://arxiv.org/abs/2403.02817


 

 


tag:blogger.com,1999:blog-4093139800580227296.post-5748546899964593523
Extensions
LLMs != Security Products
Show full content

Cybersecurity stocks took a dive after Anthropic released a blog post titled “Making frontier cybersecurity capabilities available to defenders" What stood out was not the post itself, but the market reaction. Companies tied to endpoint protection, cloud security, and other traditional cybersecurity products were affected, even though the post had little direct relevance to those companies.

That reaction highlights a disconnect between the perceived capabilities of “AI” and its actual impact on cybersecurity products, a disconnect that likely extends beyond the market. To make sense of that gap, it helps to start with what is actually meant by ‘AI’ in this context. Usage of the term AI (short for Artificial Intelligence) has increased sharply since the release of ChatGPT in November of 2022. In practice, much of what is labeled “AI” today is better described as large language models (LLMs). For readers unfamiliar with LLMs, a common definition is:

“A large language model (LLM) is a type of artificial intelligence that can understand and create human language. These models learn by studying huge amounts of text from books, websites, and other sources.”

What makes LLMs fascinating and applicable to our modern life is how they solved (on a surface level) a field of AI called Natural Language Processing (NLP). For readers not familiar with NLP, autocomplete, email spam filters and auto-correct are all examples of NLP. Here is a definition of NLP.  

“A field in Artificial Intelligence, and also related to linguistics, focused on enabling computers to understand and generate human language.”

Long-time readers of this blog may recall that I previously used a sub-field of NLP, Natural Language Generation (NLG) to automatically create descriptions of disassembled functions via API calls. On their own, LLMs require text for both training and inference. They are not autonomous systems;  without prompts, they do not function. This distinction is important when discussing AI and cybersecurity, because evaluating or classifying security events requires context that does not exist as text as input to a prompt. That context has to be generated by additional software.

Generating the context requires an understanding and access to the complete lifecycle of the security event that is being used for the context. Walking through this lifecycle matters because it highlights how much logic exists before an event ever becomes text.

A classic example of a security event is a process initiating an outbound network connection directly to an IP address. How that event is handled varies widely depending on the type of security product and where it operates in the OSI model. For this example, assume the product operates at Layer 7, the application layer. The event pipeline in this case includes several distinct steps. A kernel-mode driver or user-mode component monitors process creation and relevant networking APIs. The destination IP address is evaluated to ensure it is not local, then serialized into text and logged. That log data is subsequently forwarded to a file-based or cloud-based centralized logging system. Even this simplified path omits important actions such as blocking the connection or terminating the process. Writing code is not the same as building a security product, and LLMs do not possess the authority or signal access required to determine whether an IP address is benign or malicious. An LLM can describe an alert very well; it cannot, on its own, determine whether that alert represents malicious behavior without pre-existing detection logic, telemetry, or intelligence-derived indicators of compromise.

In practice, an agent is an LLM placed inside a loop, where it can inspect the current state of a system, run tools or commands, observe the results, and decide what to do next until it reaches some stopping point. Without the output of those tools and commands, the LLM provides no value; it has nothing to reason over. The surrounding software is what produces the text that gives the model context.

As of this publication date, LLMs are not going to replace cybersecurity products. These systems are large, long-lived codebases, and their value is not defined by code generation alone. What matters is the telemetry collected and the logic built on top of that telemetry to determine whether the text describing an event represents something benign or something malicious. Large language models can help explain security events, but they don’t replace the systems that detect them, and confusing the two is how markets end up reacting to the wrong things.

tag:blogger.com,1999:blog-4093139800580227296.post-5558360803804730927
Extensions
msdocsviewer
Show full content

Hello, 

I forgot to post a recent IDAPython plugin that I created for viewing Microsoft SDK documentation in IDA. Here is an example screenshot of msdocsviewer .


 The repository for the plugin can be found here.

tag:blogger.com,1999:blog-4093139800580227296.post-1793734102733311484
Extensions
Function Trapper Keeper - An IDA Plugin For Function Notes
Show full content

Function Trapper Keeper is an IDA plugin for writing and storing function notes in IDBs, it’s a middle ground between function comments and IDA’s Notepad. It’s a tool that I have wanted for a while. To understand why I wanted Function Trapper Keeper, it might be worth describing my process of reverse engineering a binary in IDA. 

Upon opening a binary, I always take note of the code to data ratio. This is can be inferred by looking at the Navigator band in IDA. If there is more data than code in the binary, it can hint that the binary is packed or encrypted. If so, I typically stop the triage of the binary to start searching for cross-references to the data. In many instances the cross-references can lead to code used for decompressing or decrypting the data. For example, if the binary is a loader it would contain the second stage payload encrypted or some other form of obfuscation. By cross-referencing the data and finding the decryption routine of the loader, I can quickly pivot to extracting the payload. Another notable ratio is if the data or code is not consistent. If the code changes from data to code and back, it is likely that the analysis process of IDA found inconsistencies in the disassembled functions. This could be from anti-disassemblers, flawed memory dumps or something else that needs attention. After the ratios, I look at the strings. I look for the presence of compilers strings, strings related to DLLs and APIs, user defined strings or the lack of user defined strings. If the latter, I’ll start searching for the presence of encrypted strings and then cross-referencing their usage. This can help find the function responsible for string decryption. If I can’t find the string decryption routine, I’ll use some automation to find all references to XOR instructions. After reviewing strings, I’ll do a quick triage of imported function. I like to look for sets of APIs that I know are related to certain functionality. For example, if I see calls to VirtualAlloc, VirtualProtect and CreateRemoteThread, I can infer that process injection is potentially present in the binary.

After the previously described triage, I have high-level overview of the binary and usually know if I should do a deep dive of the binary or if I need to focus on certain functionality (encrypted strings, unpacked, etc). If I’m doing a deep dive I like to label all functions. For my IDBs, the name of the function hints at my level of understanding of the function. The more descriptive the function name, the more I know about it. If I know the function does process injection into explorer.exe I might name it “inject_VirtRemoteThreadExplorer”. If I don’t care about the function but I need to note it’s related to strings and memory allocation I might label it “str_mem”. If I’m super lazy I might name the function “str_mem_??”, and yes you can use “?” in IDA’s function names. This is a reminder that I should probably double check the function if it’s used a lot. Once I have all the functions labeled, I can be confident of the general functionality of the binary. This is when I start digging deeper into the functions.

This can vary but with lots of malware families a handful of the functions contain the majority of the notable functionality. This is commonly where I spend the most of my time reversing. I have said it before in a previous post, that if you aren’t writing then you aren’t reversing. Since I spend lots of time in these functions, I like to have my notes close by. Notes can be added as Function comments but the text disappears once you scroll down the function, plus the text can’t be formatted or the function comments can’t be easily exported and IDA’s Notepad suffers from the same issues (minus the export). Having all the function notes in a single pane and being able to export than to markdown is super helpful. My favorite feature of the plugin is when I scroll from function to function the text refreshes for each function.  The plugin can be seen in the right of the following image. 

Having a description accessible minimizes the amount of time I have to read code I already reversed, which is useful when opening up old IDBs. I hope others find it as useful as I do. 

Here is a link to the repo.

For more information on the Navigation band in IDA check out Igor’s post.

Please leave a comment, ping me on twitter or mastodon or create an issue on GitHub

tag:blogger.com,1999:blog-4093139800580227296.post-4498550247296973306
Extensions
Recommended Resources for Learning Cryptography: RE Edition
Show full content

A common question when first reverse engineering ransomware is “what is a good resource for learning cryptography?”. Having an understanding of cryptography is essential when reversing ransomware. Most reverse engineers need to know how to identify the encryption algorithm, be able to follow the key generation, understand key storage and ensure the encryption implementation isn’t flawed. To accomplish these items it is essential to have a good foundational knowledge of cryptography. The following are some recommendations that I have found beneficial on my path to learning cryptography.  

One of the most important skills is having an understanding of how common encryption algorithms work. The best introductory book on cryptography is Understanding Cryptography: A Textbook for Students and Practitioners. It was written in a way that “teaches modern applied cryptography to readers with a technical background but without an education in pure mathematics” (source). The book also covers all modern crypto schemes commonly used. One of the best parts about the book is each chapter has a lecture on YouTube taught by the authors. This format is useful because it reinforces the concepts or adds more details to some of the more difficult topics.

After Understanding Cryptography I’d recommend a non-textbook approach using the cryptopals crypto challenges. It is basically a set of problems that progressively get harder.  You can solve the problems using a programming language of your choice. I have yet to complete the challenges but I’d recommend attempting and solving the first two sets of problems. They introduce you to a lot of foundational concepts that can actually be applied. From what I learned in the first set, I was able to easily crack XOR encrypted executable payloads. I love cryptopals so much that I created a mirror of the site and converted it to markdown so I can easily download everything via git. 
Once a foundational knowledge of cryptography has been established it is useful to see how the algorithms look when compiled. I came across this while I was reversing a family of ransomware and couldn’t correctly decrypt the data. I was able to recover the private RSA key, decrypt the AES key encrypted with the RSA private key and decrypt files using AES in CTR but the after a certain amount of decrypted bytes the data would be corrupted. In response to this I continuously reversed the code, studied AES and all it’s different modes, compiled multiple versions of AES, opened them up in a disassembler and diffed the results but the data was still corrupted. Everything pointed to AES in CTR, eventually I identified that the CTR loop had a off-by-one error and it didn’t matter because (as a colleague pointed out) they also stored the extra byte of the key. It was only when I accounted for the off-by-one error in my decryptor that I was able to successfully decrypt files.  
After this incident whenever I come across a new encryption algorithm that I don’t understand or want to learn more about; I search for references, search for source code, add them to README.md, compile the executables and upload the .exes along with the PDB to a repository named asm-examples. I find the exploration of the disassembled code along with symbols and names from the PDB to be valuable. It aids in being able to quickly identify encryption algorithms and makes the disassembled or decompiled code less intimidating. 
To recap, my goto resources for learning encryption are Understanding Cryptography: A Textbook for Students and Practitioners, cryptopals and comparing compiled binaries to the source code. This isn’t the most in-depth approach to learning cryptography but for supporting malware analysis and reverse engineering ransomware it works well.  
tag:blogger.com,1999:blog-4093139800580227296.post-2666786553102507651
Extensions
gopep (Go Lang Portable Executable Parser)
Show full content

gopep (Go Lang Portable Executable Parser) is project I have been working on for learning about Windows Portable Executables (PE) compiled in Go. As most malware analyst have noticed, there has been an uptick in malware (particularly ransomware) compiled in Go. At first glance, reverse engineering Go PE files can be intimidating. The files are commonly over 3MB in size, contains thousands of functions and have a unique calling convention that can return multiple arguments. The first time I opened up an executable in IDA, I was lucky because the plugin IDAGolangHelper was able to identify everything. The second time, I wasn't so lucky. This motivated me to port IDAGolangHelper to IDA 7.5, Python 3, convert the GUI to PyQT and include some code that parsed the Go source code and added the Go function comments to the IDB. After everything was done, my code didn't fix up the IDB. This lead me writing gopep. In IDAGolangHelper defense, the issue was because the hard-coded bytes used to identify Go version had not been updated for a couple of years. I should have checked this first or checked one of the multiple pull requests. 

gopep is a Python script that can parse Go compiled PE file without using Go. The script only relies on Pefile. There are similar scripts that are excellent for ELF executables but during my analysis I noticed they threw exceptions when parsing PE files. Below we can see the command line options that gopep currently supports, it can also be used as a class.

C:\Users\null\Documents\repo\gopep>python gopep.py -h
usage: gopep.py [-h] [-c C_DIR] [-e E_FILE] [-x EA_DIR] [-v IN_FILE] [-m MD_FILE] [-t T_FILE] [-ev ET_FILE]

gopep Go Portable Executable Parser

optional arguments:
  -h, --help            show this help message and exit
  -c C_DIR, --cluster C_DIR
                        cluster directory of files
  -e E_FILE, --export E_FILE
                        export results of file to JSON
  -x EA_DIR, --export_all EA_DIR
                        export results of directory to JSONs
  -v IN_FILE, --version IN_FILE
                        print version
  -m MD_FILE, --module-data MD_FILE
                        print module data details
  -t T_FILE, --triage T_FILE
                        triage file, print interesting attributes
  -ev ET_FILE, --everything ET_FILE
                        print EVERYTHING!

gopep is primarily for exploring structures within PE files compiled in Go but it also supports clustering. The clustering algorithm is similar to import hashing but uses a sets of symbol names and file paths that are unique to executables compiled in Go. As with most executable clustering algorithms, it can be broken by compressing the executable. The clustering can be done by passing a command of -c and a directory of files that should be clustered. I would not recommend clustering to many files using my code. You'd be better off exporting the hashes using the command -x , parsing the JSONs and then querying that way. 

The README for the project has more details on the fields parsed, my notes and a great set of references for anyone wanting to read up on what happens when Go compiles an executable. 

https://github.com/alexander-hanel/gopep

 

tag:blogger.com,1999:blog-4093139800580227296.post-2073887634562044154
Extensions
Updates
Show full content
Hello,

Some real quick updates. I have released an new version of The Beginner's Guide to IDAPython. It has been rewritten to cover changes that IDA 7.0 introduced. I plan on adding a couple of more chapters in the upcoming months on HexRays, Structures, GUIs and (the chapter I'm most excited about) using The Unicorn Engine within IDA.

Sorry for the lack updates. Lately, I have been using a combination of Twitter and MarkDown for posting content. I can be found on Twitter @nullandnull. If you don't want to follow me, add me to a list. I swear list are the most underrated feature of Twitter. I also created an account on GitHub. I still need to transfer all my old stuff from my Bitbucket account. Two kind of recent projects I created were asmdec and capstool. The README.md within the repos contain all the needed details.

As always feel free to send me an email at alexander dot hanel at gmail dot com or ping me on Twitter. 

Cheers.

tag:blogger.com,1999:blog-4093139800580227296.post-7007880273709681168
Extensions
A Primer on Cracking XOR Encoded Executables
Show full content
A while back Locky JS downloaders were downloading executable payloads encrypted with XOR. The infection chain consisted of a victim double clicking on a JS (JavaScript), JSE (Encoded JavaScript), WSH (Windows Script Host ) or another Jscript based interpreted language, the script would then connect to a compromised website, download a binary file, decrypt the binary file using XOR and then execute the decrypted executable file.  At the time I was relying on one of two analysis approaches to retrieve the decrypted payload. The first was using automated malware analysis systems to recover the dropped payload. The second was reversing obfuscated JavaScript or other languages interpreted by wscrpt.exe to find the XOR key. Once I found the key I would decrypt the network traffic carved from a PCAP to recover the Locky executable. Both of these approaches are laborious because either I was relying on automated malware analysis system or successfully deobfuscating the script to recover the key.

Side Note:
For anyone doing deobfuscation of languages interpreted by wscript.exe, I would recommend investigating  hooking APIs. Most of the APIs that need to be hooked can be identified by using an API monitor. Also with hooking it allows you to control what the APIs return. This is useful if you want to recover all URLS that sample might want to connect to. I'll try to post some example code in the next week or two. 

Since the attackers were using XOR on an Portable Executable (PE) file I decided to crack it. This is not very difficult because XOR is not a secure cipher and when used on a portable executable file a  padding attack is introduced. Cracking XOR is a four step process. The first is recovering the key size, second is recovering the key, then decrypting the data with the found key and finally checking for the correct decrypted data.

To recover the key size Hamming distance can be used. Hamming distance can be used to calculate the number of substitutions needed to change one string into the other. From a XOR cracking standpoint, the smallest hamming distance found in a XOR file is likely the XOR key size or a multiple of it. I say a multiple of it because sometimes the smallest hamming distance could be the key size times 2 or another value. For example the below output contains a list of tuples that has the hamming distance and the key size. The actual key size was 29 but the lowest hamming distance found was 58.


[(2.6437784522003036, 58), (2.6952976867652634, 29), (3.2587556654305727, 63), (3.270363951473137, 53), (3.285315243415802, 61), (3.2863494886616276, 34), (3.29136690647482, 55), (3.300850228907783, 50), (3.306188371302278, 26), (3.309218485361723, 37)]
Length: 58, Key: IUN0mhqDx239nW3vpeL9YWBPtHC0HIUN0mhqDx239nW3vpeL9YWBPtHC0H File Name: dc53de4f4f022e687908727570345aba.bin

Here is the code for computing the hamming distance. Note, the two strings must have the same size.


def hamming_distance(bytes_a, bytes_b):
    return sum(bin(i ^ j).count("1") for i, j in zip(bytearray(bytes_a), bytearray(bytes_b)))

Identifying the key size is very important. Earlier versions of my script used standard key sizes of 16,32, 64, etc but shortly after releasing my code some Locky downloaders started using a 29 byte XOR key size. This broke my code because I was not using Hamming distance to check for the key size.

The second step is recovering the key. When a Portable executable is compiled one flag is /filealign:number. The number specifies the alignment of sections in the compiled PE file. It can be found in the Portalble Executable file format in OptionalHeader under FileAlignment. All sections within the executable will need to start at an address that is a multiple of the value defined within the FileAlignment. If the FileAlignment is 0x200, and the size of a data is 0x201 then the next section will start at offset 0x400. In between the data and the start of the section is padded with NULL bytes represented as "\x00".  The file alignment padding introduces a large amount of null bytes into the executable. When null bytes are XORed the encoded data will contain the key.  Searching for the most common recurring byte patterns in a XOR encoded executable can be used to recover the key. The following code can be used to find the 32 most common occurring bytes in an executable


substr_counter = Counter(message[i: i+size] for i in range(len(message) - size))
sub_count = substr_counter.most_common(32)


The third step is XOR the data. The following code can be used to XOR data with single or multibyte keys. If you don’t understand the code I would recommend walking through each section of it. This is personally one of my favorite pieces of Python code. It covers a number of Python concepts from list comprehension, logical operations and standard functions.


def xor_mb(message, key):

    return''.join(chr(ord(m_byte)^ord(k_byte)) for m_byte,k_byte in zip(message, cycle(key)))


The last step is to verify that the key and decrypted data is correct. Since the decrypted payload is an executable file with a known file structure I used pefile to verify the data has been decrypted correctly. If the PE structure is invalid Pefile would throw an exception.


def pe_carv(data):
    '''carve out executable using pefile's trim'''
    c = 1
    for offset in [temp.start() for temp in re.finditer('\x4d\x5a',data)]:
        # slice out executable 
        temp_buff = data[offset:]
        try:
            pe = pefile.PE(data=temp_buff)
        except:
            continue
        return pe.trim()
    return None


Complete code with example output - link


"""
    Author: 
        Alexander Hanel
    Name: 
        pe_ham_brute.py
    Purpose:
         - POC that searches for n-grams and uses them as the XOR key.
         - Also uses hamming distance to guess key size. Check out cryptopals Challenge 6
         for more details https://cryptopals.com/sets/1/challenges/6
    Example: 
    
pe_ham_brute.py ba5aa03d724d17312d9b65a420f91285caff711e2f891b3699093cc990fdaae0
Hamming distances & calculated key sizes
[(2.6437784522003036, 58), (2.6952976867652634, 29), (3.2587556654305727, 63), (3.270363951473137, 53), (3.285315243415802, 61), (3.2863494886616276, 34), (3.29136690647482, 55), (3.300850228907783, 50), (3.306188371302278, 26), (3.309218485361723, 37)]
Length: 58, Key: IUN0mhqDx239nW3vpeL9YWBPtHC0HIUN0mhqDx239nW3vpeL9YWBPtHC0H File Name: dc53de4f4f022e687908727570345aba.bin
"""

import base64
import string
import sys
import collections
import pefile
import re
import hashlib

from cStringIO import StringIO
from collections import Counter
from itertools import cycle 
from itertools import product

DEBUG = True

def xor_mb(message, key):
    return''.join(chr(ord(m_byte)^ord(k_byte)) for m_byte,k_byte in zip(message, cycle(key)))


def hamming_distance(bytes_a, bytes_b):
    return sum(bin(i ^ j).count("1") for i, j in zip(bytearray(bytes_a), bytearray(bytes_b)))


def key_len(message, key_size):
    """"returns [(dist, key_size),(dist, key_size)]"""
    avg = []
    for k in xrange(2,key_size): 
        hd = []
        for n in xrange(len(message)/k-1):
            hd.append(hamming_distance(message[k*n:k*(n+1)],message[k*(n+1):k*(n*2)])/k)
        if hd:
            avg.append((sum(hd) / float(len(hd)), k))
    return sorted(avg)[:10]


def pe_carv(data):
    '''carve out executable using pefile's trim'''
    c = 1
    for offset in [temp.start() for temp in re.finditer('\x4d\x5a',data)]:
        # slice out executable 
        temp_buff = data[offset:]
        try:
            pe = pefile.PE(data=temp_buff)
        except:
            continue
        return pe.trim()
    return None

def write_file(data, key):
    m = hashlib.md5()
    m.update(data)
    name = m.hexdigest()
    key_name = "key-" + name + ".bin"
    file_name = name + ".bin"
    print "Length: %s, Key: %s File Name: %s" % (len(key),key, file_name)
    f =  open(file_name, "wb")
    fk = open(key_name , "wb")
    f.write(data)
    fk.write(key)
    f.close()
    fk.close()

def run(message):
    key_sizes = key_len(message, 64)
    if DEBUG:
        print "Hamming distances & calculated key sizes"
        print key_sizes
    for temp_sz in key_sizes:
        size = temp_sz[1]
        substr_counter = Counter(message[i: i+size] for i in range(len(message) - size))
        sub_count = substr_counter.most_common(32)
        for temp in sub_count:
            key, count = temp
            if count == 1:
                break
            temp = xor_mb(message, key)
            pe_c = pe_carv(temp)
            if pe_c:
                write_file(pe_c, key)
                return
    
data = open(sys.argv[1],'rb').read()
run(data) 

For anyone else interested in learning about crypto I'd recommend checking out Understanding Cryptography. It is a great beginner book with not a lot of math. Each chapter has corresponding video lectures on YouTube. Another resource is attempting The Cryptopals Crypto Challenges. I can not recommend the CryptoPals challenge enough. Here are my solutions so far. At one point I contemplated quitting my job so I could just focus only on the challenges. Not one of my most practical ideas but the challenges exposed many of my weaknesses in programming and mathematics. It's pretty rare to find something that points you in the direction of what you need to learn and gives you a definitive answer (cracking the challenge) when you can move on to the next area of study.   Pretty awesome. If you have any questions or comments you can ping me on Twitter, leave a comment or send me an email at alexander dot hanel at gmail dot com.
tag:blogger.com,1999:blog-4093139800580227296.post-4742883447951345216
Extensions
ObfStrReplacer & ExtractSubfile Snippets
Show full content
ObfStrReplacer is a script that replaces obfuscated variable names with easier to read strings. Some obfuscation techniques rely on common looking strings to make the code difficult to read. For example the string Illl1III111I11 is hard to distinguish from lIll1III111I11. ObfStrReplacer takes a regular expression as an argument to match obfuscated strings, it will then add all matches to a set and replace the matches with a unique string.  11ll1III111I11 would become _carat. All renamed strings start with "_". In the image above we can see the obfuscated code on the left and the de-obfuscated code on the right.

Please see the command line example in the source code for details on usage. I have confirmed it works well on obfuscated ActionScript.  The code blindly replaces matches. It does not check for the reuse of variable names within the scope of different functions. I plan on adding this at a later date. Please leave a VT hash in the comments if you have an example.

ObfStrReplacer Source Code

ExtractSubfile is a simple modification to hachoir subfile's search.py. It is used to extract embedded files. The carving functionality was already included in hachoir-subfile but not exposed.


__@___:~/hachoir-subfile crsenvironscan.xls 
[+] Start search on 126444 bytes (123.5 KB)

[+] File at 0 size=80384 (78.5 KB): Microsoft Office document
[+] File at 2584 size=52039 (50.8 KB): Macromedia Flash data: version 9

[+] End of search -- offset=126444 (123.5 KB)
Total time: 1 sec 478 ms -- global rate: 83.5 KB/sec
__@___:~/$ python ExtractSubFile.py  crsenvironscan.xls 
[+] Start search on 126444 bytes (123.5 KB)

[+] File at 0 size=80384 (78.5 KB): Microsoft Office document => /home/file-0001.doc
[+] File at 2584 size=52039 (50.8 KB): Macromedia Flash data: version 9 => /home/file-0002.swf

[+] End of search -- offset=126444 (123.5 KB)


In the second and third lines at the end of the output we can see a document and SWF were carved.

ExtractSubFile Source Code

tag:blogger.com,1999:blog-4093139800580227296.post-5653797400910764523
Extensions
Base91 & Angler SWFs
Show full content
If anyone is curious the encoding that Angler is using in their SWFs is base91. The encoding was hinted at in an excellent article by Palo Alto Networks but was only identified as a function named DecodeToByteArray. Below are my notes to decode and decompress the embedded SWF. 

___*____$ swfextract c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511 
Objects in file c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511:
 [-i] 1 MovieClip: ID(s) 4
 [-F] 1 Font: ID(s) 1
 [-b] 1 Binary: ID(s) 5
 [-f] 1 Frame: ID(s) 0

___*____$ swfextract c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511 -b 5
___*____$ ls
c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511  output.bin  xxxswf.py
 
___*____$ hexdump -C output.bin | head

00000000  40 5a 7a 55 7b 5a 78 30  46 3b 49 26 52 48 43 5d  |@ZzU{Zx0F;I&RHC]|
00000010  40 62 66 40 40 6d 32 7b  59 25 52 5d 75 75 62 55  |@bf@@m2{Y%R]uubU|
00000020  59 4d 53 61 30 34 2a 76  7b 5e 21 74 39 5a 5b 7d  |YMSa04*v{^!t9Z[}|
00000030  62 3f 38 42 3d 5f 51 6b  24 5b 23 3a 50 2c 2c 5e  |b?8B=_Qk$[#:P,,^|
00000040  22 7b 6e 6b 23 69 21 48  2b 35 54 60 24 22 2e 36  |"{nk#i!H+5T`$".6|
00000050  58 6c 75 6d 6d 4c 54 67  48 28 5a 6a 44 4b 30 63  |XlummLTgH(ZjDK0c|
00000060  37 2a 23 3f 53 78 6c 57  4a 67 68 60 48 45 76 67  |7*#?SxlWJgh`HEvg|
00000070  35 2e 79 4a 35 3c 46 6c  5b 47 46 3f 79 42 30 47  |5.yJ5<Fl[GF?yB0G|
00000080  35 6d 3c 67 2c 54 7b 59  42 2b 6a 4f 50 2b 3b 65  |5m<g,T{YB+jOP+;e|
00000090  79 26 26 3c 30 7c 65 59  7a 59 5e 57 22 4b 72 4b  |y&&<0|eYzY^W"KrK|

While reviewing the data I noticed all of the bytes were valid ASCII. This usually infers base64 but the characters '@'' or '$' meant it must be a modified version it. A mistake I made after deobfuscating the ActionScript was I only cursory looked at the decoder. The code and data had the patterns of base64 and I blindly assumed it was. If it was a modified version of base64 I could reconstruct all the chars from the table. This can be done by reading each character from the data into a set. From there I would need to find the right sequence of chars. Strangely, this hackish approach lead me to the encoding.

In [1]: f = open("output.bin", "rb")

In [2]: d = f.read()

In [3]: o = set([])

In [4]: for x in d:
            o.add(x)

In [5]: "".join(sorted(o))
Out[5]: '!"#$%&()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~'

In [6]: len(o)
Out[6]: 91

91?  As in Base91? Weird, never heard of that. A search lead me to some code written by Adrien Beraud which confirmed the ActionScript is indeed Base91.

After the data is decoded with base91 each byte is XORed. Initially the key is set to a hard coded value then the key becomes the previous byte that was encoded. Once the XOR loop is completed it is decompressed with zlib. The initial XOR key is not static. In PAN's write up the key is 91 and in mine it was 75. The key can be found with a decompiler (Trillix or JPEXS) or a disassembler (swfdump). The later can be done to extract the XOR key from the command line. swfdump -a can be used to get the assembly of the ActionScript. Searching for bitxor and pushint should provide the XOR key.

___*____$ swfdump -a c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511 > asm.as
___*____$ swfdump vi asm.as

        00045) + 2:1 callpropvoid <q>[public]::I1lllIII111I11, 1 params
        00046) + 0:1 pushint 75   <- KEY
        00047) + 1:1 convert_u
        00048) + 1:1 setlocal r4
        00049) + 0:1 pushint 0
        00050) + 1:1 setlocal r5
        00051) + 0:1 label
        00052) + 0:1 getlocal r5
        00053) + 1:1 getlocal r3
        00054) + 2:1 getlocal_0
        00055) + 3:1 getproperty <q>[private]::1Ill1III111I11
        00056) + 3:1 getproperty <q>[public]::+ll1III111I11
        00057) + 3:1 getproperty <l,multi>{[public]""}
        00058) + 2:1 lessthan
        00059) + 1:1 iffalse ->81
        00060) + 0:1 getlocal r3
        00061) + 1:1 getlocal r5
        00062) + 2:1 getproperty <l,multi>{[public]""}
        00063) + 1:1 getlocal r4
        00064) + 2:1 bitxor       <- XOR
        00065) + 1:1 convert_u


Quickly written Python code for decoding and extracting the second SWF. The key will likely need to be modified.

# The Base91 code is written by Adrien Beraud
# https://github.com/aberaud/base91-python/blob/master/base91.py

# Base91 encode/decode
#
# Copyright (c) 2012 Adrien Beraud
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
#   * Redistributions of source code must retain the above copyright notice,
#     this list of conditions and the following disclaimer.
#   * Redistributions in binary form must reproduce the above copyright notice,
#     this list of conditions and the following disclaimer in the documentation
#     and/or other materials provided with the distribution.
#   * Neither the name of Adrien Beraud, Wisdom Vibes Pte. Ltd., nor the names
#     of its contributors may be used to endorse or promote products derived
#     from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#  

import struct
import sys
import zlib

base91_alphabet = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',
 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
 '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '!', '#', '$',
 '%', '&', '(', ')', '*', '+', ',', '.', '/', ':', ';', '<', '=',
 '>', '?', '@', '[', ']', '^', '_', '`', '{', '|', '}', '~', '"']

decode_table = dict((v,k) for k,v in enumerate(base91_alphabet))

def decode(encoded_str):
    ''' Decode Base91 string to a bytearray '''
    v = -1
    b = 0
    n = 0
    out = bytearray()
    for strletter in encoded_str:
        if not strletter in decode_table:
            continue
        c = decode_table[strletter]
        if(v < 0):
            v = c
        else:
            v += c*91
            b |= v << n
            n += 13 if (v & 8191)>88 else 14
            while True:
                out += struct.pack('B', b&255)
                b >>= 8
                n -= 8
                if not n>7:
                    break
            v = -1
    if v+1:
        out += struct.pack('B', (b | v << n) & 255 )
    return out

def main():
    f = open(sys.argv[1], 'rb')
    x = f.read()
    d = decode(x)
    dd = ""
    key = 75
    for y in d:
        dd += chr(y ^ key)
        key = y
    o = zlib.decompress(dd)
    kk = open( sys.argv[1] + "-out.bin", "wb")
    kk.write(o)
    kk.close()

main()

output.bin is the binary data extracted using swfextract. The above Python code is stored in angler-decoder.py. After running the script the decoded SWF is saved to output.bin-out.bin. Then I use xxxswf.py to verify the SWF is present.


___*____$ python angler-decoder.py output.bin 
___*____$ ls
angler-decoder.py  c34266299460225c0354df5438417924579641095ffd7588a42d8fae07ae8511  output.bin  output.bin-out.bin  xxxswf.py
___*____$ python xxxswf.py output.bin-out.bin 

[SUMMARY] Potentially 1 SWF(s) in MD5 d41d8cd98f00b204e9800998ecf8427e:output.bin-out.bin
 [ADDR] SWF 1 at 0x0 - CWS Header
___*____$ python xxxswf.py -d output.bin-out.bin 

[SUMMARY] Potentially 1 SWF(s) in MD5 d41d8cd98f00b204e9800998ecf8427e:output.bin-out.bin
 [ADDR] SWF 1 at 0x0 - CWS Header
  [FILE] Carved SWF MD5: 5d4c794c3a3011da71cc31d5fd7015ce.swf

The extracted second SWF is also obfuscated.  

My "cleaned up" ActionScript

package 
{
    import flash.display.*;
    import flash.system.*;

    public class ExtendedMovieClipFunction extends MovieClip
    {
        private var DEUNCOMPRESSED_BUFFER:Object;
        private var _CLASS_BUFFER:Class;
        private var FuncNameToStrInstance:AssignFuncNameToString;
        private var int_0:uint = 0;
        private var _uint_0:uint = 0;
        private var _uint_255:uint = 255;
        private var _object2:Object;
        private var _object3:Object;

        public function ExtendedMovieClipFunction(param1:Object = null)
        {
            this.FuncNameToStrInstance = new AssignFuncNameToString();
            #  a SWF file from other domains than that of the Loader object can call Security.allowDomain() to
            #  permit a specific domain
            Security[this.FuncNameToStrInstance.allowDomain]("*");
            var _loc_3:* = ApplicationDomain[this.FuncNameToStrInstance.currentDomain];
            var  ldr:Loader :* = _loc_3[this.FuncNameToStrInstance.getDefinition](this.FuncNameToStrInstance.flash.display.Loader) as Class;
            this.DEUNCOMPRESSED_BUFFER = new  ldr:Loader ;
            this._CLASS_BUFFER = _loc_3[this.FuncNameToStrInstance.getDefinition](this.FuncNameToStrInstance.flash.utils.ByteArray) as Class;
            
            ## The Stage class represents the main drawing area.
            if (this[this.FuncNameToStrInstance.stage])
            {
                this.FuncEventListener();
            }
            else
            {
                this[this.FuncNameToStrInstance.addEventListener](this.FuncNameToStrInstance.addedToStage, this.FuncEventListener);
            }
            return;
        }// end function

        public function 1_object1(param1:Object, param2:int) : void
        {
            param2++;
            return;
        }// end function

        private function FuncEventListener(param1:Object = null) : void
        {
            this[this.FuncNameToStrInstance.removeEventListener](this.FuncNameToStrInstance.addedToStage, this.FuncEventListener);
            this[this.FuncNameToStrInstance.addEventListener](this.FuncNameToStrInstance.enterFrame, this.I1111IIIlllIl1);
            var _loc_2:* = new ExtendedByteArrayFunction();
            var DECODE_BUFFER:* = new this._CLASS_BUFFER();
            this.CONSTRUCT_KEY();
            this.BASE91(_loc_2, _loc_2[this.FuncNameToStrInstance.length], DECODE_BUFFER);
            this.func_123(DECODE_BUFFER);
            var _loc_4:* = 75;
            var INDEX:* = 0;
            
            // XOR loop 
            if (INDEX < DECODE_BUFFER[this.FuncNameToStrInstance.length])
            {
                var _loc_6:* = DECODE_BUFFER[INDEX] ^ _loc_4;
                _loc_4 = DECODE_BUFFER[INDEX];
                DECODE_BUFFER[INDEX] = _loc_6;
                INDEX++;
                ;
            }
            # XORs the data then uncompresses 
            DECODE_BUFFER[this.FuncNameToStrInstance.uncompress]();
            this.DEUNCOMPRESSED_BUFFER[this.FuncNameToStrInstance.loadBytes](DECODE_BUFFER);
            this[this.FuncNameToStrInstance.addChild](this.DEUNCOMPRESSED_BUFFER);
            ;
            var _loc_8:* = null;
            return;
            ;
            return;
        }// end function

        private function I1111IIIlllIl1(param1) : void
        {
            if (this.currentFrame == 200)
            {
                this.I1ll1III111I11(new Number(2));
                return;
            }
            return;
        }// end function

        # Create Key  
        private function CONSTRUCT_KEY() : void
        {
            this._object2 = new this._CLASS_BUFFER();
            this._object3 = new this._CLASS_BUFFER();
            var _loc_2:* = 0;
            _loc_2 = 65;
            
            if (_loc_2 < 91)
            {
                this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                _loc_2++;
                ;
            }
            _loc_2 = 97;
            
            if (_loc_2 < 123)
            {
                this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                _loc_2++;
                ;
            }
            _loc_2 = 48;
            
            if (_loc_2 < 58)
            {
                this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                _loc_2++;
                ;
            }
            _loc_2 = 33;
            
            if (_loc_2 < 48)
            {
                
                
                if (_loc_2 == 34 || _loc_2 == 39 || _loc_2 == 45)
                {
                }
                else
                {
                    this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                }
                _loc_2++;
                ;
            }
            _loc_2 = 58;
            
            if (_loc_2 < 65)
            {
                this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                _loc_2++;
                ;
            }
            _loc_2 = 91;
            
            if (_loc_2 < 97)
            {
                if (_loc_2 == 92)
                {
                }
                else
                {
                    this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                }
                _loc_2++;
                ;
            }
            _loc_2 = 123;
            
            if (_loc_2 < 127)
            {
                this._object3[this.FuncNameToStrInstance.writeByte](_loc_2);
                _loc_2++;
                ;
            }
            this._object3[this.FuncNameToStrInstance.writeByte](34);
            
            var _loc_3:* = 0;
            _loc_3 = 0;
            
            if (_loc_3 < 255)
            {
                this._object2[_loc_3] = 255;
                _loc_3++;
                ;
            }
            _loc_3 = 0;
            
            if (_loc_3 < this._object3[this.FuncNameToStrInstance.length])
            {
                this._object2[this._object3[_loc_3]] = _loc_3;
                _loc_3++;
                ;
            }
            return;
        }// end function

        public function func_123(param1) : uint
        {
            var _loc_2:* = 0;
            if (this._uint_255 != 255)
            {
                param1[param1[this.FuncNameToStrInstance.length]] = this.int_0 | this._uint_255 << this._uint_0;
                _loc_2 = _loc_2 + 1;
            }
            return _loc_2;
            return;
        }// end function

        public function BASE91(param1, _length:uint, param3) : uint
        {
            var _loc_4:* = 0;
            var _loc_5:* = 0;
            var _int_8191:* = 8191;
            _INDEX = 0;
            
            # previously IF
            while (_INDEX < _length)
            {
                if (this._object2[param1[_INDEX]] == 255)
                {
                }
                else
                {
                    if (this._uint_255 == 255)
                    {
                        this._uint_255 = this._object2[param1[_INDEX]];
                    }
                    else
                    {
                        #   _uint_255 =  _uint_255 + * len(_object3)
                        this._uint_255 = this._uint_255 + this._object2[param1[_INDEX]] * this._object3[this.FuncNameToStrInstance.length];
                        this.int_0 = this.int_0 | this._uint_255 << this._uint_0;
                        
                        this._uint_0 = this._uint_0 + ((this._uint_255 & _int_8191) > 88 ? (13) : (// label, 14));
                        
                        # increament _loc_8
                        var _loc_8:* = _loc_5;
                        _loc_5 = _loc_5 + 1;
                        
                        # move to out buffer
                        param3[_loc_8] = this.int_0 & 255;
                        
                        this.int_0 = this.int_0 >> 8;
                        this._uint_0 = this._uint_0 - 8;
                        if (this._uint_0 > 7) goto 160;
                        this._uint_255 = 255;
                    }
                }
                _INDEX++;
                ;
            }
            return _loc_5;
            return;
        }// end function

    }
}


tag:blogger.com,1999:blog-4093139800580227296.post-3768212603328861723
Extensions
Exploring the Top 100 ebooks of The Pirate Bay
Show full content

I wrapped up an analysis of the Top 100 ebooks of the Pirate Bay.  Rather than posting to code I decided to use a notebook viewer. All the data and code can be found on my bit-bucket repo. Cheers.
tag:blogger.com,1999:blog-4093139800580227296.post-3196299622921546680
Extensions
The Beginner's Guide to IDAPython
Show full content
In my spare time for the past couple of months I have been working on an ebook called "The Beginner's Guide to IDAPython". I originally wrote it as a reference for myself - I wanted a place to go to where I could find examples of functions that I commonly use (and forget) in IDAPython.  Since I started the book I have used it many times as a quick reference to understand syntax or see an example of some code. I hope others will find it equally useful.  The book is not a static document. I already have a list of content/topics that I would like to write about., like a cover.. Please feel free to email me if you would like a topic added, have a correction or would like to say hi. My email is a the bottom of the introduction . The ebook can be found  in the below link.


https://leanpub.com/IDAPython-Book

The price is free (move the slider to left) but has a suggested price of $14.99. In all honesty I don't care if you purchase it. A purchase would be nice but I'd rather you learn something from it.

Logistics
I wrote the book in markup language. I used StackEdit [1] as an editor. I paid for a sponsor account. This allowed me to download it in a PDF. I did version control and hosting via bit-bucket [2]. Not sure why Dan [3] and I are the only people on it.  Bit-Bucket is awesome. Unlimited free private repos for the win.  I'm using leanpub [4] as the distributor for my ebook. Ange Albertini is also using leanpub to publish an ebook called  Binary is beautiful [5]. Disclaimer his book will be way better than mine.

I'd like to thank Hexacorn for all his feedback and support.

1. https://stackedit.io/
2. https://bitbucket.org/
3. https://bitbucket.org/daniel_plohmann
4. http://leanpub.com/
5. https://leanpub.com/binaryisbeautiful

Updates:  Usual grammar issues...


tag:blogger.com,1999:blog-4093139800580227296.post-4122995335963622383
Extensions
Dyre IE Hooks
Show full content
I recently wrapped up my analysis of Dyre. A PDF document can be found in my papers repo. Most of the document focuses on the different stages that  Dyre interacts with the operating system. There are still some areas that I'd like to dig deeper into. For now it should be a good resource for anyone trying to identify a machine infected with Dyre or wanting to know more about the family of malware.

During the reversing process I found one part of Dyre functionality worthy of a post. As with most banking trojans Dyre contains functionality to hook APIs to log browser traffic. Typically to get the addresses of the APIs the sample will call GetProcAddress or manually traverse the portable executable file format to resolve symbols. If you are unfamiliar with the later technique I'd highly recommend reading section 3.3 of "Understanding Windows Shellcode" by Skape [1]. Dyre attempts to hook APIs in firefox.exe, chrome.exe and iexplorer.exe. It uses the standard GetProcAddress approach for resolving symbols in firefox.exe, is unsuccessful in chrome.exe and uses the GetProcAddress approach for the APIs LoadLibraryExW and CreateProcessInternalW in iexplorer.exe. Dyre hooks two APIs in WinInet.dll but it does it in a unique way. Dyre will read the image header timedatestamp [2] from WinInet. This value contains the time and date from when Wininet was created by the linker during compiling.  It will then compare the timedatestamp to a list of timedatestamps stored by Dyre.  The list contains presumably every time stamp for WinInet.dll since '2004-08-04 01:53:22' to '2014-07-25 04:04:59'.  Below is an example of the values that can be found in the list.

seg000:00A0C05F           db    0
seg000:00A0C060 TimeStampList dd 4110941Bh              ; DATA XREF: TimeStamp:_loopr
seg000:00A0C064 dword_A0C064 dd 0                       ; DATA XREF: TimeStamp+1Cr
seg000:00A0C064                                         ; TimeStamp:loc_A07A0Dr ...
seg000:00A0C068           dd 411095F2h <- Time stamp
seg000:00A0C06C           dd 0         <- WinInet index
seg000:00A0C070           dd 4110963Fh
seg000:00A0C074           dd 0
seg000:00A0C078           dd 4110967Dh
seg000:00A0C07C           dd 0
seg000:00A0C080           dd 411096D4h
seg000:00A0C084           dd 0
seg000:00A0C088           dd 411096DDh
seg000:00A0C08C           dd 0
seg000:00A0C090           dd 41252C1Bh
seg000:00A0C094           dd 0
.....
seg000:00A0C0AC           dd 1
seg000:00A0C0B0           dd 435862A0h
seg000:00A0C0B4           dd 2
seg000:00A0C0B8           dd 43C2A6A9h
seg000:00A0C0BC           dd 3
....
seg000:00A0D230           dd 4CE7BA3Fh
seg000:00A0D234           dd 78h
seg000:00A0D238           dd 53860FB3h
seg000:00A0D23C           dd 79h
seg000:00A0D240           dd 53D22BCBh
seg000:00A0D244           dd 7Ah

Values converted to time

>>> datetime.datetime.fromtimestamp(0x411095F2).strftime('%Y-%m-%d %H:%M:%S')
'2004-08-04 01:53:22'

>>> datetime.datetime.fromtimestamp(0x53D22BCB).strftime('%Y-%m-%d %H:%M:%S')
'2014-07-25 04:04:59'    

If the timedatestamp is not present or an error occurs Dyre will send the hash of WinInet to the attackers server. If the hash is not found it will send WinInet back to the attackers. Below are some of the strings responsible for displaying errors for the command and control.

'/%s/%s/63/file/%s/%s/%s/'
"Check wininet.dll on server failed"
"Send wininet.dll failed"

If the timedatestamp is found in the list the next value is used as an index into another list. For example if the timedatestamp was 4802A13Ah it would be found at the 49th entry and the next value would be 0x15 or 21.

Data
seg000:00A0C1E8           dd 4802A13Ah  <- '2008-04-13 18:11:38'
seg000:00A0C1EC           dd 15h  <- 21 index

Assembly to read index value

seg000:00A07A0D           movsx   edx, word ptr ds:TimeStampIndex[eax*8] ; edx = 21
seg000:00A07A15           lea     edx, [edx+edx*2] ; edx  = 63
seg000:00A07A18           mov     edx, ds:offset[edx*4]
seg000:00A07A1F           mov     [ecx], edx            ; save off value

Python: calculate offset
Python>hex(0x0A0D3E0 + (21+21* 2) * 4)
0xa0d4dc

Read
seg000:00A0D4DC           dw 0F3Ch  0x0f3C offset to inline hook in wininet

The value 0xF3C + the base address of WinInet is the function prologue for ICSecureSocket::Send_Fsm. Dyre uses this to know the address to place it's hooks.

ICSecureSocket::Send_Fsm(CFsm_SecureSend *)
    
77200F37    90              NOP
77200F38    90              NOP
77200F39    90              NOP
77200F3A    90              NOP
77200F3B    90              NOP
77200F3C  - E9 C7F0398A     JMP 015A0008   <- Inline hook
015A0008    68 4077A000     PUSH 0A07740
015A000D    C3              RETN

00A07740    55              PUSH EBP
00A07741    8BEC            MOV EBP,ESP
00A07743    83EC 08         SUB ESP,8
00A07746    894D FC         MOV DWORD PTR SS:[EBP-4],ECX
00A07749    68 2077A000     PUSH 0A07720
00A0774E    FF75 08         PUSH DWORD PTR SS:[EBP+8]
00A07751    FF75 FC         PUSH DWORD PTR SS:[EBP-4]
00A07754    FF15 94DEA000   CALL DWORD PTR DS:[A0DE94]
00A0775A    8945 F8         MOV DWORD PTR SS:[EBP-8],EAX

It will also hooks ICSecureSocket::Receive_Fsm in the same fashion.

Closing 
Rather than calling GetProcAddress (the hooked APIs are not exportable) Dyre stores the timedatestamp and patch offset of every known version of WinInet to avoid triggering heuristic based scanners. Seems like an arduous approach but still kind of cool. Another interesting fact is Dyre has the ability to patch Trusteer's RapportGP.dll if found in the browser memory. Dyre is actually a family of malware worthy of a deep dive. At first glance I ignored it because everything looked pretty cut & paste. I'd recommend others to check it out. If you find anything useful please shoot me an email. Cheers.

Hash Analyzed 099c36d73cad5f13ec1a89d5958486060977930b8e4d541e4a2f7d92e104cd21
  1. http://www.nologin.org/Downloads/Papers/win32-shellcode.pdf
  2. http://msdn.microsoft.com/en-us/library/ms680313.aspx
tag:blogger.com,1999:blog-4093139800580227296.post-8328825138554297060
Extensions
reg+displ
Show full content
I have been reversing Dyre in my spare time. I'm hoping to have a full analysis out in the next week or two. Something kind of annoying about Dyre is it uses what looks like a massive structure to store it's data and function pointers. For example in the image below we can see it it passing a handle stored at [eax+0x130] to WaitForSingleObject.
Manually tracing the code or searching for all cross references is kind of painful to find what populated the value. Since the displacement is kind of unique due to it's value of 0x130 or 304 it can be targeted very easily in IDAPython.

import idautils 
import idaapi
displace = {}

# for each known function 
for func in idautils.Functions():
    flags = idc.GetFunctionFlags(func)
    # skip library & thunk functions 
    if flags & FUNC_LIB or flags & FUNC_THUNK:
        continue  
    dism_addr = list(idautils.FuncItems(func))
    for curr_addr in dism_addr:
        op = None
        index = None 
        # same as idc.GetOptype, just a different way of accessing the types
        idaapi.decode_insn(curr_addr)
        if idaapi.cmd.Op1.type == idaapi.o_displ:
            op = 1
        if idaapi.cmd.Op2.type == idaapi.o_displ:
            op = 2
        if op == None:
            continue 
        if "bp" in idaapi.tag_remove(idaapi.ua_outop2(curr_addr, 0)) or \
               "bp" in idaapi.tag_remove(idaapi.ua_outop2(curr_addr, 1)):
            # ebp will return a negative number
            if op == 1:
                index = (~(int(idaapi.cmd.Op1.addr) - 1) & 0xFFFFFFFF)
            else:
                index = (~(int(idaapi.cmd.Op2.addr) - 1) & 0xFFFFFFFF)
        else:
            if op == 1:
                index = int(idaapi.cmd.Op1.addr)
            else:
                index = int(idaapi.cmd.Op2.addr)
        # create key for each unique displacement value 
        if index:
            if displace.has_key(index) == False:
                displace[index] = []
            displace[index].append(curr_addr)
The above code will create a dictionary of all the displacement values in known functions. A simple for loop can be used to find the address and disassembly of all uses for the defined displacement value.
Python>for x in displace[0x130]: print hex(x), GetDisasm(x)
0x10004f12 mov     [esi+130h], eax
0x10004f68 mov     [esi+130h], eax
0x10004fda push    dword ptr [esi+130h]  ; hObject
0x10005260 push    dword ptr [esi+130h]  ; hObject
0x10005293 push    dword ptr [eax+130h]  ; hHandle
0x100056be push    dword ptr [esi+130h]  ; hEvent
0x10005ac7 push    dword ptr [esi+130h]  ; hEvent
Python>
With the addresses it makes it easy to find where the value is populated.


The dictionary created by the script is named displace. It will contain all displaced values.  Not super 1337 but still useful. Cheers.
tag:blogger.com,1999:blog-4093139800580227296.post-5965031587380336215
Extensions
Backtrace POC - Stack Strings
Show full content
Example 1 Hex View There are a number of tools that cover char strings in IDA. If you are not familiar with char strings it's a low hanging obfuscation technique to thwart analyst from viewing the strings inside of an executable. Some notable tools and posts on this topic are [1] & [2]. In the image above you can see the string DBG. Odds are if we were viewing the executable in a hex editor or using strings this wouldn't stick out.

Example 1 Assembly View If we were watching the stack of the executable at run time we would see something constructed similar to the string/comment above.
Example 2  The code can be run in two modes the first is by selecting the code and the double clicking the script in IDA (ALT+F9). In the example above we can see the string "W32Time". My code attempts to reconstruct the stack memory. The buffer can be accessed via a list object.str_buff. In the Output window above you can see the content of the buffer dumped to standard out. This makes it easy to format the data and access it via an index. The commented data is an example of how the string would look on the stack in Ollydbg. The second way to execute the code is to pass an address within a function to object.run( address ). This will try to rebuild the stack for the whole function. All of this is done statically. Char strings that are populated via registers (such as mov [ebp+var_c], bl when bl is 0x4f in the example 1 image) are traced back using backtrace.py. For more details on backtrace please see the the following link.

As previously mentioned this topic has already been covered. I'm posting this code because it's a good example of using backtrace.py. I had fun working on this one. The code handles all examples I have found so far. There is an issue with formatting constructed wide char strings. Not exactly sure of the best approach. I tried to keep the data flexible so it should be easy to write a function to format the data.

[1]. Automatic Recovery of Constructed Strings in Malware by Jay Smith of FireEye - link
[2]. Finding Byte Strings using IDAPython by Jason Jones of Arbor Networks - link 

Repo - Link

Code for reviewing

"""
Author:
    Alexander Hanel 
Date:
    20140902
Version:
    1  - should be good to go.
Summary:
    Examples of using the backtrace library to rebuild strings

TODO:
    * How to deal with printing wide char strings?
    * What is the size of the frame buffer if GetFrameSize returns something
      smaller than the frame/stack index or the IDA does not recognize the function?

Notes:
    idaapi.o_phrase # Memory Ref [Base Reg + Index Reg]
    o_phrase   =  idaapi.o_phrase    #  Memory Ref [Base Reg + Index Reg]    phrase
    o_displ    =  idaapi.o_displ     #  Memory Reg [Base Reg + Index Reg + Displacement] phrase+addr

Useful Reads
    http://smokedchicken.org/2012/05/ida-rename-local-from-a-script.html
    http://zairon.wordpress.com/2008/02/15/idc-script-and-stack-frame-variables-length/
"""
import sys, os, logging, copy
from binascii import unhexlify
# Add the parent directory to Python Path
sys.path.append(os.path.realpath(__file__ + "/../../"))
# import the backtrace module
from backtrace import *

class Frame2Buff:
    def __init__(self):
        self.verbose = False
        self.func_start = idc.SelStart()
        # SelEnd() returns the following selected instruction
        self.func_end = SelEnd()
        self.esp = False
        self.ebp = False
        self.comment = True
        self.frame_size = None
        self.bt = None
        self.str_buff = None
        self.comment = True
        self.formatted_buff = ""
        self.format = True

    def run(self, func_addr=None):
        """ run and create Frame2Buff"""
        # check if code is selected or if using the whole function
        if self.func_start == BADADDR or self.func_end == BADADDR:
            if func_addr == None:
                if self.verbose:
                    print "ERROR: No addresses selected or passed"
                return None
        if func_addr:
            self.func_start = idc.GetFunctionAttr(func_addr, FUNCATTR_START)
            self.func_end = idc.GetFunctionAttr(func_addr, FUNCATTR_END)
        if self.func_start == BADADDR:
            if self.verbose:
                print "ERROR: Invalid address"
        self.frame_size = GetFrameSize(self.func_start)
        try:
            self.bt = Backtrace()
            self.bt.verbose = False
        except ImportError:
            print "ERROR: Could not import Backtrace - aborting"
        self.func_end = PrevHead(self.func_end)
        self.populate_buffer()
        if self.format:
            self.format_buff()
        if self.comment:
            self.comment_func()

    def populate_buffer(self):
        curr_addr = self.func_start
        self.str_buff = list('\x00' * self.frame_size)
        while curr_addr <= self.func_end:
            index = None
            idaapi.decode_insn(curr_addr)
            # check if instr is MOV, [esp|ebp + index], variable
            if idaapi.cmd.itype == idaapi.NN_mov and idaapi.cmd.Op1.type == idaapi.o_displ:
                if "bp" in idc.GetOpnd(curr_addr, 0):
                    # ebp will return a negative number
                    index = (~(int(idaapi.cmd.Op1.addr) - 1) & 0xFFFFFFFF)
                    self.ebp = True
                else:
                    index = int(idaapi.cmd.Op1.addr)
                    self.esp = True
                if idaapi.cmd.Op2.type == idaapi.o_reg:
                    # value needs to be traced back
                    self.bt.backtrace(curr_addr, 1)
                    # tainted means the reg was xor reg, reg
                    # odds are being used to init var.
                    if self.bt.tainted != True:
                        last_ref = self.bt.refsLog[-1]
                        idaapi.decode_insn(int(last_ref[0]))
                        data = idaapi.cmd.Op2.value
                    else:
                        # tracked variable has been set to zero by xor reg, reg
                        curr_addr = idc.NextHead(curr_addr)
                        continue
                elif idaapi.cmd.Op2.type != idaapi.o_imm:
                    curr_addr = idc.NextHead(curr_addr)
                    continue
                else:
                    data = idaapi.cmd.Op2.value
                if data:
                    try:
                        hex_values = hex(data)[2:]
                        if hex_values[-1] == "L":
                            hex_values = hex_values[:-1]
                        if len(hex_values) % 2:
                            hex_values = "0" + hex_values
                        temp = unhexlify(hex_values)
                    except:
                        if self.verbose:
                            print "ERROR: Unhexlify Issue at %x %s (not added)" % (curr_addr, idc.GetDisasm(curr_addr))
                        curr_addr = idc.NextHead(curr_addr)
                        continue
                else:
                    curr_addr = idc.NextHead(curr_addr)
                    continue
                # GetFrameSize is not a reliable buffer size
                # If so append to buffer if index is less than
                # 2 * frame size. If more likely an error
                if self.ebp or self.esp:
                    cal_index = index + len(temp)
                    if cal_index > self.frame_size:
                        if cal_index < (self.frame_size * 2):
                            for a in range(cal_index - self.frame_size):
                                self.str_buff.append("\x00")
                                if self.verbose:
                                    print "ERROR: Frame size incorrect, appending"
                if self.ebp:
                    # reverse the buffer
                    temp = temp[::-1]
                    for c, ch in enumerate(temp):
                        try:
                            self.str_buff[index - c] = ch
                        except:
                            if self.verbose:
                                print "ERROR: Frame EBP index invalid: at %x" % (curr_addr)
                if self.esp:
                    for c, ch in enumerate(temp):
                        try:
                            self.str_buff[index + c] = ch
                        except:
                                print "ERROR: Frame ESP index invalid: at %x" % (curr_addr)
            curr_addr = idc.NextHead(curr_addr)
        # reverse the buffer to match index
        if self.ebp == True:
            self.str_buff = self.str_buff[::-1]
            self.str_buff.pop()



    def format_buff(self):
        self.formatted_buff = ""
        temp_buff = copy.copy(self.str_buff)

        if self.ebp == True:
            temp_buff = temp_buff[::-1]
            temp_buff.pop()

        if self.str_buff:
            for index, ch in enumerate(temp_buff):
                try:
                    if ch == "\x00" and temp_buff[index + 1] != "\x00":
                        self.formatted_buff += " "
                except:
                    pass
                if ch != "\x00":
                    self.formatted_buff += ch

    def comment_func(self):
        idc.MakeComm(self.func_end, self.formatted_buff)

"""
Example:
    Create a buffer of the whole function

x = Frame2Buff()
x.run(here())  # func adddr

"""
x = Frame2Buff()
x.run() # select data
tag:blogger.com,1999:blog-4093139800580227296.post-3393136258891340531
Extensions
Renaming Simple Functions
Show full content
Simple Function
The above function is very simple. Let's ignore the actual code but think about the codes functionality from a generic standpoint. The code pushes arguments on to the stack, calls APIs, compares return values from the APIs and then returns one or zero. In most instance these simple functions do not need to be analyzed. By reading the API names most of the functionality can be inferred and easily renamed to something like "RegCreateAndSetValue".  After seeing these simple functions many times I realized that many of these functions could automatically be renamed. If broken down into steps it would look like this.
  1. API names from a function are extracted
  2. Sub-strings from the APIs are extracted
  3. Search for a common sub-string throughout all API names. 
  4. If a sub-string is common throughout all, create a name from the sub-strings. 
Step 1

    def get_apis(self, func_addr):
        flags = GetFunctionFlags(func_addr)
        # ignore library functions
        if flags & FUNC_LIB or flags & FUNC_THUNK:
            logging.debug("get_apis: Library code or thunk")
            return None
        # list of addresses
        dism_addr = list(FuncItems(func_addr))
        for instr in dism_addr:
            tmp_api_address = ""
            if idaapi.is_call_insn(instr):
                # In theory an API address should only have one xrefs
                # The xrefs approach was used because I could not find how to
                # get the API name by address.
                for xref in XrefsFrom(instr, idaapi.XREF_FAR):
                    if xref.to == None:
                        self.calls += 1
                        continue
                    tmp_api_address = xref.to
                    logging.debug("get_apis: xref to %x found", tmp_api_address)
                    break
                # get next instr since api address could not be found
                if tmp_api_address == "":
                    self.calls += 1
                    continue
                api_flags = GetFunctionFlags(tmp_api_address)
                # check for lib code (api)
                if api_flags & idaapi.FUNC_LIB == True or api_flags & idaapi.FUNC_THUNK:
                    tmp_api_name = NameEx(0, tmp_api_address)
                    if tmp_api_name:
                        self.apis.append(tmp_api_name)
                else:
                    self.calls += 1

Step 2 & 3

    def match_apis(self):
        self.matched = False
        api_set = set(self.apis)
        # Optional Threshold. Only check functions with more than 2 apis
        if self.calls <= self.threshold and len(self.apis) > 1:
            api_tokend  = []
            # for each api in function
            for api_name in api_set:
                # for each tokenized string in API name
                for item in self.tok.tokenizer(api_name):
                    if item is None or item is "A" or item is "W":
                        continue
                    api_tokend.append(item)
            # Count occurrence of strings.
            count_tmp = Counter(api_tokend)
            # if a common string is found in all APIs
            # return True and the count strings
            for string, count in count_tmp.items():
                if count == len(set(self.apis)):
                    self.matched = True
                    self.count_strings = count_tmp
                else:
                    logging.debug("match_apis: API count and API sub-string don't match")
        else:
            logging.debug("match_apis: calls above threshold or API count is 1")

A lot of the heavy lifting for parsing out the sub-strings is built into the tokenizer module in TT&SS. For more information and usage I'd recommend the following post

Step 4


    def create_string(self):
        if self.count_strings == "" or self.matched is False:
            return
        # Sort strings by highest occurrence
        sort = sorted(self.count_strings, key=self.count_strings.get, reverse=True)
        name = ""
        # if a function contains all the same API multiple times
        # might be possible to modify to deal with wrapper code also
        if self.calls == 0 and len(set(self.apis)) == 1 and len(self.apis) > 1:
            self.func_name = self.apis[0] + str(len(self.apis))
            return
        for each in sort:
            # ignore Wide or Ascii
            if each.upper() == "A" or each.upper() == "W":
                continue
            # Convert to CamelCase for easier reading and space
            tmp = each[0].upper() + each[1:]
            name += str(tmp)
        # replace white space with underscore
        name = name.replace(" ", "_")
        logging.debug("create_string: string created %s", name)
        self.func_name = name

If we were to apply that logic plus some other random stuff we would have the following..

I think this is pretty cool. I like the idea of combining other domains of knowledge such as Natural Language Text Processing to reversing. Sadly functions  simple functions or APIs that all contain a similar sub-string are rare. The rarity happens because a lot APIs that share similar functionality use generic APIs such as "CloseHandle" to close out a process. This API does not contain any of the sub-strings so it will fail the similarity test. I'm currently toying with an idea of using thresholds on matches or whitelisting certain APIs. Creating API sets as was used in the generic renaming of functions in IDAScope is another option. The main issue with that approach is categorizing of APIs by functionality. There are lot of little things for this project hence why I'm releasing it as a POC.  Below is the output of attempting to rename 456 functions in a Zeus IDB.


The VirtualProtect2 contains a "2" because the API was called twice from a function. The API names that end with an underscore and a value are for calculated names that happen multiple times.

The source POC is named w_sims.py and can be found in the POCS dir in the repo. The source also contains some code to identify wrapper functions. The code is currently setup to run the SimilarFunctions and the wrapper class on all the known functions. If you would like to run the wrapper class or experiment on other function tweak the execute options at the bottom of the code. The code is still being tweaked and fixed. I have been using this code off and on for a couple of weeks. I have seen some issues while importing the modules but  I think I got those ironed out. If anything breaks, you have feedback etc please leave a comment. 
tag:blogger.com,1999:blog-4093139800580227296.post-5317353917483985929
Extensions
Random Applocker Thoughts
Show full content
While reading through the Windows Internals book I came across an interesting feature called AppLocker.
AppLocker provides a robust experience for IT administrators through new rule creation tools and wizards. For example, IT administrators can automatically generate rules by using a test reference computer and then importing the rules into a production environment for widespread deployment. The IT administrator can also export a policy to provide a backup of the production configuration or to provide documentation for compliance purposes. Your Group Policy infrastructure can be used to build and deploy AppLocker rules as well, saving your organization training and support costs.
Source
The paragraph above is a good description of it's functionality. The tool focuses on policy based security. For example prevent kazaa.exe from running in our enterprise environment. Applocker has three ways of blocking files from executing. The three are publisher, path/file and hash. The publisher is by far the most interesting because it is a little more generic but still targets a decent characteristic. For example if Company A is targeted by Malware B that is always signed with a stolen or legitimate certificate XXX. Company A could create a policy or rule to target the publisher, product name, file name or file version of that signed file. If that rule is ever triggered the file could be blocker or logged as an event and then feed into a SIEM.  The second useful example is in the event of a mass spam campaign. If an organization received 15k emails with a zip attachment. It's almost guaranteed 1% of that population will execute the file within the zip. This espically true if inbox cleanup could take an hour or two.  Most email spam campaign attachments contain a static hash. If an organization had a user report the spam campaign an analyst could create a Applocker rule based on the file hash and push it out as a new policy rule. Odds are the turn around time on pushing out an Applocker policy rule would be faster than getting an AV signature update.

Dear Microsoft Employees,
In future releases of Applocker could you please have an option to use the parent process as a filter for rules?  For example in Haifei Li slide 21 in Exploring in the Wild: A Big Data Approach to Application Security Research (and Exploit Detection) [link], he mentions that out of half million Microsoft Documents the only time they saw MSCOMCTL.OCX loaded was during exploitation. If a rule could be created by an Applocker user to alert when MSCOMCTL.OCX was loaded by a Microsoft Application it could give early alerting on possible exploitation. The same concept could also be applied to Adobe Reader, Java and other commonly exploited third party applications.

Disclaimer
I'm currently not working in an enterprise environment so I can't test these thoughts. Applocker is only available on premium versions of Windows 7 and up.

Kind of a cool feature. Here is a video overview. Anybody have any success stories using AppLocker?

tag:blogger.com,1999:blog-4093139800580227296.post-864720033242284969
Extensions
garts.py
Show full content

'''
Name:
    garts.py (Get all referenced text strings)
Version:
    1.0 
Date:
    2014/05/21
Author:
    alexander<dot>hanel<at>gmail<dot>com
Description:
    garts.py is a simple string viewer for IDA. It will iterate through
    all known functions, look for possible string refercences and then
    print them. This is super helpful for dealing with strings in Delphi
    executables, finding missed strings, having the exact location of
    where a string is being used or where data is possibly going to be
    decrypted.  

    Example Output 
    Address      String
    0x1a701045                  <- new line char. Not the best starting example..         
    0x1a7010bd   #command
    0x1a701199   SOFTWARE\Microsoft\Windows\CurrentVersion\Run
    0x1a7011be   govShell

    Xref Example
    .text:1A7010BD                 push    offset aCommand ; "#command"
    .text:1A7010C2                 lea     eax, [ebp+var_110]
    ....
    .text:1A701199                 push    offset SubKey   ; "SOFTWARE\\Microsoft\\Windows\\CurrentVersi"...
    .text:1A70119E                 push    80000001h       ; hKey
    .text:1A7011A3                 call    ds:RegOpenKeyA

    The script also calls the helpful idautils.strings and then adds all
    the found strings to the viewer window.

    Any ideas, comments, bugs, etc please send me an email. Cheers. 
'''

import idautils
class Viewer(idaapi.simplecustviewer_t):
    # modified version of http://dvlabs.tippingpoint.com/blog/2011/05/11/mindshare-extending-ida-custviews
    def __init__(self, data):
        self.fourccs = data
        self.Create()
        self.Show()

    def Create(self):
        title = "A Better String Viewer"
        idaapi.simplecustviewer_t.Create(self, title)
        c = "%s %11s" % ("Address", "String")
        comment = idaapi.COLSTR(c, idaapi.SCOLOR_BINPREF)
        self.AddLine(comment)
        for item in self.fourccs:
            addy = item[0]
            string_d = item[1]
            address_element = idaapi.COLSTR("0x%08x " % addy, idaapi.SCOLOR_REG)
            str_element = idaapi.COLSTR("%-1s" % string_d, idaapi.SCOLOR_REG)
            line = address_element + "  " +  str_element
            self.AddLine(line)
        return True

    def OnDblClick(self, something):
        value = self.GetCurrentWord()
        if value[:2] == '0x':
            Jump(int(value, 16))
        return True

    def OnHint(self, lineno):
        if lineno < 2: return False
        else: lineno -= 2
        line = self.GetCurrentWord()
        if line == None: return False
        if "0x" not in line: return False
        # skip COLSTR formatting, find address
        addy = int(line, 16)
        disasm = idaapi.COLSTR(GetDisasm(addy) + "\n", idaapi.SCOLOR_DREF)
        return (1, disasm)


def enumerate_strings():
    display = []
    # interate through all functions 
    for func in idautils.Functions():
        flags = GetFunctionFlags(func)
        # ignore library code 
        if flags & FUNC_LIB or flags & FUNC_THUNK:
            continue
        # get a list of the addresses in the function. Using a range of < or >
        # if flawed when the code is obfuscated. 
        dism_addr = list(FuncItems(func))
        # for each instruction in the function 
        for line in dism_addr:
            temp = None
            val_addr = 0
            if GetOpType(line,0) == 5:
                val_addr = GetOperandValue(line,0)
                temp = GetString(val_addr, -1)
            elif GetOpType(line,1) == 5:
                val_addr = GetOperandValue(line,1)
                temp = GetString(val_addr, -1)
            if temp:
                # in testing isCode() failed to accurately detect if address was code
                # decided to try something a little more generic 
                if val_addr not in dism_addr and GetFunctionName(val_addr) == '':
                    if GetStringType(val_addr) == 3:
                        temp = GetString(val_addr, -1, ASCSTR_UNICODE)
                    display.append((line, temp))

    # Get the strings already found
    # https://www.hex-rays.com/products/ida/support/idapython_docs/idautils.Strings-class.html
    s = idautils.Strings(False)
    s.setup(strtypes=Strings.STR_UNICODE | Strings.STR_C)
    for i, v in enumerate(s):
        if v is None:
            pass
        else:
            display.append((v.ea, str(v)))

    sorted_display = sorted(display, key=lambda tup:tup[0])
    return sorted_display


if __name__ == "__main__":
    ok = enumerate_strings()
    Viewer(ok)

Link to Repo
tag:blogger.com,1999:blog-4093139800580227296.post-2996072362004241590
Extensions
ex_pe_xor.py
Show full content
For anyone else who doesn't want to manually carve out single byte XOR encoded executables.

C:\Documents and Settings\Administrator\Desktop\x>ex_pe_xor.py bad.bin
 * Encoded PE Found, Key 0x21, Offset 0x0
 * exe found at offset 0x0

C:\Documents and Settings\Administrator\Desktop\x>dir

04/30/2014  08:36 PM    <DIR>          .
04/30/2014  08:36 PM    <DIR>          ..
04/30/2014  08:36 PM            24,576 1.exe   <- carved
04/30/2014  05:44 PM            24,576 bad.bin
04/30/2014  08:06 PM             3,526 ex_pe_xor.py

Pefile must be installed.

## detects single byte xor encoding by searching for the 
## encoded MZ, lfanew and PE, then XORs the data and 
## uses pefile to extract the decoded executable. 
## written quickly/poorly by alexander hanel 

import sys
import struct
import pefile
import re
from StringIO import StringIO 

def get_xor():
    # read file into a bytearray
    byte = bytearray(open(sys.argv[1], 'rb').read())

    # for each byte in the file stream, excluding the last 256 bytes
    for i in range(0, len(byte) - 256):
            # KEY ^ VALUE ^ KEY = VALUE; Simple way to get the key 
            key = byte[i] ^ ord('M')
            # verify the two bytes contain 'M' & 'Z'
            if chr(byte[i] ^ key) == 'M' and  chr(byte[i+1] ^ key) == 'Z':
                    # skip non-XOR encoded MZ
                    if key == 0:
                            continue
                    # read four bytes into temp, offset to PE aka lfanew
                    temp = byte[(i + 0x3c) : (i + 0x3c + 4)]
                    # decode values with key 
                    lfanew = []
                    for x in temp:
                            lfanew.append( x ^ key)
                    # convert from bytearray to int value, probably a better way to do this
                    pe_offset  = struct.unpack( '<i', str(bytearray(lfanew)))[0]
                     # verify results are not negative or read is bigger than file 
                    if pe_offset < 0 or pe_offset > len(byte):
                            continue
                    # verify the two decoded bytes are 'P' & 'E'
                    if byte[pe_offset + i ] ^ key == ord('P') and byte[pe_offset + 1 + i] ^ key == ord('E'):
                            print " * Encoded PE Found, Key 0x%x, Offset 0x%x" % (key, i)
                            return (key, i)
    return (None, None)

def getExt(pe):
        if pe.is_dll() == True:
            return 'dll'
        if pe.is_driver() == True:
            return 'sys'
        if pe.is_exe() == True:
            return 'exe'
        else:
            return 'bin'
            
def writeFile(count, ext, pe):
        try:
            out  = open(str(count)+ '.' + ext, 'wb')
        except:
            print '\t[FILE ERROR] could not write file'
            sys.exit()
        # remove overlay or junk in the trunk
        out.write(pe.trim())
        out.close()
                        
def xor_data(key, offset):
        byte = bytearray(open(sys.argv[1], 'rb').read())
        temp = ''
        for x in byte:
            temp += chr(x ^ key)
        return temp
        
def carve(fileH):
        if type(fileH) is str:
            fileH = StringIO(fileH)
        c = 1
        # For each address that contains MZ
        for y in [tmp.start() for tmp in re.finditer('\x4d\x5a', fileH.read())]:
            fileH.seek(y)
            try:
                pe = pefile.PE(data=fileH.read())
            except:
                continue 
            # determine file ext
            ext = getExt(pe)
            print ' *', ext , 'found at offset', hex(y) 
            writeFile(c,ext,pe)
            c += 1
            ext = ''
            fileH.seek(0)
            pe.close

def run():
    if len(sys.argv) < 2:
        print "Usage: ex_pe_xor.py <xored_data>"
        return 
    key, offset = get_xor()
    if key == None:
        return
    data = xor_data(key, offset)
    carve(data) 
    
run()
tag:blogger.com,1999:blog-4093139800580227296.post-627757750724460156
Extensions
Upatre, rtrace and XP EOL
Show full content
A couple of days while reading my RSS feeds I noticed an article entitled Daily analysis note: "Upatre" is back to SSL? on the Malware Must Die! blog. During their analysis they mentioned that they didn't solve the obfuscation. I had recently wrapped up an analysis of different obfuscation techniques that Upatre uses. So I thought I'd take a crack at it. After glancing at the sample for a couple of seconds I thought to myself it was a variant of the obfuscated dword XOR sample set. I did a quick educated guess hit F9 in ollydbg and nothing happened. Which is strange because typically Upatre will write the sample to the %TEMP% directory and then try to download it's payload. Instead of the expected activity, I was looking at ExitProcess. At this point my interest was peeked and I wanted to find the anti-debugging...but I didn't care enough to read the assembly code. The code looked somewhat obfuscated.



Rather than reversing the obfuscation I decided to take a different route. Ollydbg has a useful feature called Run Trace or rtrace for short. It's great for tracking down anti-debugging or logging changes to registers. It's not a feature I would recommend for tracing over large amounts of code. Creating a PinTool would be a better choice for this type of task but rtrace is good for small jobs. Here are three excellent reads on using Pintool for similar tasks 1, 2 and 3.

rtrace log Rtrace logs the address, thread, instructions, and the register changes. This is very useful for if you want to review how specific registers are changed.  The rtrace output can be saved to a file in Ollydbg by selecting View, Run Trace, right-click, and Log to file. Since rtrace logs can be quite large it's best to only trace code that is worth logging. Breakpoints can be used for creating the boundaries of the trace data. Rather than working with rtrace log as a text file I decided to create an ordered dictionary and save it to a JSON using Python. Once the rtrace log is converted to a JSON I can populate the IDB with interesting values. Some values that I thought were interesting were the count of how many times an address got called, the amount of unique register values for an instruction. For example if the following instruction sub     ecx, 0D733EFF3h is called X number of times; how many different values of ECX were there? If the set is 1 but the instruction was called 6,000 different times we know it's a static value and not give it another thought. Here is the Python code that I hacked together. 


'''
Name:
    rtrace_2_idb.py
Date:
    2014/04/??
Author:
    alexander<dot>hanel<at>gmail<dot>com
'''
import sys
import json
import collections

##########################################################
def run():
    'this function creates an ordered dict saved to a json from an rtrace log'
    rtrace = collections.OrderedDict()
    # rtrace log passed as argument
    with open(sys.argv[1]) as f:
        for line in f:
            addr, reg_values = parse_line(line.rstrip())
            if addr:
                try:
                    addr = int(addr,16)
                except:
                    pass
                if rtrace.has_key(addr):
                    if len(reg_values) == 0:
                        rtrace[addr].append([])
                        continue 
                    for val in reg_values:
                        rtrace[addr].append(val)
                else:
                    rtrace[addr] = []
                    if len(reg_values) == 0:
                        rtrace[addr].append([])
                        continue 
                    for val in reg_values:
                        rtrace[addr].append(val)
    save_off(rtrace)
            
def save_off(rtrace):
    'save the data to a file named rtrace.json'
    with open("rtrace.json", 'w') as f:
        json.dump(rtrace, f)

def parse_line(line):
    'parses the rtrace log' 
    temp = line.split('\t')
    try:
        return (temp[0], list(temp[3:]))
    except:
        return None, None                 

##########################################################

def show_data():
    'prints all the modified registers for an address in a rtrace log'
    try:
        print rtrace_data[str(here())]
    except:
        print "Error: never called"

def show_next():
    'prints next called address. Kind of buggy on ret'
    try:
        temp = rtrace_data.items()[rtrace_data.keys().index(str(here())) + 1]
        print hex(int(temp[0]))
    except:
        print "Error: Could not be found"
    
def populate():
    'populate the IDB with counts and set values'
    import idautils
    import idaapi
    global rtrace_data
    rtrace_data = get_json()
    idaapi.add_hotkey("Shift-A", show_data)
    idaapi.add_hotkey("Shift-S", show_next)
    for func in idautils.Functions():
        flags = GetFunctionFlags(func)
        if flags & FUNC_LIB or flags & FUNC_THUNK:
            continue  
        dism_addr = list(FuncItems(func))
        for line in dism_addr:
            com = format_data(rtrace_data, line)
            if com:
                MakeComm(line, com)

def format_data(rtrace_data, line):
    try:
        instr_data = rtrace_data[str(line)]
    except:
        return None
    count = len(instr_data)
    # Empty lists are not hashable for sets. 
    try:
        data_count = len(set(instr_data))
    except:
        data_count = None
    if data_count:
        comment = "C: 0x%x, S: 0x%x" % (count, data_count)
    else:
        comment = "C: 0x%x" % (count)
    return comment
                    
            
def get_json():
    try:
        f = open("rtrace.json")
        js = json.load(f, object_pairs_hook=collections.OrderedDict)
    except e:
        print "ERROR: %s" % e
    return js

##########################################################
    
if __name__ == '__main__':
    try:
        sys.argv[1]
        run()
    except:
        populate()
        
# the sys.arv[1] exception is a hack to guess how the script is being
# called. If there is an exception it is being run in IDA.

To execute the code above pass the output log file to the script, copy the rtrace.json to the working directory of the script and the IDB and then execute the script in IDA. This will give an output as seen below. The C is for Count and S is for Set. Glancing over other functions it is easy to see which instructions are responsible for looping and decoding data.
Since the executable only had 17 functions it was easy to identify instructions that were not called and then investigate why they weren't. Notice the fourth and fifth block was not called. This is due to the returned values of what is called at EBX.


Since rtrace logs all modified register values it is easy to access those values. Included in the script is the ability to print all those values. This can be done by selecting the address and pressing Shift-A. To print the next address called, select the address and the press Shift-S. The arrow is the address of the selected line.


Once I followed the address in the output window it lead me to what called ExitProcess. Now all I needed to do was investigate the calls of EBX in the first block of the previous to see what is being checked and patch the ZF flag results to continue execution. What makes this sample interesting (and actually worth reading the code) is Upatre is using an undocumented technique to determine if it is running on a Windows NT 6.0 or higher. I'm unaware if this techniques works on Windows Vista. I have only tested on Windows XP SP3 (NT 5.1) and Windows 7 ( NT 6.1).


The malware calls RtlAcquirePebLock and NtCurrentTeb twice. On Windows XP when RtlAcquirePebLock is called the first time ECX, EDX and EAX is over written. ECX will be the return address of RtlAcquirePebLock, EDX will be the address of PEB FastPebLoclk which is a pointer to _RTL_CRITICAL_SECTION and EAX will be zero. On the second call only EAX will be over written. On Windows 7 when RtlAcquirePebLock is called EAX will become zero and ECX will be equal to the Thread Information Block (TEB) ClientId. On the second call to RtlAcquirePebLock EAX will be zero, EDX will be the TEB CliendId but ECX will be equal the the TEB. On the second call to RtlAcquirePebLock  if  ECX is equal to TEB or the return of NtCurrentTeb the sample is running on NT 6.0 or higher.

Below is the code rewritten in C++

#include <windows.h>
#include <stdio.h>
#include <iostream>

int getECX() {
 int value = 0;
 //Moves edx into 'value'
 _asm mov value, ecx;
 return value;
}

int getEAX() {
 int value = 0;
 //Moves edx into 'value'
 _asm mov value, eax;
 return value;
}

int getEDX() {
 int value = 0;
 //Moves edx into 'value'
 _asm mov value, edx;
 return value;
}
int main()
{
   HMODULE hModule = GetModuleHandleA(("ntdll.dll"));
   FARPROC pRtlAcquirePebLock = GetProcAddress(hModule, "RtlAcquirePebLock");

   HMODULE h2Module = GetModuleHandleA(("ntdll.dll"));
   FARPROC ntcur = GetProcAddress(h2Module, "NtCurrentTeb");

   pRtlAcquirePebLock();
   // ecx, eax & edx are modifed when RtlAcquirePebLockis called
   int ecxValue = getECX();
   int eaxValue = getEAX();
   int edxValue = getEDX();

   // call current teb
   ntcur();
   eaxValue = getEAX();

   if (eaxValue == ecxValue)
    std::cout << "EAX & ECX Match - second stage" << std::endl;
   if (eaxValue == edxValue)
    std::cout << "EAX & EDX Match - second stage" << std::endl;

   // lock again 
   pRtlAcquirePebLock();
   ecxValue = getECX();
   eaxValue = getEAX();
   edxValue = getEDX();

   // current teb 
   ntcur();
   eaxValue = getEAX();

   if (eaxValue == ecxValue)
   std::cout << "EAX & ECX Match - second stage aka > XP" << std::endl;
} 

The changes in the return values are caused by the difference in how RtlAcquirePebLock calls RtlEnterCriticalSection. In Windows XP RtlEnterCriticalSection is called by being passed a pointer  from PEB FastPebLockRoutine. Since the PEB is writable from user mode the FastPebLockRoutine, it can be over written to cause a heap overflow. See Refs 1 and 2. Below we can see the difference between XP and Win7 for RtlAcquirePebLock.
_RtlAcquirePebLock XP _RtlAcquirePebLock Win7
Pretty interesting technique for avoiding Windows XP. Thanks to Hexacorn for help and feedback. If  I wanted to continue executing the malware I would either need to patch the return of the second call to RtlAcquirePebLock or patch the comparison. Please feel free to send me any feedback, criticism or notes via email (in the python code), hit me up on Twitter,  or leave a comment. Cheers.

Hash - 891F33FDD94481E84278736CEB891D1036564C03

References
[1] http://net-ninja.net/article/2011/Sep/03/heap-overflows-for-humans-102/
[2] http://www.exploit-db.com/papers/13178/
http://www.debugwin.com/home/fastpeblock
http://msdn.moonsols.com/winxpsp3_x86/PEB.html
tag:blogger.com,1999:blog-4093139800580227296.post-6081369436711962343
Extensions
Upatre: Sample Set Analysis
Show full content

Hi. I recently wrapped up an analysis of Upatre. My original intention was to write a generic C2 decoder/extractor for the executables. After analyzing ten samples I realized it was not feasible due to the different encoding algorithms and obfuscation. After analyzing the ten samples I became interested in how Upatre obfuscates/encodes it's executables. My analysis is based off of 94 unique Upatre samples. My sample set might be little skewed because I grabbed the first 100 files sorted by file size with a type zip. Having the files in their original zip file allowed me to have their original file names. I would like to thank VirusTotal for access to the samples and Glenn Edwards for feedback.

Google Docs (HTML)
Git Repo (PDF)
tag:blogger.com,1999:blog-4093139800580227296.post-4911836677753170760
Extensions
PE Skeletons
Show full content
I know this topic has been beaten to death but I thought I'd share a technique for detecting single byte XOR executables in file streams. Recently while looking at a file stream I instantly knew there was an encoded XOR executable file in it. This got me thinking, why can I spot this and can I script it up? Since executable files are a defined structure they have a standard skeleton to them. It's not always easy to see the skeleton if we just look at the hex bytes.


We can add some color via the following Python code. 

import matplotlib.pyplot as plt
import numpy as np
import sys

def main():
        data = open(sys.argv[1], 'rb').read()[:512]
        dlist = bytearray(data)
        print len(dlist)
        plotters = np.array(dlist)
        plotters.shape = (32,16)
        #plt.bone()
        plt.flag()
        plt.axis([0,16,0, 32])
        plt.pcolormesh(plotters)
        plt.show()
        plt.savefig('test.png')
        plt.close()

if __name__ == '__main__':
        main()
                   

The code reads the first 512 bytes of a file, puts each byte into a bytearray and then plots the color in the same structure as the hex dump. If we were to pass an executable file to this script we would get the following pretty picture. 
The bottom left hand corner byte is 0x4d 'M' the second is 0x5a 'Z' and so on. If we were to XOR the executable with  0x88 we would get the following image.
XOR 0x88 Key If we were to think about the red in the first image and blue in the second image as negative space we would see the PE skeleton. Okay, that was cute, now let's see if we can detect this in a file stream. Since the Portable Executable have a standard structure. The beginning starts with 'MZ', jump 0x3C bytes, read four bytes to get the address of the PE, then check if "PE" is at the read offset. This is a highly dumb down version. Check out PE101 by Ange Albertini for an awesome introduction if my definition is unclear. Since these are standard steps all we have to do is check for the same structure but with XORed data.


import sys
import struct

# read file into a bytearray
byte = bytearray(open(sys.argv[1], 'rb').read())

# for each byte in the file stream, excluding the last 256 bytes
for i in range(0, len(byte) - 256):
        # KEY ^ VALUE ^ KEY = VALUE; Simple way to get the key 
        key = byte[i] ^ ord('M')
        # verify the two bytes contain 'M' & 'Z'
        if chr(byte[i] ^ key) == 'M' and  chr(byte[i+1] ^ key) == 'Z':
                # skip non-XOR encoded MZ
                if key == 0:
                        continue
                # read four bytes into temp, offset to PE aka lfanew
                temp = byte[(i + 0x3c) : (i + 0x3c + 4)]
                # decode values with key 
                lfanew = []
                for x in temp:
                        lfanew.append( x ^ key)
                # convert from bytearray to int value, probably a better way to do this
                pe_offset  = struct.unpack( '<i', str(bytearray(lfanew)))[0]
                # verify results are not negative or read is bigger than file 
                if pe_offset < 0 or pe_offset > len(byte):
                        continue
                # verify the two decoded bytes are 'P' & 'E'
                if byte[pe_offset + i ] ^ key == ord('P') and byte[pe_offset + i + 1] ^ key == ord('E'):
                        print "Encoded PE Found, Key %x, Offset %x" % (key, i)
~                                                                                 
Speed, false postives testing, etc are all probably areas of improvement for the code.
If we were to run this on the executable XORed with 0x88 we would be present with the following output Encoded PE Found, Key 88, Offset 0. Here is a script that will automatically find a XOR encoded executable and carve it out using the above code and Pefile.

Kind of a cool technique to use the Portable Executable structure to find XOR exes. It only works on single byte executables. Could be modified for 2 or 4 bytes. Not sure about anything higher. A brute force approach would probably be better for key byte size of anything higher than 4. The skeleton is prevalent when an executable is XORed with a key of five bytes in size. Using gray tones can help show the skeleton because the contrast is dulled.

Useful Links
http://tomchop.me/2012/12/yo-dawg-i-heard-you-like-xoring/
http://hiddenillusion.blogspot.com/2013/01/nomorexor.html
http://playingwithothers.com/2012/12/20/decoding-xor-shellcode-without-a-key/
http://www.cloudshield.com/blog/advanced-malware/how-to-efficiently-detect-xor-encoded-content-part-1-of-2/
http://digital-forensics.sans.org/blog/2013/05/14/tools-for-examining-xor-obfuscation-for-malware-analysis


Side Note:
I have to admit I'm a huge fan of using ByteArrays now. I wish I could have of learned of them sooner. They are very useful for writing decoders. It remove a lot of the four play of checking the computed size ( value & 0xFF), using ord() and using chr().
tag:blogger.com,1999:blog-4093139800580227296.post-5740380352240341117
Extensions
xxxswf.py updates
Show full content
A new version of xxxswf.py has been pushed to it's repo. The current build handles the extracting and decompressing of LZMA compressed SWFs. In order to decompress ZWS SWFs pylzma will need to be installed.

__@____:~/projects/swfs/ZWS$ python xxxswf.py -x c026ebfa3a191d4f27ee72f34fa0d97656113be368369f605e7845a30bc19f6a 

[SUMMARY] Potentially 1 SWF(s) in MD5 d41d8cd98f00b204e9800998ecf8427e:c026ebfa3a191d4f27ee72f34fa0d97656113be368369f605e7845a30bc19f6a
 [ADDR] SWF 1 at 0x4008 - ZWS Header
  [FILE] Carved SWF MD5: 14c29705d5239690ce9d359dccd41fa7.swf

At least ninety percent of the code has been rewritten. A lot of bugs were fixed. The current build is 1.9.1. I'm still wanting to add more features and test newly added functionality but due to the increase use of malicious ZWS I decided to push this out. I have tested it against a couple of hundred malicious files and I have found no errors when running xxxswf.py from the command line. All of the traditional features when invoked from the command line have been tested and our stable. New features include being able to create an xxxswf instance and calling functions for prepping or converting the file stream and scanning the extracted SWF(s). If we wanted to decompress a Microsoft Zip+XML file and then extract embedded SWF(s), this would be a prepping function. I need to write more functions to figure out a good work flow. Once I get this sorted out I'll push out the 2.0 version. If you find any errors please send me an email, leave a comment or ping me on twitter.
tag:blogger.com,1999:blog-4093139800580227296.post-6262148663577168368
Extensions