noorqureshi/security-skills/ctf-crypto

CTF crypto challenges

When it applies

The handout is a short encryption script (chall.py, rarely sage), a set of numbers (n, e, c / intercepted ciphertexts), or an nc service that encrypts/signs things for you. This skill covers the CTF shapes — reading the script for the flaw, XOR games, homemade ciphers, and math tooling. For the deep attack mechanics it defers to crypto-rsa-attacks and crypto-oracle-attacks; run the RSA battery there first.

Why it works

CTF crypto is a code review against math: the script is short, self-contained, and contains exactly one mistake the author planted — a reused nonce, a factorable n, an invertible homegrown round function, or an oracle the service hands you. Find the one line that looks "clever" and the challenge is usually done; the rest is algebra.

Method

1. Read the script as a spec, top to bottom. Note exactly what's given (printed values) and

what's hidden (keys, seeds, flag). The flaw is almost always in: key/nonce generation (random instead of secrets, timestamp seeds, getPrime with small bits, reused k), the composition (same key twice, encrypt-then-leak-something), or the padding (raw RSA, ASCII armor that leaks length).

1. Triage encodings before crypto. Half of "crypto" challenges are encoding stacks: base64/32/

85, hex, morse, brainfuck, base-N with a custom alphabet, repeated base64. CyberChef's magic wand or a quick decode loop settles these in minutes — don't reach for sage on a base85.

1. RSA → run the known battery. Extract n, e, c; hand the key to RsaCtfTool + factordb,

then check the CTF-specific shapes explicitly (crypto-rsa-attacks for mechanics):

• Multiple keys/ciphertexts in one file → gcd pairwise (shared prime), common modulus

(same n, two e), or Håstad broadcast (same message, small e, several n).

• p and q generated from a weak PRNG / shared high bits / q = next_prime(p) → Fermat or

regenerate the PRNG stream.

• Extra printed values (d mod (p-1), p+q, dp, partial bits of p) → Coppersmith-style

recovery (sage) or direct algebra; these "hint leaks" are the intended path.

1. XOR challenges.

• Single-byte XOR: brute all 256 keys, score by English frequency (xortool automates).
• Repeating-key XOR: guess key length by Hamming distance / index of coincidence, solve each

column as single-byte.

• Multi-time pad (same keystream XORs several messages, classic otp reuse): crib-drag —

XOR two ciphertexts and guess words ("the ", "flag{") at each position; each confirmed crib extends the keystream.

• XOR with a known plaintext anywhere (a known header, the flag format flag{) recovers

keystream bytes directly.

1. Encryption services are oracles — script them. nc service that encrypts your input:

• Your input prepended to the flag, ECB → byte-at-a-time flag recovery (crypto-oracle-attacks).
• Same plaintext always → same ciphertext: codebook lookup / chosen-plaintext mapping.
• Decrypt-anything-but-the-flag → malleability games: bit-flip CBC, RSA blinding

(c·r^e mod n, unblind the result).

• It always helps to send structured probes: all-zero blocks, single-byte deltas, length sweeps

(leaks block size and mode).

1. Homegrown ciphers → model, don't guess. Custom rounds (add-rotate-xor, matrix mods,

LFSRs, "Feistel-ish"): symbolically execute or invert them.

• Linear operations over GF(2) → write as linear equations, solve with z3 or sage.
• Small state (LFSR) → Berlekamp–Massey from a known keystream.
• S-box + permutation networks → differential or just brute the tiny key space the author used.
• Ask: is every step invertible given the key? Then the only question is the key space —

brute 16/24/32-bit keys before doing anything clever.

1. Bring the math tools. z3 for constraint-shaped problems (flag bytes satisfying check

equations); sympy for modular algebra and small discrete logs; sage for lattices (LLL / Coppersmith / hidden-number problems); long_to_bytes/bytes_to_long endianness discipline throughout.

Gotchas

• The printed "leak" is intentional. If the script hands you hint = p ^ q or

leak = d & ((1<<512)-1), that's the solve path — stop looking for a second bug.

• Python `random` is not crypto — random.seed(time.time()) or Mersenne Twister output

visible anywhere → clone the state (624 consecutive 32-bit outputs fully recover MT19937).

• `e` and `phi` mistakes — wrong totient (using n for (p-1)(q-1)) makes a key that looks

right but fails; verify by decrypting a known value (crypto-rsa-attacks).

• Multi-prime RSA (n = pqr) — factorization tools still work; remember phi = (p-1)(q-1)(r-1).
• Encoding is not encryption — if "ciphertext" decodes cleanly to printable text, keep

peeling encodings before assuming a cipher.

• Byte-vs-int conversion bugs in *your* solver — leading zero bytes vanish in

bytes_to_long; pad the recovered plaintext back to the expected length.

Verify success

The recovered plaintext contains the flag in the event's format, and you can re-derive it deterministically (script reproduces the key/plaintext from the given artifacts, not from luck).

References

Mechanics: crypto-rsa-attacks (factordb, Fermat, Wiener, Håstad), crypto-oracle-attacks (padding oracle, ECB byte-at-a-time, length extension); tooling: z3, sage (LLL/Coppersmith), xortool, CyberChef. Triage via ctf-methodology.