The first thing to try is attacking the RNG. He's using time(NULL), which limits the number of bits tremendously. Of course, he could have used dev/random, but it's worth a try.
Say he made the challenge this year; then the time has to be between 3C4899DD and 3D372B06, or 25 bits. We should obviously start at the near date & work backwards.
The encryption looks straightforward:
Code:
int cipher_sub_encrypt(const unsigned char *inblk, unsigned char *
outblk, int blksize, unsigned char *key) {
int i,mod;
static int keyoffset=0;
mod=(int) key[0];
for (i=0;i<blksize;i++) {
if (!(i%mod)) {
keyoffset=((keyoffset+1)&0xff);
}
outblk[I]=key[((((int) inblk[I])+keyoffset)&0xff)+1];
}
return(blksize);
}
key is a structure: 1 byte that controls how often keyoffset updates (call it
u), followed by a static 256-byte permutation. That means that it's a simple substitutuin cipher (like the cryptograms in the newspaper) except that every
u bytes, the cipher will change slightly-- if a,b,c -> q,r,z in the first
u bytes, then b,c,d -> q,r,z in the second
u bytes.
We can loop through the values for
u and look at the statistics of the bytes in chunks of size
u. We also have some known plaintext.