	Know Thy Language! The importance of knowing (your enemy's) language	Advanced Javascript
24 February 1998	by Jack of Shadows
	Courtesy of Fravia's page of reverse engineering
fra_00xx 98xxxx handle 1100 NA PC	Well, I see that I'm called "enemy" in here... suppose I'am, sort of, if you have been passing so many hours before entering :-)
	There is a crack, a crack in everything That's how the light gets in
Rating	(x)Beginner (x)Intermediate ( )Advanced ( )Expert

We'll see how some knowledge about the language our enemy is using can help us.

Know Thy Language!
The importance of knowing (your enemy's) language
Written by Jack of Shadows

Introduction

I have struggled with Fravia+'s puzzle far too long but at least I have found a neat approach after some 8 hours of useless work. I won't talk about all dead ends since they really aren't interesting. We'll just take a short look at a program that spits out correct name of advanced javascript page as a very first of produced solutions.

Tools required

My_private_web_page_grabber (no, it's not available for free distribution)
Delphi for page list generation (I hate C, so sue me!)
Turbo Pascal for simple filters
Van Buerg's List for browsing the page listings
Conan Doyle's Study in Scarlet

Target's URL/FTP

Fravia+'s help for hopeless lusers

Essay

Let's do it by the numbers skimming over unimportant stuff real fast.

Page grabbing

I have used some multipurpose program I have partially written over last few months. Amongst other things it can serve as a page catcher. I'm mentioning this only because I haven't seen similar approach yet.

Page listing generation

I have written a simple recursive program in Delphi (console app, of course, without any UI). Saw a result. Took a deep sleep.

Narrowing the search

Few days later I have read the Fravia+'s help for hopeless lusers. Hmm, no numbers and ending in "r"! That should help. Rerun the program with filter. Still more than 60.000 possible pages :---(

I must mention that at this stage I have blatantly overlooked all Fravia+'s hints (and there weren't just one) that page name contains only small letters. Shame on me!

Using the brain (at last)

After I have tried several stupid approaches (I am not brave enough to mention them at all) an interesting thought occurred to me. Since the name I'm searching for is very probably written in English it should (because of it's length - 9 chars (+1)) follow letter frequency for English language quite close. Even the letter pair frequencies should resemble English quite well. There was one more problem - I had no English texts at hand (yep, nonEnglishPerson, as you have already guessed). I didn't even bother to fire up my old Encarta. "If you can use the net, then use it!" I have said, surfed to nearest Project Guttenberg and downloaded Conan Doyle's Study in Scarlet.

Then I wrote an easy program to calculate letter pair frequencies from this novel. Hardly a rocket science so I won't repeat it here.

Finding the solution

Only one piece remaining. A simple program that calculated every page's probability according to precomputed table and prepended page name with that number. It's here:


program scanfreq;



uses

  SysUtils,

  MSString; // just a little library of mine



var

  occur: integer;

  table: array ['A'..'Z','A'..'Z'] of integer;

  tablef: array ['A'..'Z','A'..'Z'] of double;



  procedure ReadTable; // read precomputed table, convert integer numbers to reals

  var

    f: file;

    c1,c2: char;

  begin

    Assign(f,'freq.dat');

    Reset(f,1);

    BlockRead(f,occur,SizeOf(occur));

    BlockRead(f,table,SizeOf(table));

    Close(f);

    for c1 := 'A' to 'Z' do

      for c2 := 'A' to 'Z' do

        tablef[c1,c2] := table[c1,c2] / occur;

  end; { ReadTable }



  function Process(ln: string): string;

  var

    sum: double;

    i: integer;

  begin

    ln := Upper(ln);

    sum := 0;

    for i := 1 to 8 do  // sum the probability for given word

      sum := sum + tablef[ln[i],ln[i+1]];

    Process := Str2(Round(sum*1000),4);

  end; { Process }



var

  f,g: text;

  ln: string;



begin

  ReadTable;

  Assign(f,'codes.txt');  Reset(f);

  Assign(g,'freq.txt');   Rewrite(g);

  while not Eof(f) do begin

    Readln(f,ln);

    Writeln(g,Process(ln),' ',ln);

  end;

  Close(g);

  Close(f);

end.

I have sorted the results in descending order and browsed. I found the right page name in line 41 (Fravia, that should really be 42 (for the moment I thought that I have found the Question), shame on you for putting off-by-one error into your puzzle;-). Not bad, really, the answer has surfaced almost to the top.

Refining

And then I have entered advanced javascript pages, read the essays and realized how stupid I was to even consider the possibility of uppercase letters. Just for a little reality check I have filtered my page list to contain only lowercase page names and rerun the frequency calculation program. Not very surprising - in the resulting file the right page name surfaced to the first place. Here are the first ten entries:


 114 avfurther

 112 vafurther

 111 vdcurther

 111 dvcurther

 105 verilvher

  79 aveuslher

  77 pcxeawher

  77 cpxeawher

  75 evrilvher

  69 ulwaporer

Final Notes

Lesson should be obvious - use language rules when checking the passwords. Use the net if you don't have all the data you need.

You are deep inside Fravia's page of reverse engineering, choose your way out:

Back to the javascript entrance

Advanced Javascript

homepage

links

search_forms

+ORC

students' essays

academy database

reality cracking