RolfRolles
September 16th, 2009, 21:09
If you who would like to take your reverse engineering skills to the next level, I suggest you drop whatever you're doing and dig into VINE, the newly-released static analysis component of the BitBlaze project ("http://bitblaze.cs.berkeley.edu/vine.html"): http://bitblaze.cs.berkeley.edu/vine.html. Among other things, this is the technology behind automated exploit generation (in its various guises), automated vulnerability signature generation, etc. You can see a full list of their publications here ("http://bitblaze.cs.berkeley.edu/bitblaze-pub.html"): http://bitblaze.cs.berkeley.edu/bitblaze-pub.html. Also, BAP (a spin-off) was released recently here ("http://bap.ece.cmu.edu/"): http://bap.ece.cmu.edu/.
Be forewarned that reading VINE or BAP requires knowledge of the following; I have suggested reading material inline.
* OCaml (at an advanced level)
* * explicit continuation-passing style
* * the module system
* * labeled/optional arguments
* * polymorphic variants
* * OCaml's weird OOP implementation
Read: "The functional approach to programming", and the OCaml manuals
* compiler theory
* * intermediate languages
* * data flow analysis
* * SSA form
Read: "Modern Compiler Implementation in ML", and "Advanced Compiler Design and Implementation"
* program analysis
* * lattice theory
* * formal semantics
* * weakest preconditions
* * symbolic execution
* * theorem proving
* * abstract interpretation
Read: "Introduction to Lattices and Order (2nd edition)", "Semantics with Applications: An Appetizer", "Principles of Program Analysis", "A Discipline of Programming", and every BitBlaze paper from start to finish.
Be forewarned that reading VINE or BAP requires knowledge of the following; I have suggested reading material inline.
* OCaml (at an advanced level)
* * explicit continuation-passing style
* * the module system
* * labeled/optional arguments
* * polymorphic variants
* * OCaml's weird OOP implementation
Read: "The functional approach to programming", and the OCaml manuals
* compiler theory
* * intermediate languages
* * data flow analysis
* * SSA form
Read: "Modern Compiler Implementation in ML", and "Advanced Compiler Design and Implementation"
* program analysis
* * lattice theory
* * formal semantics
* * weakest preconditions
* * symbolic execution
* * theorem proving
* * abstract interpretation
Read: "Introduction to Lattices and Order (2nd edition)", "Semantics with Applications: An Appetizer", "Principles of Program Analysis", "A Discipline of Programming", and every BitBlaze paper from start to finish.