Say I had a pointer in the register esi to a bit of memory, how can i follow this in the dump?

I'm not exactly sure what you mean, but if you want to check whenever the program accesses that memory, why not setup a hardware break-on-access breakpoint?

Well, say eax is a pointer to a space of memory. I can see what eax is, but not whats in the memory at eax which I need to see. If I right click and goto follow in dump->memory address it brings me to a location, but doesn't tell me where the data starts, so I have no idea where the data is

say you have this
mov eax,401008
after you process this instruction eax will be 401008
now if you right click and follow in dump
dump will show you what is there at 401008

at 401008 there may be an ascii null terminated string like "this is what you are looking for"

or 401008 may hold one more address like 401067

or it may be a constant like 0000000A

if it holds an address you may follow it also just select the dword and right click you get two more options like follow in dump or follow in dissembler

deciphering is an art olly is not a magicians wand and it does not have extra sensory perception

say i have
push eax
just before a call. And eax is a pointer to a piece of data that I need, say, 00ADBBA0
if i right click, goto follow in dump, and press memory address. It brings me to some data in the dump, but I don't know where abouts it is in the dump, I dont know where the dword starts. Can you help with that?

Not dword, i mean data.

I'm afraid you'll have to make yourself more clear. It sounds like you are trying to find the start of data memory, which doesn't exist, as data can be stored anywhere (even in code section).

No...
There is a pointer to some data in a register while debugging. And I need to be able to find that data where ever it is, and obtain it.

Right click register and "follow in dump".

I've tried that. It goes to a place in the dump, but it doesn't tell me where the data starts, so really I am as clueless as before. Help?

When you use Follow in Dump, Olly jumps in the Hex-Dump Window exactly to the _first Byte_ to which your adress points. You should see left in the address field your value form eax, or whatever you tried to follow...