I apologize for the length and rambling nature of the following post
but I feel both are necessary. I would appreciate helpful comments on how to proceed, or on my immediate problem of getting IDA to load a DOS file correctly.

I realize this is off-topic in most aspects but there is one angle I would like to pursue that is right up our alley. Modern hard drives use code in two places: one place is in the processor chip and the other is on the disk itself on a platter. This firmware, or microcode, is used to run the processor and to govern all aspects of reading/writing to disk as well has head positioning and the spindle motor.

One problem is that the drive has a diagnostic routine it goes through before making itself available to the motherboard BIOS. If it doesn't complete that diagnostic because of damaged firmware on the disk, it will sit there ticking and wont be recognized by the BIOS. Apparently damaged firmware is a major cause of hard drive failure, on some brands in particular.

I am an electronics technician and have repaired hard drives in the old days. I read this thread with some interest and horror: http://www.woodmann.com/forum/showthread.php?t=7301&highlight=hard+drive. My interest came from hearing that people have opened the head chamber and got the drive working again, There is a feeling out there akin to paranoia that opening the head chamber is the kiss of death. My horror came hearing about people tapping on the heads while the drive is spinning, etc. You just don't do that. If that head touches down, it's game over. It's only flying a few thou above the surface.

Anyway, I would like to reverse the firmware in the main processor to gain more control over the disk operation. Most hard drives have a 'safe mode' that can be entered by using an extra jumper on the master/slave select pins. In safe mode, the start up routines are apparently bypassed, although I haven't tried that yet. There is another jumper for 'factory mode' which needs some serious looking at.

BTW...access to the processor is governed by the ATA standards which are freely available. You can interrogate it and hopefully make it cough up it's firmware, then put it back in an amended form.

I have a piece of software put out by a computer company to correct issues in the firmware of an older 10 gig drive. I am theorizing that the software must address the disk processor to change it's firmware and all of that code should be available in this software. I need to reverse this software, which creates a floppy, and integrates a firmware binary into the floppy boot process somehow.

The program is a 16 bit DOS app, so I am reviewing DOS. I'm somewhat stymied right now because I'm using IDA and it does not create proper segments for the DOS app. In the DOS header, there is a double-word that specifies the initial CS:IP as 129C:0001. In IDA, it shows up as SEG 0:000229D0, and this seg is right at the end of the disassembly. It's easy to see that the correct address segment of 129C is pretty close to the 229D0 of IDA, but it's displaced by 20010 bytes.

I realize there's a difference between the CS:IP specified in the file header and the disassembly. IDA has flagged a lot of code as data, however, and I'd like to amend that. I'd like to also arrange the disassembly with it's proper segments. Is that possible and where is that information in a DOS file?

I've had a lot of frustration trying to manipulate the segments in IDA. It keeps asking for a base address but does not explain it very well. Given that the file is an exe file, it would presumably have more than one 64K segment. Then again, I'm confusing the DOS format with the format of the NE file which uses similar segmentation. Can someone shed some light on this?

I've never really understood the relocatability of segments in DOS and NE files. That's probably what's bugging me now but I've spent so much time trying to visualize the reasoning behind it and I still don't get it. I understand the offset clearly but it's the notion that the CS part can take on other values that stumps me. Obviously, IDA felt OK about relocating it to 0:000229D0 when the app itself specified 129C as it's base.

It seems to me I should load it into IDA without segments and create my own. If I enter 000129C0 as it's base, it will start yapping about a negative offset.

Before someone berates me for not doing a proper search, let me assure you that I've done a lifetime of research on hard drives alone in the past few weeks. I have been furiously translating Russian and Chinese text, and my head hurts. I have spent a good deal of time on this DOS problem, both on the net and in this forum. I read one article in the forum on this kind of segmentation but it didn't turn on a light. There seems to be pitifully little left on the net about DOS other than cursory outlines.

There was an article I read once in this forum by someone who was working with microcontrollers. It would be nice to converse with someone who understood the ins and outs of them vis-a-vis the firmware, with the reversing of it as an aim.

Howdy,

Have you poked around over at programmers heaven yet ?
They have lots of older DOS type stuff.

As for reversing old firmware, I dont quite understand how you could change settings on a hard drive any other way than through the bios.
Perhaps some research in changing the MBR will yield some clues.

Woodmann

Somewhere around here I have a disassembler, that I bought over 10 years ago specifically for DOS, however it's on 5 1/4 floppy, (if I still have it), when I get a chance I'll take a look and if I can find it - I'll search through my junk and see if I still have a 5 1/4 drive - even if I know the name - it may be available somewhere on the web.

SiGiNT

Various versions of IDA are known for not disassembling MZs properly. I prefer v3.6 since that seems to work quite well.

You might want to take a look at Sourcer. It's not interactive, but it's one of the older intelligent disassemblers.

The "segments" in MZs are actually only partial segments, since they often overlap.

How's it going? Thanks for the reply and the tip. I'll follow up on that but most of what I've seen on the net is aimed at DOS as an adjunct to Windows programming. I did have an excellent Peter Norton book on DOS, but I may have fallen into the old trap of throwing it out, thinking I'd never need it again. :-)


this drive is beyond the BIOS, so to speak, it is no longer pining for the fjords, as the shopkeeper from 'Notlob' (palindrome for Bolton) claimed in the Monty Python parrott sketch. Even without the data cable attached, it is seeking and ticking away. That indicates bad firmware, a bad head, a bad preamp or another malfunctioning part of the drive electronics board. I have been almost certainly assured by an expert in Russia that it's in the firmware.

With regard to changing settings on the drive itself through firmware, the entire functioning of a hard drive today is done through code in the eprom of its central processor, in my case a 176 pin DSP. The larger modern drives actually have flash capability to change the firmware, but in the older one to which I referred, it is done by addressing the eprom part of the processor somehow. I'm trying to find a data sheet on my controller (a Texas Instruments D741667APGF) but it is proprietary information that TI wont even provide.

Some of the code is kept on the disk in a special service area. Also, in that area can be found data that affects bad sectors and a host of other parameters affecting drive operation. That was apparently done so the manufacturers wouldn't have to reprogram the processor each time they wanted to modify the operating code. Unfortunately, if that microcode on the disk surface messes up, the mechanism gets confused.

In that way, it's like the MBR code, but that code is a bootstrap for the operating system. The drive goes through its own bootup routine with or without an OS loaded. In fact, it goes through it without a BIOS in place. If it doesn't complete it's routine, it gets downright ornery.

The motherboard BIOS only gets into the act after the drive has initialized itself and performed it's diagnostics. Part of those diagnostics is a S.M.A.R.T check, which is the disk's built in method of estimating its own health. The disk(s) are driven by a spindle motor that is a high precision unit. It is a three phase motor operating as a servo unit, and the data for driving it and maintaining it's speed is all built into the microcode. On top of that, the head positioning mechanism has to work in conjunction with the spindle motor and is itself a high precision servo mechanism. There is a microchip positioned on the positioning mechanism inside the head chamber that acts as a preamplifier and conditioner for data read and writtewn to disk.

There's no way a motherboard BIOS could control that kind of sophistication. In fact, there are pieces of software available in Russia that address the drive directly without going through the BIOS. They advise you to disable reference to the disk drives in BIOS and to unload the drivers in Windows. They operate in DOS mode normally, and I imagine they'd need DOS commands to address the output ports on the motherboard, but other than that, they use the ATA standard to operate the drive.

Each drive has to conform to the ATA standard which specifies each one of the conductors in a parallel data ribbon cable and/or the serial standard. There are 16 conductors/pins for data, and the rest of them are for addressing, control and power. Through the use of an address/control bus, the main processor on the drive can be controlled as to seeks, reads and writes, but the ATA standard also specifies codes for addressing the drive microcontroller, and I think some of those codes are commands to load/dump the processor eprom. There may be a password involved for the on-disk portion, but there is software in Russia that can bypass that apparently.

If my theory is correct, I should be able to dump the processor code and rewrite it to bypass the diagnostic routines. It may be easier than that, however, since a safe mode is available to do that. Alternately, as a technician, I could intercept the hardware signals from the processor to its peripheral chips. There are dedicated chips on board to encode/decode data and control the spindle and head positioner motors. Data which enters the processor as parallel data, leaves as serial data. That data has to be encoded to a special code for writing to disk and another chip is responsible for selecting heads, reading/writing, and data recovery from the heads after reading. The intelligence for all that is in a code split between the microprocessor's eprom and the microcode on the disk itself.

Some of this microcode is available on sites in Russia. If anyone is interested, let me know.

My problem is to get the drive past it's diagnostic stage. Or, I may have to change chips and/or the head mechanism itself. There's an excellent article on that here: hxxp://hddguru.com/content/en/articles/2006.02.17-Changing-headstack-Q-and-A/ . A search of this site may prove useful to those interested. There is an English speaking forum, although most of the good stuff seems to happen on the Russian forum.

A parting word on palindromes. Here's one that goes beyong "Madam, I'm Adam". Try, "A man, a plan, a canal -- Panama". :-)

I would appreciate that information, or the app, if you can dig it up. I read somewhere, maybe on this site, that 'debug' is still available on XP. That amazed me, so I did a file search and sure enough there it was.

The thing I like about IDA is the commentary you get about code, like for interrupts. It tells you what the interrupt does, for example, it addresses video, DMA or a controller. Those are important clues for me, if I can ever get some semblance of order out of the disassembly. Of course, I may have to get beyond my laziness and consult Ralph Brown.

Thanks. I'll look into that.


I get an outbreak of hives every time I think of Sourcer. I may be absolutely stupid, but I could never get Sourcer to work for me. It kept complaining about the 640K limit in DOS, even after I'd freed over 700k of memory for it (You can do that in DOS by freeing up areas between 640k and 1M). I kept asking myself redundantly why the designer was messing around with the 640K DOS limits when the flat memory model and extended-mode DOS were available.



that's the part I can't quite grasp. They extended the 640K limit to 1 M almost by slight of hand. When you see an address like CS = 12F7, it actually means 12F70. As you know, 0xFFFF is 64K (65535bytes) and 0xFFFFF is 1M (1048575 bytes). In DOS, there is an in-built limit due to the processor address limitations of the day. The old 8086 had a 16 bit address bus and could only address 0xFFFF bytes. However, by hanging an 0x0 on the end of the 0xFFFF, they managed to extend that to 1M.

The 1M can be thought of as 16 x 64K and the 640K DOS user limit was 10 x 64K. The segments were numbered 0x0 to 0x9 for the 640K area and 0xA to 0XF were for system use. If I start with a CS:IP of 0000:0000, I can run to 0000:FFFF in my first segment, then I have to move to 0001:0000.

Already you can see the slight of hand, because the IP section has materialized 64K from nowhere. By the time I reach CS:FFFF, I should be finished and limited to 64K. But that extra 0x0 tacked onto the end changes everything. There is no physical memory on an 8086 chip to account for it, but it works.

Adding to my confusion is the overlapping you mention. If I'm at code section 1234:0001, that can be equivalent to 3456:0001. I don't get that. Somehow, my poor brain can't grasp what is going on. When the code is written, and the data added, it is surely concrete. There is 640K of it in 10 segments. I can sense what is going on by what I have already described, but something is missing.

It's obvious that the code segment can only address 64K of memory. I have read Intel's account of the theory, but somehow I'm not smart enough to take it in. Mabe I need to go read it again.

Those two are not equivalent... 1234:0001 is lin addr 12341, while 3456:0001 is 34561. Maybe this might be of help? It's an excerpt from a book I wrote on PC programming:

Howdy,

If the microcode is in a damaged sector, how can it be repaired/replaced ?
Can it be put in another sector ?

I have a million questions about this. I have only had a terminal hard drive problem maybe twice. All the other times I was able to gain access to the information on the drive and recover most of it.
I am talking about drives 10 G or less.

Perhaps the two times I was unable to recover the drives was due to eprom failure.
There is microcode written to the disk but something has to interpet that data.
Without some type of specialized tool like an eprom programmer, I dont see how this can be done on a pc.

Woodmann

thanks for your detailed reply. The light is starting to go on. I had forgotten that the 8088/8086 processors had 20 bit address lines and could address 1 Mb. The problem, then, as you put it, seemed to be with the internal 16 bit structure like registers, etc., and they had to address 1 Mb with 16 bits.

Following up on your point about the linear addressing, 1 Mb would require 0xFFFFF bits to address it fully, but with a 16 bit register being limited to 0xFFFF bit for addressing, a system had to be developed to accomplish that. Is that correct?

Using one of your examples, with cs:ip = 2037:3080, I have to take the base of the cs at 20370, shift it left 4 bits, then add the offset as follows:

20370
3080+
------
234F0 = linear address of cs:ip (note: the formating on the forum is displacing the 3080...the 3 should be under the 0 next to the 2).

You say there are 4096 combinations of cs:ip available, and I take that to be based on the fact that only 3 digits overlap after the 4 bit left shift. So, 3 bits give you 0xFFF = 4096.

Does this apply to 32 bit flat model programming as well? I've had trouble translated the CS:IP form to the linear form, and back again. A typical CS value might be 0C34 and an IP might be 00040000. Can I shift and add as in the DOS model, or is their a different procedure?

With respect to right and left normalization, I am used to seeing in IDA the form x000:xxxx. That's what is throwing me off. In an NE program, IDA will often disassemble it with successive segments. In the DOS app I am trying to disassemble, it puts segment 0:xxxx right at the end of the disassembly.

To work with raw binary data in IDA, I would need to find the lengths of each segment and enter that in IDA. I have no idea where to look for that information at this point.

I'm just learning that process right now because I have to translate everything from either Russian or Chinese. There is software from wxx.acelabs.ru called the PC3000 that is repudetly 'the' software for this job. The most recent version for Windows comes complete with a dongle, a PCI card and a $1000+ price tag, in Russia. The North American distributor is charging $10,000, bless his little capitalist heart.

Don't ask me how this happened, but some blaggard has hacked the software. The earlier DOS-only versions used a HASP dongle, and this blaggard has replaced it with an emulator. Can you imagine the cheek? The DOS software comes with different plugins for different models of hard drive.

With this software, you can do many low-level operations, like re-writing the microcode to disk, using a good copy. The defects list is contained in what is called a G-List with a backup in a P-List, I think. There are also various modules in the service zone that affect different facets of the drive's operation. I would imagine a damaged sector in the service area would be treated much like a bad sector elsewhere. As far as I understand, however, damage to the sector itself is not an issue. It's simply code that has gotten scrambled, maybe by a power fail.

This site is a good starting point: hxxp://forum.hddguru.com/?c=2 . It has links at the top of the page to other parts of the site and you can find links throughout the forum. I have a few more if you need them. Once I get the literature I have translated, I'd be glad to share it. This site has it own low-level software, MHDD, which you can d/l. It also offers Victoria, referred to below.

'Victoria' is freeware, but I don't think you can write the modules with it. The author hints at more but does not specify. ( hxxp://hdd-911.com ) . Get out your translator. I'm using three of them because each translates Russian differently. Between the three I manage to get the gist of what is being said. At that, you have to get used to their slang. A drive is a 'screw', a platter is a 'pancake' and a motherboard is a 'parent payment'.


I'm surprised at how little information is available on this in the West. Anything I've found has the typical forum gurus exhorting people for opening up the head chambers. They claim it kills the drive but no one can tell you why. Although it's an off-topic subject on our board, I'd like to see some of our minds attack the situation.

I'm off on my own tangent since hardware is my specialty. I have traced 90% of the circuit board on my drive and I'm surprised how much sense it is making, even without the data on the chips.



The eprom programmer would only be good on the main ROM. Even at that, there are devices called EEPROMS (no typo) called electrically-erasable programmable ROMS which are used in ROM-flashing procedures. I'm sure that's what is available in the drive processors.

With respect to the microcode in the service area, it's similar to the MBR on the disk for an OS. I don't know if you can gain external access to that area using commands to the disk processor. It must be possible or the app I have would never have worked. It was a modification to existing, pre-flash firmware and had to affect either the on-board ROM, the microcode on the disk, or both. That's what I'm hoping to find out, either through IDA or getting out my trusty DOS version of ice. I wouldn't want to go flashing my good ROM with that old stuff, however, so I'm being cautious.

An interesting point is that lines of hard drives tend to use the same chips. For example, I picked up a 10 gig drive the other day that has almost exactly the same set of chips as the the 60 gig model. I can't imagine them changing the firmware all that much, although they would have to. The 10 gig has one platter and 2 heads while my 60 gig has 3 platters and 6 heads. If the manufacturers can acess the code, they must be accessing both the code on the processor and on the disk.

Howdy,

Thanks for answering my questions, it all makes some sense now .

As for why opening the hard drive is the kiss of death,
One tiny speck of dust can ruin many sectors.
Then again, when the drive is junk, open it and have at it.
Whats the worst that can happen ?

I have opened a few hard drives and actually spun the disks to check the bearings. They ran long enough for me to get what I needed off of them.

Good luck with this project. Please keep me informed.

Woodmann

the assumption is that dust is dust. What the heck is it anyway? Could be fibres, flakes of skin, cigarette debris, including tar, or a myriad of stuff. When I repaired hard drives many years ago, the drive surfaces were very smooth. Nowadays they are even smoother and they use a lubricant on them. That combination makes things tend to adhere to the surface, including heads.

The heads themselves are microfilms and fly a few thou above the surface. If you break that air cushion, and they touch down, it's pretty well game over for that track,those adjacent to it, and probably the head itself. I have taken care to remove any surface debris with 99% isopropyl alcohol but I have probably ruined the surface life by removing the lubricant. I can't get at the surfaces under the top platter either.

I watched a speck of something turn into a smear under a head. I was able to power down and clean off the stain. But it's a never ending proposition. I'm looking into creating a clean area using an air filter, and working on the drive like a surgeon with a mask, hat and some kind of lint-free clothing.

No. From 286 PM on, they are unrelated.
each segment refers to an hidden area that contains a base and a limit. When you load a segment, the processor load the related base and the limit into a 'cache' register from a table (GDT/LDT).
So linear address is segment_hidden_base+IP. If paging is off, linear addres == physical address, as for 286. Else, linear addres is virtualized, and paging translation is needed to bring it to be a physical address.

Regards,
Maximus

If I recall many years ago, the documentation on the old huge Perkin Elmer hard-drives stated that the heads rode above the surface of the platter only about 100-200 Microinches or .0001-.0002, yet I changed out many platters in a regular workplace environment and they generally lasted months before another crash - which really wasn't catastrophic - because they only held about 2.5 megs each - if possible work on the drive in some sort of a "glove box" with positive air pressure, and don't smoke, after work clean the platter with a lint free wipe and Isopropyl alcohol, inspect sideways and blow off with a canned air duster.

SiGiNT

I worked on the early 18" cartridge type that held only 5 mb. I changed heads on customer's premises, sometimes in bakeries with grain dust everywhere. There were only 1000 tracks per inch, however, and the head was driven from a linear actuator that positioned itself off a graticuled guide.

Thanks for the tips on modern devices. I've heard about the shoebox approach but I might need to get really close to the device for magnification. I'm working on devising a system with an air purifier close by, one of those using a hepa-filter. It's virtually impossible to see between platters, though.

I have employed the lint-free wipes with isopropyl (99%), and that works fairly well. I did not know about the lubricant at first and wondered why the alcohol left an oily residue. I plan to work on some old, cheap drives first.

I have read about the descriptors in an Intel manual but I still have trouble relating the selector, which I think is the CS portion (??), to the linear address. For example, changing CS:IP = 0C67:00400000 to a linear address like 800D0050, or whatever.

Sometimes, when I do a search in softice, I come up with an 8xxxxxxxx address like that and I'd like to translate it back to a CS:IP.

thanks for info.

In protectedmode there is no direct relationship between the segmented address and the linear address, unlike realmode. The segment base is obtained from the descriptor table, and the segment value is just an index into the descriptor table. The offset is then added to that segment to get the final hardware address. It's not really that difficult to understand, especially if you look at the diagrams in the Intel manual.

My own experience with repairing hard drives tells me they're actually much more durable than manufacturers would like you to think. Newer platters use a silver-colored magnetic coating which is much harder than the iron oxide media used in older drives. Unless the head lifts and gouges its edge into the platter while it's spinning at full speed ( ~75mph at the edge for a 7200rpm drive), the data on the platter won't be affected. The use of embedded servo in newer drives also increases the durability - the head alignment becomes less critical. I have a 40Gb drive that I opened to unstick the heads, and it's still running fine after more than a year, even though I didn't really take any precautions to ensure a Clean environment. Much of the dust gets flung off the platters by the force when it spins up.

thanks for your help. I'll have to get into the Intel manual again.



I have much the same feeling with regard to the paranoia in vogue regarding the fragility of these drives. If I was repairing the drives for paying customers, I would certainly take less chances.

With regard to the dust, I think it depends on what it is. Organic particles from human skin, with a carbon base, would tend to create greater problems if heated. I know from working on drives professionaly years ago that cigarette smoke was deadly. The tars adhered to the heads and I'm sure that caused the heads to fail due to interference with the aerodynamics of the heads.

Thanks for your thoughts on that.