How to move G-List bad sectors into P-List and reassign LBA

Question

I have an HDD with 10319 bad sectors(reallocated sectors), this is the SMART information of the drive; There are 3097029168 sectors, bad sectors/total sectors is 0.00000333190274945450562188570256307, I would like to keep using it until the reallocated sector count increases.

From what I gathered, the bad sectors are marked as bad and reallocated into G-List(Growing Defect List), when the operating system tries to access the sectors in G-List, the read/write requests are redirected automatically to good sectors located in a spare area inaccessible to the OS otherwise by the hard disk controller, the redirection causes delays, decreases performance.

However I know hard drives, HDD and SSD alike, are "born" with defects, because modern manufacturing technology isn't perfect, the manufacturers do a remapping before the disks are shipped, and allocated the defects into P-List(Primary Defect List), and P-List sectors do not affect performance.

I am wondering, is there a way to renumber the sectors, in such a way that the bad sectors don't have assigned addresses, so that the bad sectors are impossible to access or even be listed, effectively making the disk bad sector free, just with smaller size? Or can the G-list sectors be moved into P-list?

For example, using logical block addressing(LBA), if there are a total of 10000 sectors, the sectors with LBA's 42, 256, 365, 1024 are bad, after renumbering, sectors in LBA range 0-41 will remain unchanged, sectors in LBA range 43-255 will be sectors 42-254 (-1), sectors in LBA 257-364 will be 255-362 (-2), sectors in LBA 366-1023 will be 363-1020 (-3), sectors 1025-9999 will be 1021-9995 (-4), so that there will be 9996 continuous good sectors? There must be a way to read G-List information, is there any method to achieve this so that I will have a good disk with 3,097,018,849 continous good sectors(provided it doesn't deteriorate)?

Update: I had found this:

Once the drive layout is defined in the manufacturing process the search for defects can occur. Keep in mind that this is all occurring before the definition of the Logical Block Addressing scheme is applied. The “P” list consists of identified physical locations of “Bad” sectors on the hard disk drive. This list is then used to assist in the Logical Block Addressing of drive space for the writing of data. Here is the trick…During manufacturing, the layout of the LBA blocks occurs by shifting the LBA block sequence past the “Bad” sector. The LBA layer simply skips over the identified “Bad” location in the “P” list and continues sequentially in LBA order. The “P” list is a list of “skipped” physical locations on the hard disk drive platters.

https://www.forensicfocus.com/articles/forensic-imaging-of-hard-disk-drives-what-we-thought-we-knew-2/#:~:text=The%20physical%20geometry%20layout%20of,by%20cylinders%2C%20heads%20and%20sectors.&text=The%20%E2%80%9CP%E2%80%9D%20list%20is%20a%20list%20of%20%E2%80%9Cskipped%E2%80%9D,on%20the%20hard%20disk%20platter.

It seems I can skip the G-List sectors by moving them into P-List, and redefine Logical Block Addresses to skip P-List, how can I do it?

Edit: I had found out I can achieve this with PC-3000, but I can't afford it, does there exist any free and open source alternatives to PC-3000?

Edit1: It seems I was right, the P-List is a skip list, they don't have LBA's, the process described above happened in factory during assigning LBA's, and I had already copied all important data that can't be downloaded online off the disk successfully without errors and then already wiped my disk clean with these:

diskpart /s %userprofile%\desktop\wipedisk.txt
format G: /fs:NTFS /a:4096 /x /p:1 /y
format G: /fs:NTFS /a:4096 /x /q /y
pause

file wipedisk.txt

select disk 1
clean all
create partition primary
format fs=ntfs
assign letter=G
quit

I am serious I want to move G-list into P-list.

Update: I had just found out the technical term for my intended operation, and confirmed low-level format can do it:https://en.wikipedia.org/wiki/Sector_slipping

Answer 1

First of all: It is untrue, that primary defects impose no performance penalty. It is a matter of statistics, that the likelyhood of needen a track change during a read operation over sequential sectors increases, when the number of sectors on a track decreases.

Now: The primary defect list is created before any data is stored on the drive, it is a skip list, while the grown defect list is maintained with data already on the drive, so it is a redirect list. If you were to convert the entries for GB-Sectors into entries for PB-Sectors, you would lose that redirection information and need to reformat the drive. With a drive already in such a bad shape I assume, that the procedure of moving everything off, then reformating, then moving everything back on again would most likely create so many new GB-Sectors as to make it pointless.

With 0.00033% of sectors reallocated you see a likelyhood of again ca. 0.00033% to hit one, so you will suffer two repositions every ca. 3000 reads without taking into account any caching. I do not think, that the performance increase after caching would be appreciable.

EDIT

As requested in the comments, here is a short how-to on low-level formating: Since I don't know, if Windows can do this reliably (or even at all), the easiest way is to boot from a Linux USB stick, if you still use Windows.

• Step one: Identify the disk. This is easy for an USB disk, as by booting with it unplugged, then observing /var/log/syslog will tell you easily, which disk it is (e.g. /dev/sdb).
• Step two: Do the format. First install scsitools (sudo apt install scsitools und dpkg flavours), then just run scsiformat -t 5 /dev/sdX with the disk you identified in the first step.

Answer 2

No one has bothered to reverse engineer the firmware, because it is pointless.

The installed firmware is digitally signed with the signature of the manufacturer, so it is impossible to replace with your own version. (I don't know if it's also encrypted, but that's certainly possible.)

This is an important security measure that should reassure you that the disk you buy has not been infected with viruses on the way to your computer.

Answer 3

I am wondering, is there a way to renumber the sectors, in such a way that the bad sectors don't have assigned addresses, so that the bad sectors are impossible to access or even be listed, effectively making the disk bad sector free, just with smaller size?

That is exactly what the firmware in all drives with "reallocated sectors" is doing: The logical block address (LBA) is remapped transparently, and the OS doesn't notice any difference. In fact, the OS cannot access any of the bad sectors ("the bad sectors don't have assigned addresses"), nor the spare sectors (so if you want, those don't have assigned addresses, either).

So to the OS, it looks like the disk is bad sector free. Many OS do have provisions for bad sector lists that come from a time where the firmware didn't reallocated sectors, but with a firmware that does (and basically all modern harddrives do), those OS-level bad sector provisions have become useless.

(Though people are still frequently worried about bad sectors, and keep asking here what they need to do about them. And the answer is: nothing)

---

But the bad sectors in G-list are merely redirected,

I am not sure what your understanding of "they are merely redirected" is. They are "merely redirected" in the same way that the P-list "merely redirects": for the OS they don't exist, and can't be accessed.

the redirection decreases performance

In the sense that "a read of a sequence continuous sectors is now no longer continuous, and requires head movement", yes. Not in the sense "the redirection takes time to perform", that is negligible. So with a random access pattern, there's is no difference in performance.

And the P-list causes the same performance issue.

and the bad sectors can be directly accessed when doing full disk operations like full format, clean all..

They won't be accessed. The firmware keeps ignoring them.

And I/O to bad sectors, redirected or not, will likely make more bad sectors...

Why should it? There are no longer any bad sectors, as far as the OS is concerned. You can't make "more bad sectors" by only accessing good sectors. (And you can't even make "more bad sectors" by accessing bad sectors).

I want to move the bad sectors in G-list to P-list so that they don't get counted by smart 05 rsc,

You can't (unless you reverse engineer the firmware), and even if you could, you wouldn't get anything out of it.

and the number of total sectors displayed will decrease by the number of bad sectors,

That's not necessary: The number of spare sectors (that's never shown anywhere) already decreased when the sectors were reallocated.

Example: In the beginning your disk showed a capacity of 1000 sectors, and it had 50 spare sectors, for a total of 1050 "only visible to firmware" sectors. Over time it found 20 bad sectors and reallocated them. Now it has 20 reallocated spare sectors, 30 available spare sectors, and still shows a capacity of 1000 sectors.

Your desire to do something here comes from not understanding how the firmware works. The firmware already does what you want to do manually.

to all softwares they officially don't exist,

To all software the above 50 spare sectors, no matter if reallocated or not, already offically don't exist. All software can access exactly 1000 sectors on the hard disk.

no new bad sectors will ever be created by reading/writing to them

As the bad sectors can now no more longer be accessed within in the 1000 "reported" sectors, nobody can read or write them.

Answer 4

First:
2TB disks appeared on the market around January 2009.
20TB disks were previewed in June last year (2019).

If doubled capacity is a single generation shift, then that is a difference of four generations maybe even close to five.
Counting three-year periods; it is at least three generations.

Why bother with so old hardware?

Second:
There likely is a difference between production imperfections, and faults (bad sectors) that appears after the disk has been in use for some time.

The new faults may be caused by e.g.

• technology dependent causes,
• degradation of a magnetic surface,
• aging of the material or by
• mechanical wear

... where at least the last is likely to escalate with time and use.

In the end:
Judging the risk of further damage and the wish to retain ALL data in a safe manner makes me go with the only safe alternative;

Replace the disk and gain somewhere near ten times the data storage space.

Answer 5

Do not answer this question - it will benefit a few at the expense of many!

Technically there is no reason why there are two lists but there is a reason to show broken sectors arising during lifetime and to prevent users from deleting this important diagnostic information.

With the intended manipulation of the author you can easily lure people intent believing that a broken disk is still good.

allocated the defects into P-List(Primary Defect List), and P-List sectors do not affect performance.

This is just a pretext. Looking up a spare sector number in a list of n broken sectors costs log(n) steps in a tree structure. This applies to both lists, assuming they are both maintained in a tree structure.

This is outright bad - somebody who wants information about odometer manipulation.

In a comment you state:

But the disks were made with defects, manufacturing imperfections are inevitable, then in your logic, all disks should be broken, there would be no usable disks!

Where do I state that? Which logic implies that? Please explain!