Why doesn't the tool treat a product name written with stretched letters as a repeat of the same name without them?

Where stretched letters come from

Some sellers use the tatweel to make a product name look wider in an Instagram post or an ad headline, or to justify a line in a design. Later the names get copied from posts into a stock list or product file, and the same list ends up holding a plain 'white musk' in Arabic, a stretched copy of it, and a third copy with vowel marks.

A worked example

Note that the option handles vowel marks and tatweel together, hence its name: the kasra and the stretching characters are all set aside for comparison, while the letters themselves stay as they are.

Steps

1. Open the Duplicate-line remover and paste the list.
2. Tick 'Arabic: ignore diacritics and tatweel when matching'.
3. Check 'Lines that repeated' to see which names were grouped and in which form each was kept.

The kept line is still stretched?

The option applies to matching only and never edits a line. If the stretched copy is first in the list, it is the one that stays, stretching and all. We tried a two-line list with the stretched name first and the plain one second: one line was removed and the stretched one stayed.

If you want clean names with no stretching, remove the tatweel from the whole list with the Arabic tatweel remover before removing duplicates. On that same list it removes three tatweel characters, the two lines become identical, and the duplicate remover then catches the repeat without any option turned on.

What the option does not unify

• Alef forms: we tried عطر أصيل and عطر اصيل with the option on, and both lines stayed.
• Ya and alef maqsura, or ta marbuta and ha.
• Arabic-Indic and Western digits.

These are all different letters, not marks placed on letters, and the tool never changes letters. If your list mixes them, make the spelling consistent by hand where it differs before removing repeats.