How do I find out how many product names in my file contain stretched letters before I clean it?

Copy the product name column and paste it into the tatweel remover, one name per line. Under the result are three numbers: «Tatweel found» tells you how many kashida characters (U+0640) the whole file holds, and «Words affected» how many words contained at least one. Those two numbers are the size of the problem, before you change anything in your file.

Published · 3 min read

Why check before cleaning

A tatweel is easy to miss, especially a single one inside a long word. Yet one is enough to make «قهوة عربية» in your file different from the «قهوة عربية» a customer types into search. The check answers practical questions: is this two products or half the catalog? Does it need a manual review or a full clean? And where did it come from? If the stretches cluster in products added by one staff member or copied from one supplier, you have found the source.

An example from a small file

Four product names, one per line: «قهوة عربية 250 غ» with one tatweel after the ba in «عربية», «تمر سكري» with two tatweels after the mim in «تمر», then «هيل حب» and «زعفران إيراني» with none.

  • Tatweel found: 3.
  • Tatweel removed: 3 (with the default «Remove every tatweel»).
  • Words affected: 2, namely «عربية» and «تمر».

The first number counts characters; the third counts words. The gap between them tells you something: 3 characters in 2 words means one word held more than one. If you saw, say, 40 characters in 5 words, the stretches are long and probably deliberate, perhaps pasted from a marketing post.

What the counters do not tell you

The tool counts words, not lines. If one product name has two stretched words, they count as two. To see exactly which lines were affected, compare the result with the original: the lines that changed are the ones that had a stretch. In the Text comparison tool, for example, you can paste both versions and see the lines that differ. The tatweel itself no longer shows in the result box because it was removed, so comparing is how you see where it was.

Steps

  1. Export the product names from your store or copy the column from your sheet.
  2. Paste them into the Arabic tatweel remover, one name per line.
  3. Note the «Tatweel found» and «Words affected» numbers. If both are zero, a message says the text has no tatweel and there is nothing more to do.
  4. Copy the clean result. Line order and count never change, so you can paste it back into the same column.

After cleaning: look for repeats

Cleaning sometimes reveals duplicate products that were hiding. Adding a fifth line to the example, «قهوة عربية 250 غ» typed with no stretch, made that clear: before cleaning the two lines differed, and afterwards they were identical, character for character. That is usually a product added twice. Run the list through the Duplicate-line remover to see the repeated lines and how often each occurs, then decide in your store which copy to delete.

Common questions

Does the tool change anything else in the product names?

No. It deletes the tatweel only; digits and units like «250 غ», letters and diacritics stay as they are.

My file has thousands of products; can it cope?

The tool was tested with 100,000 lines. Above about 4,000 lines automatic processing stops and you press «Remove tatweel» yourself.