What's the difference between removing diacritics and normalizing hamza forms and ta marbuta, and which one does my store search need?
They are two different operations. Removing diacritics deletes harakat, tanwin, shadda and sukun and leaves the letters alone: «أَحْمَد» becomes «أحمد» with its hamza, not «احمد». Normalization changes the letters themselves: أ, إ and آ become ا, and ta marbuta becomes ha. A search that must find «أحمد» when a customer types «احمد» needs normalization on top of diacritic removal, and this tool does not normalize.
Why the tool leaves hamza alone
To a computer, «أ» is not an alef with a mark on it; it is a letter in its own right with its own code, like «ب» or «ت». The same goes for إ, آ, ؤ and ئ. The diacritic remover deletes marks only, so it never touches these letters. Running «أَحْمَد» removed 3 marks, two fathas and a sukun, and the hamza stayed: «أحمد».
Three more words, «إِسْلَام آمِنَة مَدْرَسَة», came out as «إسلام آمنة مدرسة». Nine marks went, while the hamza under the alef, the madda in «آمنة» and the ta marbuta in «آمنة» and «مدرسة» all stayed. That is by design: the tool page says plainly that it does not fold alef, ya or ta marbuta forms.
So what is normalization?
Normalization is a decision to treat different letters as one for search. Its best-known Arabic rules:
- أ, إ, آ and the alef wasla ٱ all become ا.
- Ta marbuta ة becomes ha ه.
- Alef maksura ى becomes ya ي.
These rules change spelling. «مدرسة» becomes «مدرسه», which is a spelling mistake if a reader sees it. That is why normalization is normally applied to a hidden search copy and to what the customer types into the search box, never to the text people read.
Which one does your store search need?
Usually both, in different places. Customers rarely type vowel marks, and many skip hamza too. If your product names are vocalized, removing diacritics alone lets «عُود» match «عود». But a customer typing «اسوارة» will not find «أسوارة» unless the search system folds hamza forms.
- Diacritic removal: you can do it yourself on product names before uploading, by pasting them into the Arabic diacritic remover and copying the result.
- Normalization: it belongs in your platform's search settings or your site's code, because it has to apply both to the stored text and to what the customer types. Check what your platform offers; we cannot vouch for how any particular one behaves.
Where normalization lives in our tools
One of our tools offers it: the URL slug generator, because a link is a place where «إدارة» and «ادارة» really should be one address. It has alef, ta marbuta and alef maksura folding options, all off by default. Running «أَحْمَد مَدْرَسَة» through it gave «أحمد-مدرسة» without folding and «احمد-مدرسه» with alef and ta marbuta folding on. It builds links with hyphens between words, though, so it is not a tool for normalizing ordinary text.
Common questions
Is there a group in the diacritic tool that removes hamza?
The «Other combining marks» group removes a hamza only when it was typed as a separate mark over a plain alef, which is rare in text typed on a keyboard. The ready-made letter «أ» is never touched.
Does the tool remove tatweel as part of normalizing?
No. Tatweel is neither a diacritic nor normalization, and this tool only counts it. A separate tool removes it.