I'm adding search over Quranic verses on my site; which diacritic groups should I strip from the search copy while keeping the displayed text intact?

Why two copies?

Vocalized Quranic text is not text to clean up. The marks are part of its orthography, and the tool page itself advises against stripping them from Quranic text. Your site search, however, compares characters one by one, and users type «بسم الله الرحمن الرحيم» with no marks. The usual approach is to store two fields per verse: the full text for display and a stripped copy for matching. The engine searches the second and shows the first, so nobody ever sees the stripped copy.

What the default groups give you

The vocalized basmala «بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ» with the default groups gave «بسم الله الرحمٰن الرحيم», with 15 marks removed: 10 harakat, 3 shadda and 2 sukun. Look at «الرحمٰن», though: the superscript alef is still there, because its group is off by default. The tool flags it with a note that one mark was kept because its group is off.

With «Superscript (dagger) alef» turned on, the result became «بسم الله الرحمن الرحيم», with 16 marks removed. That is the copy that matches what users type.

Uthmani sources: a different kind of sukun

Digital mushaf text in the Uthmani script sometimes writes the sukun with a small sign from the Quranic annotation group rather than the usual sukun. Running «قُلۡ هُوَ ٱللَّهُ أَحَدٌ ۝١» with the default groups plus the superscript alef gave «قلۡ» with its sign still over the lam, so it will not match «قل». The tool reported two marks kept because their group was off.

With «Quranic annotation and pause signs» turned on, it became «قل هو ٱلله أحد ١»: the sukun sign went, and so did the end-of-ayah sign, leaving its number. That does no harm in a hidden search copy, and it is one more reason never to use that copy for display. With a source in ordinary modern spelling, the default groups plus the superscript alef are usually enough.

What the tool does not solve: alef wasla and spelling

In the example above «ٱلله» kept its alef wasla. That is a letter in its own right, not a mark, so the tool leaves it alone, and «ٱلله» will not match «الله». Likewise «ٱلْعَٰلَمِينَ» becomes «ٱلعلمين» with no alef once the superscript alef goes, while users type «العالمين». These are spelling differences, not diacritics. Fixing them takes a normalization step in your own search code, such as replacing the alef wasla with a plain alef, plus your own decision on the special orthography words. The tool does not fold letters and does not claim to.

Steps

1. Copy the verses from your trusted source, one per line, and keep them unchanged for display.
2. Paste a copy into the Arabic diacritic remover.
3. Keep tanwin, harakat, shadda and sukun on, and turn on «Superscript (dagger) alef». If the source is Uthmani, also turn on «Quranic annotation and pause signs». Leave «Other combining marks» off.
4. Check the note about kept marks under the result, then copy the result and store it as a second field for search. Line order never changes, so each stripped line stays aligned with its verse.

Notes

• If the text contains stretched letters, this tool counts them but does not remove them. Use the Arabic tatweel remover on the search copy.
• Apply the same stripping to what users type in the search box before matching, since some users type partial vowel marks.