ToolSahla, Tools Sahla

Encoding an Arabic word to Base64 with another tool gave an error, or odd symbols when decoded. Why?

Because the browser shortcut btoa only accepts characters that fit in one byte, and Arabic letters do not, so it either fails or the text gets mangled somewhere along the way. This tool first turns the text into UTF-8 bytes, two per Arabic letter, then encodes those bytes, and reverses both steps when decoding, so the word comes back unchanged.

Published · 3 min read

Base64 encodes bytes, not letters

Base64 knows nothing about letters. It takes a run of bytes and writes them with sixty-four symbols. So before any encoding, text has to become bytes, and the question is how. The agreed way today is UTF-8, where an Arabic letter takes two bytes and a Latin letter one.

btoa, the oldest quick method in browsers, does not convert to UTF-8. It assumes every character is one byte. We tried it on مرحبا (hello) and it refused with 'Invalid character', because Arabic letters do not fit in one byte. Some tools work around that in various ways, succeed at encoding, and leave a problem that shows up on decoding.

The odd symbols on decoding

The other side of the problem appears when the encoding is right and the decoding is old-fashioned. We took the correct encoding of مرحبا and decoded it with atob, btoa's old partner, and got this instead of the word:

Ùرحبا

The ten bytes are correct, but each one was read as a separate character instead of every two being read as one Arabic letter. If you see symbols like Ù and Ø in text that should be Arabic, that is usually the sign of this mistake.

What this tool does

We encoded مرحبا and got:

2YXYsdit2KjYpw==

The tool reported 10 bytes encoded, for five letters. Decoding the output brought back مرحبا unchanged. We also tried a phrase with digits and spaces, سعر 45 ريال (price 45 riyals), which came to 18 bytes: seven Arabic letters at two bytes each, then two digits and two spaces at one byte each.

  1. Open the Base64 encoder/decoder tool and choose 'Encode'.
  2. Type or paste the Arabic text.
  3. Copy the output. To check, choose 'Decode' and paste it: your text should come back letter for letter.

Why the output is longer than you expect

Every 3 bytes become 4 symbols in Base64. With Arabic the byte count doubles first, so a five-letter word became 16 symbols including the two padding signs. That is normal, not an error, and it means a length-limited field in some system can fill up faster with Arabic than with English.

A reminder: this is not encryption

Anyone can decode the output in a second, as we just did. If the aim is to hide a password or customer detail, Base64 does not do that job. And if the Arabic text is going into a link rather than into Base64, the right tool is the URL encoder/decoder.

Common questions

Does the tool handle emoji?

Yes, because it goes through UTF-8 as well. An emoji usually takes four bytes.

Base64 from an old system looks wrong in this tool. Why?

If the bytes are not valid UTF-8, the tool shows no garbled text; it says the result is not text and offers to download the bytes. The original text may be in a legacy encoding.

Does the encoding itself differ between tools?

For the same text in UTF-8 the output should be identical. Differences come from how letters are turned into bytes before encoding.