URL Encoding and Percent-Encoding: How Query Strings, UTF-8 Bytes, and Reserved Characters Work

Why Web Addresses Require Character Encoding#

A URL is not just a plain text string; it is a structured hierarchical identifier divided into specific functional components: scheme (https://), authority (example.com), path (/api/v1/search), query string (?q=term), and fragment (#section). Certain ASCII characters serve as structural boundaries or delimiters.

For example, the question mark (?) marks the beginning of the query component, the ampersand (&) separates key-value parameters, and the equals sign (=) assigns a value to a parameter key. If a user searches for the phrase "Rock & Roll = Fun?", transmitting those raw delimiter characters would confuse the web server and break query string parsing.

RFC 3986: Reserved vs Unreserved Characters#

The authoritative specification governing URI syntax is IETF RFC 3986. The specification strictly divides the ASCII character space into three distinct categories: unreserved characters, reserved characters, and other (illegal) characters.

How Percent-Encoding Works at the Byte Level#

The transformation process follows a strict two-step pipeline. First, the character is encoded into its binary UTF-8 byte representation. Second, each byte is converted into a two-digit uppercase hexadecimal number preceded by the percent sign (%).

For example, the standard space character has an ASCII hex value of 0x20, so its percent-encoded representation is %20. For multi-byte UTF-8 characters like the Euro currency sign (€) or the rocket emoji (🚀), each individual byte is encoded consecutively.

Character: Space
ASCII Hex: 0x20
Encoded:   %20

Character: & (Ampersand)
ASCII Hex: 0x26
Encoded:   %26

Character: € (Euro Symbol)
UTF-8 Hex: 0xE2 0x82 0xAC
Encoded:   %E2%82%AC

Character: 🚀 (Rocket Emoji)
UTF-8 Hex: 0xF0 0x9F 0x9A 0x80
Encoded:   %F0%9F%9A%80

encodeURI() vs encodeURIComponent() in JavaScript#

JavaScript provides two global built-in functions for URL encoding. Using the wrong function is one of the most widespread causes of broken hyperlinks and malformed API requests.

• encodeURI(): Encodes a complete URL. It purposefully preserves protocol delimiters like "http://", "/", "?", and "&" so that the overall URL structure remains navigable.
• encodeURIComponent(): Encodes an individual component, such as a query parameter value or path segment. It encodes reserved delimiters like "&", "=", and "/" to ensure they are treated as literal text values rather than structural boundaries.

const param = "A & B / C = 100%";

// ❌ Incorrect: encodeURI does not encode '&', '=', or '/'
const badUrl = "https://api.example.com/search?q=" + encodeURI(param);
// Result: https://api.example.com/search?q=A%20&%20B%20/%20C%20=%20100%25
// Server sees separate params: q="A ", B="", C=" 100%"

// ✅ Correct: encodeURIComponent encodes all delimiters safely
const goodUrl = "https://api.example.com/search?q=" + encodeURIComponent(param);
// Result: https://api.example.com/search?q=A%20%26%20B%20%2F%20C%20%3D%20100%25
// Server correctly receives one single parameter value: "A & B / C = 100%"

Common Query String Pitfalls: Spaces and the Plus (+) Sign#

A common point of confusion arises from the difference between the RFC 3986 URI standard and the HTML application/x-www-form-urlencoded form submission standard.

Under HTML form submission standards (historically used by web browsers submitting POST/GET forms), space characters were converted into plus signs (+). However, in modern REST APIs and RFC 3986 paths, spaces must be encoded as %20. If your server receives a literal plus character (+), decoding it with a form parser might inadvertently turn it into a space, causing issues with base64 strings or math expressions.

Security Considerations: Parameter Pollution and Injection#

Failing to encode user-supplied values before concatenating them into URLs exposes applications to significant security hazards:

• HTTP Parameter Pollution (HPP): Unencoded ampersands allow malicious users to inject additional parameters into backend API queries.
• Open Redirect Exploits: Maliciously constructed redirect URLs containing unencoded control characters can bypass naive hostname validation filters.
• Cross-Site Scripting (XSS): Reflecting unencoded URL parameters directly into server-rendered HTML can execute arbitrary JavaScript in the victim’s browser.

Frequently Asked Questions about URL Encoding#

Here are direct answers to common developer questions regarding URL encoding standards:

• Should I encode the entire URL or only parameter values? Only encode individual query parameter values and path variables using encodeURIComponent. Never pass a full URL string to encodeURIComponent.
• Are percent-encoded characters case-sensitive? RFC 3986 recommends uppercase hexadecimal digits (%2A rather than %2a), and modern parsers treat them equivalently.
• How can I test URL encoding safely? You can use Softnag’s online URL Encoder / Decoder to inspect byte conversions locally in your browser with zero server uploads.