My category pages repeat under ?sort= and ?filter= links. Should I block them in robots.txt, and how do I write the rule so nothing slips through?
Blocking sort and filter links can save crawl effort on a large catalogue, as long as the pages you want found are reachable without them. But the preset rule /*?sort= only matches when sort comes straight after the ?, so a link like /category/oud?page=2&sort=price slips past it. Add a second rule, /*&sort=, to catch sort in any other position, and test each form in the tool.
Where the link slips through
The pattern /*?sort= means: anything, then ?sort=. In /category/oud?sort=price, sort is the first parameter, so the rule matches. In /category/oud?page=2&sort=price, page comes after the ?, and the text ?sort= appears nowhere, so the rule does not match and the link stays allowed. That is exactly what the tester in the Robots.txt builder showed.
Parameter order is set by your store, your theme or even the link a visitor clicked, so never assume sort always comes first.
Three patterns compared
- /*?sort=: matches ?sort=price, but not ?page=2&sort=price.
- /*&sort=: matches ?page=2&sort=price, that is, sort when it is not the first parameter.
- /*sort=: matches both, but is broad: it matches any path containing sort= anywhere, even /category/sort=x. /resort-wear/ did not match, because it has no = sign.
Combining the first two is more precise than the third, because it only catches the real parameter. The same idea applies to filter:
User-agent: *
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?filter=
Disallow: /*&filter=
Steps
- In the tool you can start from the 'Block URL parameters' preset, which writes /*?sort=, /*?filter= and /*?session=.
- Under each, add its twin with & in place of ?.
- In 'Test a path', try all three shapes: parameter first, parameter second, and the category page with no parameters.
- Make sure the category page itself is still allowed, since that is what you want in search.
Is blocking the right fix at all?
The tool itself warns on that preset: a canonical tag is often the better fix, because a blocked URL cannot be read and consolidated with its main page. It also warns that * and $ are defined in RFC 9309, but simpler crawlers treat them as ordinary characters, in which case the rule blocks nothing for them. In our test a warning appeared for every line using a pattern.
On a small store with a few dozen products, sort links may not deserve a rule at all. On a large catalogue where filtering produces thousands of combinations, they may. Keep sort links out of the sitemap you build with the XML sitemap generator, and never block a URL you listed there. On Salla or Zid you usually do not control the file, so read what the platform publishes and test it the same way.
Common questions
Does the tool understand parameter order?
No. It matches the text as written, which is why you need a pattern for each position.
Does the rule treat Sort and sort differently?
Yes, paths are case-sensitive.
Does blocking remove links already indexed?
No. Blocking stops crawling, not indexing, and the links can stay in results without a description.