Repository navigation
Python: fix(redis): escape RediSearch query syntax in memory search text - #10039
Open
Fish (wxhking) wants to merge 2 commits into
Open
Fish (wxhking) wants to merge 2 commits into
Fish (wxhking) wants to merge 2 commits into
Conversation
Fish (wxhking)
temporarily deployed
to
github-app-auth
November 3, 2026 13:11 — with
GitHub Actions
Inactive
Fish (wxhking)
requested review from
Darren Cohen (dargilco) and
sophia-ramsey
as code owners
November 3, 2026 13:11
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Escaping the text before embedding makes hybrid search use different embedding input from stored content, potentially changing retrieval rankings; preserve the original embedding input and add a regression assertion.
1 open issue
What changed in this PR
Addresses #10002 by escaping RediSearch syntax in user messages used for Redis memory retrieval, aiming to prevent punctuation from causing syntax errors that abort agent runs.
Changes:
- Adds a search-text escaper shared by text and hybrid queries, leaving spaces unescaped.
- Adds regression tests for punctuation-heavy messages and escaped text passed to hybrid queries.
| File | Description |
|---|---|
python/packages/redis/tests/test_context_provider_edges.py |
Adds text-query escaping cases and a hybrid-query regression test. |
python/packages/redis/agent_framework_redis/_context_provider.py |
Escapes search text before constructing Redis queries. |
Review effort: Balanced
Author
|
@microsoft-github-policy-service agree |
RedisContextProvider passed the user's message straight into TextQuery / AggregateHybridQuery as the search text. RediSearch reads several punctuation marks as query syntax there — '|' unions terms, '?' is a fuzzy marker, backticks quote exact phrases — so ordinary messages like 'ps aux | grep python' or 'Why ?' reached the engine as malformed grammar (e.g. '@content:(ps | aux | | | grep | python)') and raised a syntax error that failed the whole agent run; the model was never called. Escape the search text with a TokenEscaper configured for the RediSearch syntax characters before building either query. Spaces stay unescaped, so the terms keep tokenizing and BM25 scoring works as before. Tag filter values were already escaped through _vector_store's escaper; only the free-text path was exposed. Fixes microsoft#10002
Fish (wxhking)
force-pushed
the
fix/redis-search-text-escaping
branch
from
November 8, 2026 02:17
4dfe2ea to
37bb445
Compare
redisvl's TextQuery escapes every term itself: _tokenize_and_escape_query
runs the default TokenEscaper per token, and that escaper also escapes the
backslash a hand-escaped term already carries. The pre-escaped '|' from the
TokenEscaper built here therefore landed as a literal backslash-pipe that
was re-escaped again — the produced query still carried the bare '|', so
'ps aux | grep python' stayed broken, and tokens like 'v1.2' picked up a
backslash that changed BM25 matching.
Instead of escaping, strip the syntax characters redisvl's escaper does not
cover ('|', '?', backtick) and rejoin the remaining tokens as plain terms;
TextQuery then tokenizes and escapes each of them again as designed.
Suggested-by: Aman Agnihotri <163747435+its-amann@users.noreply.github.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Fixes #10002
Summary
RedisContextProvider._redis_searchpassed the user's message straight intoTextQuery/AggregateHybridQueryas the search text. RediSearch reads several punctuation marks as query syntax there —|unions terms,?is a fuzzy marker, backticks quote exact phrases — so ordinary messages likeps aux | grep python,Why ?or a markdown table reached the engine as malformed grammar (e.g.@content:(ps | aux | | | grep | python)) and raisedIntegrationInvalidRequestException: Redis text search failed: ... Syntax error, failing the whole agent run before the model was ever called.The search text is plain prose, not a query: escape the RediSearch syntax characters with a
TokenEscaperconfigured for them before building either query._vector_store's escaper; only the free-text path was exposedTextQuery(text='ps aux | grep python').query_string()yields'@content:(ps | aux | | | grep | python)'before the fix; after it the terms are escaped and the broken| |shape is goneTest plan
python/packages/redis/tests/test_context_provider_edges.py: all five failing messages from the issue are asserted to produce an escaped query string (no unescaped| ?/| |shapes, escaped chars present), for both the text and hybrid-vector paths — 7 new casespytest tests -m 'not integration'inpython/packages/redis— 337 passed, 58 deselected (integration), no new failures