Skip to content

MDEV-40180 Cache constant args in JSON_EQUALS and JSON_OVERLAPS #5455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
VasuBhakt wants to merge 1 commit into
base: 10.11
Choose a base branch
from
Open

MDEV-40180 Cache constant args in JSON_EQUALS and JSON_OVERLAPS #5455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 104 additions & 32 deletions sql/item_jsonfunc.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,48 +622,107 @@ bool Item_func_json_equals::val_bool()
longlong result= 0;
int arg_num= 0;
String a_tmp, b_tmp;
String *a= nullptr, *b= nullptr;
THD *thd;
json_engine_t je;
bool a_const= args[0]->const_item(), b_const= args[1]->const_item();

if ((null_value= args[0]->null_value || args[1]->null_value))
return 1;

String *a= args[0]->val_json(&a_tmp);

@gkodinov gkodinov Aug 5, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes beyond the scope of the preliminary review, so consider it optional. But I wanted to air it out anyway:

I believe that arguments to these functions broadly fall into the following categories:

  1. an SQL constant
  2. a predicate
  3. something repeating coming from a table.

This is very similar to the regular expressions case IMHO.

In case 1 you know that a constant is a constant and can pre-parse at "compile time" (e.g. at fix_length_and_dec as Item_func_regexp_match() does).

Case 2 should be optimized differently to start with: index or something. Failing that, It is highly unlikely IMHO that arguments in this case would be grouped together (sorted on one of the arguments). So you'd end up just maintaining the cache and looking for a hit. And this might even make things worse speed-wise. And shouldn't be a target of optimization. If such an optimization is to be implemented, it needs to be implemented as a source transformation of the arguments during query compilation, e.g. using some specialized form of Item_cache that can hold the parsed JSON, if there isn't one already.

Case 3 is so unlikely that I believe that detecting and optimizing that is just hard.

Lazy caching at runtime is IMHO just making things worse: for actual constants you just add extra instructions per hit (is this the same string as what's cached, if yes, then reuse; if not cache) and this will make things worse compared to the approach taken by the regexp functions.

Thus I strongly believe that caching in fix_length_and_dec() is the right phase to do this. But it needs to be done correctly wrt prepared statements. You're saying there's a bug in these. Maybe fix that instead?

if ((null_value= a == nullptr))
return 1;
String *b= args[1]->val_json(&b_tmp);
if ((null_value= b == nullptr))
return 1;

DYNAMIC_STRING a_res;
if (init_dynamic_string(&a_res, NULL, 0, 0))
{
null_value= 1;
return 1;
}
thd= current_thd;
JSON_DO_PAUSE_EXECUTION(thd, 0.0002);
je.killed_ptr= (uint32_t *) &thd->killed;

DYNAMIC_STRING b_res;
if (init_dynamic_string(&b_res, NULL, 0, 0))
/* Process First Argument */
if (a_const && a_parsed)

@grooverdan grooverdan Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a_parsed is always the same as cached_a.str != nullptr so I think a_parsed can be eliminated.

{
dynstr_free(&a_res);
null_value= 1;
return 1;
if(a_null)

@grooverdan grooverdan Jul 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

space after if

goto return_null;
}
else
{
a= args[0]->val_json(&a_tmp);

@grooverdan grooverdan Jul 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note after next rebase the null checking of a needs to be after this. 63d779c by @vuvova corrected my mistake (thank you!), lets not regress it.

if (!a)
goto set_a_null;

if (!cached_a.str)
{
if (init_dynamic_string(&cached_a, NULL, 0, 0))

@grooverdan grooverdan Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is going to contain a normalized form of a, use a->length() as the 3rd arg for the initial size.

The failure of this is a memory allocation failure. While it hasn't been done well in other examples in this file yet, the response is:

my_error(ER_OUTOFMEMORY, MYF(0), a->length());
goto return_null

As a pushed error it shouldn't return here

goto set_a_null;
}
else
{
cached_a.length= 0;

@grooverdan grooverdan Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a comment in code here about resetting string for next value.

}

thd= current_thd;
JSON_DO_PAUSE_EXECUTION(thd, 0.0002);
je.killed_ptr= (uint32_t *) &thd->killed;
if (json_normalize_engine(&je, &cached_a, a->ptr(), a->length(), a->charset()))
{
goto set_a_null;
}

if (json_normalize_engine(&je, &a_res, a->ptr(), a->length(), a->charset()))
goto return_null;
if (a_const)
{
a_null= false;

@grooverdan grooverdan Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think can set a_null unconditionally. Its only looked at under a_const ==true anyway0.

a_parsed= true;
}
}

arg_num++;
if (json_normalize_engine(&je, &b_res, b->ptr(), b->length(), b->charset()))
goto return_null;

result= strcmp(a_res.str, b_res.str) ? 0 : 1;
/* Process Second Argument */
if (b_const && b_parsed)

@grooverdan grooverdan Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comments on first arg parsing apply here too.

{
if(b_null)
goto return_null;
}
else
{
b= args[1]->val_json(&b_tmp);
if (!b)
goto set_b_null;

if (!cached_b.str)
{
if (init_dynamic_string(&cached_b, NULL, 0, 0))
goto set_b_null;
}
else
{
cached_b.length= 0;
}

if (json_normalize_engine(&je, &cached_b, b->ptr(), b->length(), b->charset()))
{
goto set_b_null;
}

if (b_const)
{
b_null= false;
b_parsed= true;
}
}

result= strcmp(cached_a.str, cached_b.str) ? 0 : 1;
goto end;

set_a_null:
if (a_const)
{
a_null= true;
a_parsed= true;
}
goto return_null;

set_b_null:
if (b_const)
{
b_null= true;
b_parsed= true;
}

return_null:
null_value= 1;

Expand All @@ -675,8 +734,6 @@ bool Item_func_json_equals::val_bool()
a= b;
report_json_error(a, &je, arg_num);
}
dynstr_free(&b_res);
dynstr_free(&a_res);
return result;
}

Expand Down Expand Up @@ -5046,17 +5103,34 @@ bool Item_func_json_overlaps::val_bool()
json_engine_t je, ve;
int result;
THD *thd;
bool b_const= args[1]->const_item();

if ((null_value= (js == nullptr) || args[0]->null_value))
return 0;

thd= current_thd;
JSON_DO_PAUSE_EXECUTION(thd, 0.0002);

if (!a2_parsed)
if (b_const && a2_parsed)
{
val= args[1]->val_json(&tmp_val);
a2_parsed= a2_constant;
val= (cached_val.is_alloced() || cached_val.length()) ? &cached_val : 0;
}
else
{
String *v= args[1]->val_json(&tmp_val);
if (v)
{
cached_val.copy(v->ptr(), v->length(), v->charset());

@grooverdan grooverdan Jul 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still copying here.

if (b_const)
{
   if (cached_val.is_allocated())
      val= &cached_val; /* or something*/
   else
   {
      val= args[1]->val_json(&cached_val);
   }
}
else
{
  something using local val= tmp_str (as local)
}

(incomplete)

val= &cached_val;
}
else
{
cached_val.length(0);
val= 0;
}

if (b_const)
a2_parsed= true;
}

if (val == 0)
Expand Down Expand Up @@ -5092,8 +5166,6 @@ bool Item_func_json_overlaps::val_bool()

bool Item_func_json_overlaps::fix_length_and_dec(THD *thd)
{
a2_constant= args[1]->const_item();
a2_parsed= FALSE;
set_maybe_null();

return Item_bool_func::fix_length_and_dec(thd);
Expand Down
30 changes: 26 additions & 4 deletions sql/item_jsonfunc.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,9 +126,21 @@ class Item_func_json_valid: public Item_bool_func

class Item_func_json_equals: public Item_bool_func
{
DYNAMIC_STRING cached_a, cached_b;
bool a_parsed, b_parsed;
bool a_null, b_null;
public:
Item_func_json_equals(THD *thd, Item *a, Item *b):
Item_bool_func(thd, a, b) {}
Item_bool_func(thd, a, b), a_parsed(false), b_parsed(false), a_null(false), b_null(false)
{
cached_a.str= 0;
cached_b.str= 0;
}
~Item_func_json_equals() override
{
dynstr_free(&cached_a);
dynstr_free(&cached_b);
}
LEX_CSTRING func_name_cstring() const override
{
static LEX_CSTRING name= {STRING_WITH_LEN("json_equals") };
Expand All @@ -138,6 +150,11 @@ class Item_func_json_equals: public Item_bool_func
Item *shallow_copy(THD *thd) const override
{ return get_item_copy<Item_func_json_equals>(thd, this); }
bool val_bool() override;
void cleanup() override
{
a_parsed= b_parsed= false;
Item_bool_func::cleanup();
}
};


Expand Down Expand Up @@ -848,18 +865,23 @@ extern bool is_json_type(const Item *item);
class Item_func_json_overlaps: public Item_bool_func
{
String tmp_js;
bool a2_constant, a2_parsed;
String tmp_val, *val;
bool a2_parsed;
String tmp_val, cached_val, *val;
public:
Item_func_json_overlaps(THD *thd, Item *a, Item *b):
Item_bool_func(thd, a, b) {}
Item_bool_func(thd, a, b), a2_parsed(false) {}
LEX_CSTRING func_name_cstring() const override
{
static LEX_CSTRING name= {STRING_WITH_LEN("json_overlaps") };
return name;
}
bool fix_length_and_dec(THD *thd) override;
bool val_bool() override;
void cleanup() override
{
a2_parsed= false;
Item_bool_func::cleanup();
}
Item *shallow_copy(THD *thd) const override
{ return get_item_copy<Item_func_json_overlaps>(thd, this); }
};
Expand Down