Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use lambda instead of std::bind #1243

Open
wants to merge 1 commit into
base: fb-mysql-8.0.28
Choose a base branch
from

Conversation

rockeet
Copy link
Contributor

@rockeet rockeet commented Sep 10, 2022

std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

The lambda just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.
@facebook-github-bot
Copy link

@hermanlee has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot pushed a commit that referenced this pull request Nov 15, 2022
Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: #1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
alanliang pushed a commit to alanliang/mysql-5.6 that referenced this pull request Mar 4, 2023
Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243
GitHub Author: leipeng <peng@topling.cn>

Test Plan: Imported from GitHub, without a `Test Plan:` line.

Reviewers: mung

Reviewed By: mung

Subscribers: pgl, webscalesql-eng@fb.com

Differential Revision: https://phabricator.intern.facebook.com/D40858532

Tags: mysql80, accept2ship
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 12, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 15, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 16, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 17, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 18, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 26, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

fbshipit-source-id: 3104275

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

fbshipit-source-id: e0f9d5a
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jun 1, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

fbshipit-source-id: 3104275

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

fbshipit-source-id: e0f9d5a
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jun 14, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

fbshipit-source-id: 3104275

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

fbshipit-source-id: e0f9d5a
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jun 19, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

fbshipit-source-id: 3104275

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

fbshipit-source-id: e0f9d5a
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jun 23, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

fbshipit-source-id: f7431f7

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

fbshipit-source-id: 3104275

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

fbshipit-source-id: e0f9d5a
hermanlee pushed a commit to hermanlee/mysql-5.6 that referenced this pull request Oct 3, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee



---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376



------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
hermanlee pushed a commit to hermanlee/mysql-5.6 that referenced this pull request Oct 18, 2023
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee



---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376



------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Apr 25, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 7, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 8, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 9, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 10, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 13, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 15, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441

------------------------------------------------------------------------------

Add yields to Item_func::fix_fields

Summary:
When resolving an expression with many arguments (e.g. a long `IN (...)` list) we may not yield the scheduler in a timely fashion.

This should resolve stalls like
```
W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Conn state: acExitLevel=0 isGoverned=1 readmissionMode=0 admitCount=1 waitEvents=0xa7 acFilter=0x0
W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Collect time 748751 nsec, symbolize time 6554425 nsec, digest dc6129f5efe09474cfacb020cdc3521b
W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Stack trace for 1 thread(s) [2608144 tp_conn-0]:
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002c134bd Item_func_in::resolve_type(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002839275 Item_func::fix_fields(THD*, Item**)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002eca7d5 Item_func_in::fix_fields(THD*, Item**)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000295fd80 Query_block::setup_conds(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002ca4b77 Sql_cmd_delete::prepare_inner(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000282d295 Sql_cmd_dml::prepare(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002827053 Sql_cmd_dml::execute(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000282177d mysql_execute_command(THD*, bool, unsigned long long*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 0000000002807d26 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 00000000027f8238 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 00000000027f6d20 do_command(THD*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 00000000062dedd7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 00000000062f01f6 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000475447a pfs_spawn_thread(void*) [clone .__uniq.73903348312165179054628174044725776814] [clone .llvm.15823314186845019609]
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000009abae start_thread
W0127 03:04:04.517819 2608142 TpScheduler::checkStall:     @ 000000000012d17b __clone3
```

Differential Revision: D53241252

fbshipit-source-id: 7fee664
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 16, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 17, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 17, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 21, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 21, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request May 30, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 1, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 2, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 19, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 19, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 30, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Jul 31, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Aug 2, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
inikep pushed a commit to inikep/mysql-5.6 that referenced this pull request Aug 6, 2024
…read pool plugin

Summary:
1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type.
2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed.
3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield.
4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names).

Reviewed By: lth

Differential Revision: D27593084

---------------------------------------------------------------------------------------------

use lambda instead of std::bind (facebook#1243)

Summary:
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.

This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.

Pull Request resolved: facebook#1243

Reviewed By: lth

Differential Revision: D40858532

Pulled By: hermanlee

---------------------------------------------------------------------------------------------

Add yields to hash join iterator

Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler.

Differential Revision: D39948376

------------------------------------------------------------------------------

Add yield to union iterator

Summary:
This was detected by the stall detection (8.0.23 based callstack):
```
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]:
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000001204d0 ppoll
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000258d2db Sql_cmd_dml::execute(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000269c3e0 do_command(THD*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5395f mysql::thread_pool::TpTask::execute()
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*)
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416]
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000009ac0e start_thread
W0205 14:38:40.101111 2319640 TpScheduler.cpp:431]     @ 000000000012d1db __clone3
```

Cleaned up existing yield code to use a common default yield condition.

Reviewed By: preritj24

Differential Revision: D43071441
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants