Skip to content

Intergrate MultiThreadedWorkQueue to execute program ops #35356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 24 commits into from
Sep 17, 2021
Merged

Intergrate MultiThreadedWorkQueue to execute program ops #35356

merged 24 commits into from
Sep 17, 2021

Conversation

Aurelius84
Copy link
Contributor

@Aurelius84 Aurelius84 commented Sep 1, 2021 •
edited
Loading

PR types

New features

PR changes

Others

Describe

Intergrate MultiThreadedWorkQueue to execute program op

从PTB LM 模型 profile 结果来看（007机器，V100 16G）：

  • develop分支：25 ms 左右 （commitID: 652e655fe0a7df956）
  • 此PR： 26 ms 左右（3.8%即 3~4%）
  • 主要开销来源：
    • QueueGroup->WaitEmpty() （依赖组件）
    • ExecuteInstruction的变量析构
    • 两个Vector<std::atmoic> 的构建开销
    • 唤起线程和AddTask的累积开销（依赖组件）

image

what's next?

  • Add schedule strategy to replace NextInstruction.all_next_ops .

@paddle-bot-old
Copy link

paddle-bot-old bot commented Sep 1, 2021

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Sep 2, 2021
@PaddlePaddle PaddlePaddle unlocked this conversation Sep 2, 2021
phlrain
phlrain reviewed Sep 2, 2021
View reviewed changes
@Aurelius84 Aurelius84 requested a review from phlrain September 8, 2021 10:31
wanghuancoder
wanghuancoder reviewed Sep 10, 2021
View reviewed changes
wanghuancoder
wanghuancoder reviewed Sep 17, 2021
View reviewed changes
Copy link
Contributor

@wanghuancoder wanghuancoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有个想法，但暂时没必要修：感觉多线程包裹的层数太多了，从InterpreterCore，到AsyncWorkQueue到WorkQueueGroup到SinggleThread和MultiThread，已经很难看出我们的设计（单线程跑LaunchKernel，多线程跑CPU OP）了。

paddle/fluid/framework/new_executor/interpretercore.h

InterpreterCoreGarbageCollector gc_;
std::vector<paddle::platform::DeviceEvent> gc_event_;
std::unique_ptr<WorkQueueGroup> group_thread_pool_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

group_thread_pool_是不是不需要了？

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我后面提PR 改掉

wanghuancoder
wanghuancoder approved these changes Sep 17, 2021
View reviewed changes
Copy link
Contributor

@wanghuancoder wanghuancoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wanghuancoder wanghuancoder merged commit a087119 into PaddlePaddle:develop Sep 17, 2021
AnnaTrainingG pushed a commit to AnnaTrainingG/Paddle that referenced this pull request Sep 29, 2021
@Aurelius84 @AnnaTrainingG
Intergrate MultiThreadedWorkQueue to execute program ops (PaddlePaddl…
e1ae180 
…e#35356)

* format code

* format interface

* polish interface

* Remove std::memory_order

* modify into SpinLock

* remove fetch_context_pool_

* fix comment

* modify into WorkQueueGroup

* refine code

* fix pointer

* fix paddle_enforce

* split into AsyncWorkQueue

* polish code

* specify std::memory_relax

* fix atomic fetch_sub

* fix num_thread
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wanghuancoder wanghuancoder wanghuancoder approved these changes

@phlrain phlrain Awaiting requested review from phlrain

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Aurelius84 @wanghuancoder @phlrain