[202311] Stabilize PFC watchdog test

[bingwang-ms]

Description of PR

Summary:
This PR is to cherry-pick #14425 to 202311 branch.
Changes include

1. Do not use multi-process to send PFC pause frames if there is only one interface to be paused
   This is because I didn't see improvement using multiple processes to send PFC pause frames when there is only 1 port to be paused. And the multi-processing caused test flakiness in test_pfcwd_multi_port
2. Check PFCWD stats after triggering PFC pause on leaf fanout
   Before this change, there was a hard-coded 5 seconds delay. But on some platform (e.g, Mellanox) more time is needed until PFC watchdog is triggered.

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Back port request

• 202012
• 202205
• 202305
• 202311
• 202405

Approach

What is the motivation for this PR?

This PR is to stabilize PFC watchdog test.

How did you do it?

1. Do not use multi-process to send PFC pause frames if there is only one interface to be paused
2. Check PFCWD stats after triggering PFC pause on leaf fanout

How did you verify/test it?

The change is verified on a Mellanox testbed. All test cases in pfcwd/test_pfcwd_function.py are consistently passing now.

collected 5 items                                                                                                                                                                                     

pfcwd/test_pfcwd_function.py::TestPfcwdFunc::test_pfcwd_actions[str-msn2700a1-0 ^H ^HPASSED                                                                                                        [ 20%]
pfcwd/test_pfcwd_function.py::TestPfcwdFunc::test_pfcwd_multi_port[str-msn2700a1-03]  ^H ^HPASSED                                                                                                     [ 40%]
pfcwd/test_pfcwd_function.py::TestPfcwdFunc::test_pfcwd_mmu_change[str-msn2700a1-03]  ^HPASSED                                                                                                     [ 60%]
pfcwd/test_pfcwd_function.py::TestPfcwdFunc::test_pfcwd_port_toggle[str-msn2700a1-03]  ^H ^HPASSED                                                                                                    [ 80%]
pfcwd/test_pfcwd_function.py::TestPfcwdFunc::test_pfcwd_no_traffic[str-msn2700a1-03] SKIPPED (This test is applicable only for cisco-8000)                                                      [100%]

Any platform specific information?

No.

Supported testbed topology if it's a new test case?

Not a new test case.

Documentation

[mssonicbld]

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/pfcwd/files/pfcwd_helper.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Passed
flake8...............................................(no files to check)Skipped
check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
   the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
   docker container.
2. Ensure that the pre-commit package is installed:

sudo pip install pre-commit

3. Go to repository root folder
4. Install the pre-commit hooks:

pre-commit install

5. Use pre-commit to check staged file:

pre-commit

6. Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

[yanmo96]

LGTM