-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: Run test in parallel #744
Comments
You can do this with multiple process instances? |
Yep, that would be one solution. Another solution is to use Newman as a library in a simple script. We have an example for parallel collection runs here: https://github.com/postmanlabs/newman/blob/896a7b61955ee59bc8bfcfa144e7c5aa3639194e/examples/parallel-collection-runs.js All you need to change in this script is the options, where you would want to specify different options for each run (each containing a different folder name) |
Is one of both solutions possible to publish combined results (lint, json)? |
Yes, but both involve writing some aggregation logic. For the first approach, you can use a JSON reporter, and aggregate results from the reports when all the parallel runs are complete. |
@czardoz i'm running this script with 10 parallelCollectionRuns on newman 3.3.0 on linux and notice the following behaviour:
anybody else experience this? If you need specs / further details please let me know. Cheers, Simon |
Just to add my two cents - Newman has this behaviour of growing memory usage owing to the amount of details it currently needs to hold on in memory. Down the line, we need to possibly run Newman in a mode where it flushes out results to disk and not store in memory. Until then, parallel or no parallel, long running and repeated requests will have this issue. |
Please provide the parallel run feature to the newman as this is the most useful feature when some has the some huge amount of test cases........ |
@shamasis please consider this feature into the roadmap of Newman. I would like to point out the parallel execution will be really useful in the context of be able to speedup tests and have faster feedbacks (NOT for load testing purpose). |
I'm voting for parallel run and it's gonna be fantastic useful. |
Hello, thanks @flash286 for your message even if I have a little concern: I think the overall idea and vision of Postman is not to be also a Load Test tool, for this you have many other different tools and option. I see Postman as an API functional validation tool. Nothing to do with load testing. Thanks! |
This is being tracked in main postman-app-support tracker. This feature is needed, but not exactly in the way we are thinking of. We would need to think more on this. 🤔 |
@shamasis can you post the link to this "main postman-app-support tracker"? |
Please review this PR and post your thoughts if you're still interested in concurrency: postmanlabs/postman-runtime#867 |
Please let me know how to clear the newman runner instance memory once the run is completed. Currently, i'm updating the environment variables from nodejs script to environment file and then passing it to newman runner. This is working for the first newman run instance. But when again update environment file with new values and then calling newman run 2nd time, it's still running with old environment values (NOT taking newly updated values) , whereas when we open the environment file, could see that values are really updated inside it. Can anyone please let me know the solution for this - or how to clear the neman run instance memory whenever the run is completed, such that it read the environment file with same name again but with updated values. |
@czardoz, what would I add to this script to specify an environment, a data file, and a certain number of iterations to run? |
Sadly the webservice I'm testing has some very slow calls. My test suite runs 30min+ when I wait for responses. When I'd be able to run test in parallel it would be much faster! Running folders in parallel would be enough.
The text was updated successfully, but these errors were encountered: