executing-jmeter-tests-in-an-azure-pipeline

[giscus[bot]]

executing-jmeter-tests-in-an-azure-pipeline

Visual Studio Load Testing tools have been deprecated, along with Cloud Load Testing. In this post I investigate how to use JMeter as a load testing alternative.

https://colinsalmcorner.com/executing-jmeter-tests-in-an-azure-pipeline

[pabrams]

How can I get my build or release to fail when there are http 404s and such in the test results? No matter what I do, Azure devOps always says everything passes, when it doesn't.

[colindembovsky]

Hey @pabrams - I'm not sure why it's not failing. Can you share logs? When you run the tests locally, do you get failures?

[colindembovsky]

Hi @pabrams - looks like the return code for the requests is 200 which is success. Could be that the page you're testing isn't returning the right HTTP status code?

[pabrams]

What about all the 404s? Here's an excerpt from the above, for example:
org.apache.jmeter.protocol.http.parser.CssParser
2021-09-23 16:15:13,044 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:15:13,077 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:16:19,578 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:16:19,630 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:17:26,529 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:17:27,448 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:18:33,254 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:18:33,334 INFO o.a.j.r.Summariser: summary + 73 in 00:04:28 = 0.3/s Avg: 6439 Min: 42 Max: 67268 Err: 7 (9.59%) Active: 2 Started: 2 Finished: 0
2021-09-23 16:18:33,383 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404
2021-09-23 16:19:39,476 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 404

[da1-github]

I may be misunderstanding, but are you certain this works from release pipelines as you mention?
I have it generating jMeter output as shown below, but per https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/publish-pipeline-artifact?view=azure-devops , publishing artifact content from Release pipelines is not permitted.

Can you elaborate on downloading and viewing the results from a release pipeline in Azure?

[colindembovsky]

Hi @da1-github What I really mean with "release pipeline" is a YML pipeline. The old "classic" (GUI-based) Releases do have some limitations, and you should be upgrading from GUI to YML. If you click on a Task in the Release UI, you should see a "View YML" link that will help you convert from GUI to YML.