Robustness Testing - Hallo gues welcome to my blog, you can read this article with title Robustness Testing, Happy reading

IMPORTANT, MUST BE READ... : Robustness Testing
Title : Robustness Testing

Robustness Testing

IMPORTANT, MUST BE READ... I guide maintain been doing inquiry inward the surface area of robustness testing for many years, too i time inward a piece I guide maintain to explicate how that approach to testing fits into the bigger umbrella of error injection too related ideas. Here's a summary of typical approaches (there are many variations too extensions beyond these every bit yous mightiness imagine).  At the destination is a description of the robustness testing piece of job my grouping has been doing over many years.

Mutation Testing:
Goal: Evaluate coverage/effectiveness of an existing examine suite. (Also known every bit "bebugging.")
Approach: Modify System nether Test (SuT) amongst a hypothetical põrnikas too run into if an existing examine suite finds it.
Narrative: I guide maintain a examine suite. I wonder how thorough it is?  Let me pose a põrnikas (mutation) into my code too run into if my examine suite finds it. If it finds all the mutations I insert, mayhap my examine suite is thorough.
Fault Model: Source code põrnikas that is undetected past times testing
Strengths: Can discover problems fifty-fifty if code was already 100% branch-covered inward examine suite (e.g., mutate a comparing to > instead of >= to run into if examine suite exercises the equality instance for that comparison)
Limitations: Requires an existing examine suite (but, tin combine amongst automated examine generation to do additional tests automatically). Effectiveness heavily depends on inserting realistic mutations, which is non necessarily too therefore easy.

Classical Fault Injection Testing:
Goal: Determine robustness of SuT when its code or information is corrupted.
Approach: Corrupt the binary icon of the SuT code or corrupt information during run-time to run into if organisation crashes, is unsafe, or tolerates the fault.
Narrative: I guide maintain a running system. I wonder what happens if I flip a chip inward the information -- does the organisation crash?  I wonder what happens if I corrupt the compiled code -- does the organisation human activity inward an dangerous way?
Fault Model: Hardware chip flip or software-based retentiveness corruption.
Strengths: Can discover realistic failures caused past times single-event upsets, hardware faults, too software defects that corrupt computational state.
Limitations: The error model is retentiveness too plan bit-level corruption. Fault injection testing is useful for high-integrity systems deployed inward large book (they guide maintain to locomote fifty-fifty really infrequent faults), too essential for aviation too infinite systems that volition run into high rates of unmarried lawsuit upsets.  But it is arguably a chip excessive for non-safety-critical applications.

Robustness Testing:
Goal: Determine robustness of SuT when it is fed particular or odd values.
Approach: Corrupt the inputs to the SuT during run-time to run into if organisation crashes, is unsafe, or tolerates the fault.
Narrative: I guide maintain a running organisation or subsystem. I wonder what happens if the inputs from other components or sensors guide maintain garbage, unusual, or random values -- does the organisation crash or human activity inward an dangerous way?
Fault Model: Some other module than the SuT has a põrnikas that results inward particular information existence sent to the SuT.
Strengths: Can discover realistic failures caused past times probable run-time faults such every bit nothing pointers, NaNs (Not-a-Number floating signal values), corrupted input data, too inward full general faults inward modules that are non the SuT, but rather other software or sensors introduce inward the organisation that mightiness guide maintain bugs that generate particular data.
Limitations: The error model is to a greater extent than oftentimes than non that another slice of software has a põrnikas too that põrnikas volition generate bad information that kills the SuT.  You guide maintain determine how probable that is too whether it's OK inward such a instance for the SuT to misbehave. We guide maintain constitute many situations inward which such examine results are important, fifty-fifty inward systems that are non security critical.

Fuzzing:
H5N1 classical cast of robustness testing is "fuzzing," inward which random inputs are tossed into a organisation to run into what happens rather than carefully selected specific input values. My inquiry group's piece of job centers on finding efficient ways to do robustness testing too therefore that fewer tests are needed to discover system-killer values.

Ballista:
The Ballista project pioneered efficient robustness testing inward the tardily 1990s, too is yet active today on stress testing robots too autonomous vehicles.

Two fundamental ideas of Ballista are:

• Have a lexicon of interesting particular values too therefore yous don't guide maintain to stumble onto them past times risk (e.g., simply elbow grease a NULL pointer direct out rather than hold off for a random divulge generator to hap to generate a null value every bit a fuzzing input value)
• Make it slow to generate tests past times basing that lexicon on the information types taken past times a business office telephone band instead of the business office existence performed.  So nosotros don't attention if it is a retentiveness management business office or a file write existence tested - nosotros simply enjoin for instance that if it's a retentiveness pointer, let's elbow grease NULL every bit an input value to the function. This gets us first-class scalability too portability across systems nosotros test.

H5N1 fundamental do goodness of Ballista too other robustness testing approaches is that they await for holes inward the code itself, rather than holes inward the examine cases. Consider that most examine coverage approaches (including mutation testing) are interested inward testing all the code that is in that location (which is a practiced thing!). In contrast, robustness testing goes beyond code coverage to discover the places where yous should guide maintain had code to handgrip particular situations, but that code is missing. In other words, robustness testing oftentimes finds bugs due to missing code that should guide maintain been there. We discover that it is pretty typical for software to live non-robust unless this type of testing has been done it position such problems.

You tin discover to a greater extent than virtually our inquiry at the Stress Tests for Autonomy Architectures (STAA) projection page, which includes video of what goes incorrect when yous stress examine a duet robotic systems:

  https://www.rec.ri.cmu.edu/projects/stress_testing/

Our older piece of job on Ballista tin live constitute here:
  http://www.ece.cmu.edu/ koopman/ballista/

IMPORTANT, MUST BE READ...

Thank for your attention Robustness Testing

my blog Robustness Testing, Have a nice day.

Now you read article Robustness Testing this permalink article is https://fairemirima.blogspot.com/2017/11/robustness-testing.html Thank you and Best regards. You Can read nice Tips below. It was always better to choose topics that interest you or in wich you at least have some knowledge about . When creating targeted internet copywriting , you have to stick with your strong points , or everyone will know it . Make a list of all of the things and or topics that you are interested in . . . How much do you know ? Can you tell it as a story ? That is The essence of writing for the web . You Have to know your subject well , or nobody will believe you it is always better to impress someone then upset them . When Writing Targeted Internet Copywriting , you have to choose your appropriate target group of customers . without a target group of customers , you could ramble on incessantly about random subjects for days on end with no essence of a final goal . You always have to keep in mind who your customers are and what they are looking for . . . . . . . . . IMPORTANT, MUST BE READ...

Tweet