Test, Execution and Recording

“It will work…” (Probably)

Based on Photo by Ksenia Chernaya from Pexels
Passed?

No product manager wants to be confirming their solution’s viability without the backup of evidence. It is imperative to test and assure your process or product. Tests should demonstrate conformance with, all the vital parameters that comprise your design; regulatory compliance; ongoing quality.

This whole quality cycle should include test, execution and recording functions.

Appropriate Testing

Cradle TER module - Test Case
Test Case

We should accept that these tests, and the effort used, should be commensurate with the importance of the design element. There’s every reason to test whether a product can place components with sub-millimetre accuracy if we’re designing a circuit board pick and place machine, but within a few mm is probably good enough for a machine placing variable size apples in a packing crate. When testing the same product used in different situations the complexity and accuracy of the test will change. Measuring the slump and hardness of concrete for a garden path, is probably judged by eye, three level compaction in a cone testing for a road, but you’d expect samples to be taken and lab tested it is being poured to form the main tower of a suspension bridge.

Scope

It is also important to test not only for the expected criteria, but the off-norms (off from normal) too. “The program shall accept user input of their weight in kilogrammes and their height in metres. Their BMI shall be displayed as a ratio of their (weight / height) to two decimal places” The previous requirements can be thoroughly tested for a full range of human weights and heights, but then fall over at the first hurdle when a user enters 0 for their height.

Executing

Cradle TER module - Test Run
Test Run

Once developed, the set of tests may need to be run multiple times on the same product as development tweaks are made, or once on each batch of product to ensure ongoing conformity. Services, procedures and physical products can all be tested. Whether that’s timing the office evacuation during a fire drill or the alcohol content of a batch of whiskey. It is also important to recognise that each execution set may need to be full or a partial set of the tests.   Every safety harness may be subject to visual inspection and, checks on fastener operation. Every tenth to a set of detailed measurements and every two hundredth to a full destruction test. The ability to execute a set of tests from our full suite is therefore important.

 

Recording

Cradle TER module - Test Result
Test Result

Finally the test execution will produce a set of results which should be recorded. Not only does the recording provide the traceability your QA (Quality assurance) plan should aim for, but it will allow you to investigate trends. How many failures were there last month? Were there more or fewer than the previous month. Is investigation required?

 

Cradle

Cradle’s TEST module  (Test, Execution & Recording (TER)) allows you to directly link Test Cases to your requirements, needs, or design elements. You can then define Test Plans and Test Executions to group and run these tests. And as you’d expect the tool will record the Test Results against each Test Step.

For more information download an evaluation copy of Cradle or book a webinar now.

TER video

How Risky?

Risk Management

Every project has risks associated with it. They range from the risks to the programme, supply chain, staff, technology; through financial backing and cash flow; to safety and performance of the product. But just how risky is it?

Recording, quantifying, mitigating and reviewing these risks helps reduce their likelihood and severity. The more complex the project the more risks you will need to manage and the greater the range and type that needs to be managed.

Risk Register example
Risk Register

Tools

Tools to manage risk don’t make the risks go away in themselves, a shiny RAG chart does not make a project safe. However, management is all about controlling these parameters. Visualisation and quantifying risks is a management aid to ensure effort is spent in the most appropriate place to give maximum benefit, and also to ensure the smaller issues don’t get completely lost at the periphery.

Parameters That Define Risk

ParameterDescription
Likelihood
The probability of the risk occurring. This could be simply High, Medium and Low, or 9:10, 5:10, 1:10, or Daily, Monthly, Yearly.
Consequence
What impact does the risk occurring pose to the project. Again this could simply be a generic High, Medium and Low or a more ultimate Death, Hospitalisation, Injury or Catastrophic, Severe, Dangerous, Limited.
Magnitude
The importance or priority we assign to the risk given our assessment. These quantities are often associated with colours  e.g. High=Red, Medium=Orange, Low=Green. These would be the colour displayed on your RAG chart.
Dates
Some risks will only be present during certain parts of the project. Funding may only be an issue up to the point that the project is started. Manufacturing defects can be thought about but can’t start occurring until the production run starts, so don’t form part of the overall current profile until that point. Adding start and finish dates to your risks bounds them and allows you to show a chronological profile.

Finally every risk should be reviewed on a regular basis to make sure the parameters have not changed and the mitigations are still valid.

Value
Used to quantify the size of the risk should the risk’s event occur.
Owner
The person or organisation who is responsible for determining the mitigation of the risk and for monitoring how this mitigation is avoiding (negative risks) or promoting (positive risks) the risk’s associated event(s).
Mitigation
There is not much point identifying a risk if we make no effort to reduce it. Unless that is of course because it is below our threshold. The impact is low, and the probability is small and if it did occur the cost is small. In all other cases we should record what it is we intend to do to reduce the risk. We can then re evaluate the risk with the applied mitigation. We dig a hole in the street, likelihood is someone will fall down it, the impact is severe and the value would be expensive. This would generally be flagged as a High priority risk. The mitigation might be to assemble barriers before hole is dug. This does not reduce the severity of a fall or the cost to business, however the probability that someone will fall is drastically reduced, and thus the mitigation leads to a reassessment as a Medium priority risk.

Analysis

By selecting categories for each risk, grouping the likelihoods and consequences we can draw a matrix. A RAM (Risk Assessment Matrix) this provides a uniform method of quantifying the risks.

LikelihoodConsequence
1 – TBD2 – Low3 – Medium4 – High
1 – HighHighMediumHighCritical
2 – MediumMediumMediumMediumHigh
3 – LowLowLowMediumMedium
4 – TBDTBDLowMediumHigh

For each risk we decide the likelihood it will occur (its probability), the consequence of it occurring (its impact) and then look up the magnitude given to the risk (its risk priority). When we then look at the project as a whole, we can see how many of these risks are classed as high or critical. These are areas that need resource and attention first. As noted above if the risks present themselves at different times during the project it is also possible to produce a chronological risk profile.

Visualisation and Profiling

How Risky – Counting

Risk graph example
Risk Count

Risks can simply be counted, and visualised as a graph over time. This gives a good indication of the number of each type of risk we are dealing with at any point throughout the project. However we should concentrate efforts on those most critical risks.

How Risky – Costs

Total risk profile example
Total Risk Count – Profile

Once we have an idea of the number of risks we have for each period of our project we can produce a risk profile.

Unfortunately this does not give us a picture of the overall cost to the project if these risks occur.

Maximum Risk Profile example
Maximum Risk – Profile

By assigning a value to each risk we can work out the overall ‘cost’ whether that be in time delays, money or some nominal value. For any point in time we have a maximum risk exposure.

How Risky – Weighting

Weighted Risk Profile example
Weighted Risk – Profile

However, we also must take a pragmatic view to the consequence of a risk and the effort that is reasonable to expend trying to mitigate it. If we have ten risks that have a fair chance of occurring, and will have an impact on our business, it is likely that we should spend effort mitigating these. Whilst we will want to mitigate against a  catastrophic risk, if its likelihood is “once in a blue moon” the amount of effort expended must be tempered. By assigning a weighting to our risks we can ensure the overall profile is adjusted to be more meaningful. There is no exact science to this but it is a tool to help focus the projects needs.

Mitigation and Review

It is important to record all the decisions and parameters used when assessing risks and designing mitigations. Whether this be reserving funds for an unexpected cost, or adding safety barriers round the hole. We’re sorry to say that adding a high-viz isn’t the correct mitigation for every risk!

Re-evaluate

Once each risk has been calculated and a mitigation has been assigned, its value should be recalculated in light of the mitigation. For example if we had identified a risk that members of the public may fall down holes we are digging to install fibre, our mitigation may include adding plastic barriers round the hole. The probability that a person will fall down a marked and barriers surrounded hole may reduce from a likelihood of  “Highly likely” to “Not very likely“. Whilst the overall count of risks will not have been reduced, the weighted profile will have lowered as although the cost of someone falling down the hole has reduced, the likelihood has been reduced.

Risks and the appropriate mitigations will change over time. This may be because a particular likelihood has increased, an element of the project has been delivered or delayed. So it is important to add review dates into your plan to ensure they are re-evaluated correctly. After all there is no point ordering all those plastic barriers to put round the hole if a project decision to sub contract hole digging was made a month earlier. Therefore you should always be asking “How risky is my project?”

Cradle

Cradle 7.6 introduces a new Risk Management module to help in planning for and managing project  risks. All the aspects above are held in Cradle attributes and each Risk item can be linked to any other Cradle item, e.g requirement, design note, diagram. This will help you manage the risks associated with each element of your project’s needs, and solutions. You can find more in the Risk section of the Cradle manual.

Related Articles

Cradle Risks video

Cradle 7.6 – Released

We are pleased to announce the release of Cradle-7.6

Cradle 7.6 Splash screen
Cradle 7.6

Cradle-7.6 is available now for download from the downloads section of the 3SL website.  This is the latest version of Cradle.

3SL customers with active maintenance have been sent an e-mail notification, and details of which of  their enhancement requests and bug reports are included in this new release.  This is a significant Cradle release that increments the Cradle minor version number from 7.5 to 7.6. This means that the Security Codes for Cradle-7.5 will not work with this new Cradle-7.6 release. Therefore, if you want to upgrade to the new Cradle-7.6 then you must contact 3SL Support and request a new Cradle-7.6 Security Code since your existing Cradle-7.5 Security Codes will not work with this new release.

New Cradle-7.6 Capabilities

This release contains a range of new capabilities, including major new modules RISK and TEST for risk management and test execution and management, respectively – these are charged for separately – and a collection of enhancements and bugs for the existing Cradle components:

New Modules

  • Risk Management
    • Risk management is an activity undertaken, to a greater or lesser extent, by almost every organisation. It is fundamentally concerned with identifying events which, if they occur, would have a positive or negative effect on an organisation or project.
  • Test Execution
    • Test execution and recording is generic testing of a system by people who are prompted to perform a series of steps and indicate the result of each of these steps, together with some text and image notes.

New Features

  • Linux Installer
    • Install process enhanced to improve installation experience and application launching shortcuts added to improve
      Cradle integration
  • Project Setup
    • Ability to hide unused item types from the WorkBench and Web Access user interfaces
  • User Preferences
    • New text size options now available
  • Quick Access Bar
    • Add Project queries to Quick Access Bar
  • Configuration Management
    • Ability to Approve All and Reject All from the Review tab
    • Ability to Register All from the Review tab
  • Systems Modelling
    • Categories can be chosen in Project Setup to colour item type symbols on a diagram
    • Option to initialise Project Setup with SysML related elements. There is also an option to clear SysML related elements.
  • External Commands
    • Two external commands added for item re-versioned and cross reference baselined.

Performance Enhancements

We have made a number of low-level, internal, changes deep inside the Cradle code that reduce the number of interactions between Cradle clients and the database. This will reduce the time that Cradle takes to perform a variety of common operations. The effect will be more noticeable the longer the time between your Cradle clients and the Cradle Database Server (CDS). Separately, we have made some fundamental improvements in the way that Cradle interacts with Oracle and MySQL RDBMSs so that those customers who store their Cradle databases in either Oracle or MySQL will see a particularly dramatic performance improvement.

Operations that will be noticeably faster include:

  • Displaying sets of top-level items or bottom-level items in the Project and Phase sidebars
  • Querying using levels
  • Handling cross references
  • Importing information
  • Indexing and frame handling in ODBC
  • Reduced PDB access when sending alerts

Download

You must contact 3SL for new Security Code(s) for Cradle-7.6. Cradle-7.6 will not accept Security Codes from Cradle-7.5 or any previous release.

Cradle clients (WorkBench, Web Access, Document Publisher for instance) and server (Cradle CDS) versions can not be mixed. Therefore, you must upgrade all Cradle installations to 7.6.

Cradle-7.6 databases do not have the same format as Cradle-7.5 databases. Hence the Cradle-7.6 release includes a database converter for the transition from Cradle-7.5 to Cradle-7.6.  Full details are available in the 7.6 release notes, and updated manuals in addition you can always contact support@threesl.com .

Single User Products

Please note that there are no maintenance services for single-user Cradle products. Therefore, if you have purchased any of the single-user Cradle-7.5 products:

    • Cradle-RM Desktop
    • Cradle-RM Pro
    • Cradle-SE Desktop
    • Cradle-SE Pro

then you will not be able to request a new Cradle-7.6 Security Code. If you want to update your single-user Cradle system to the new Cradle-7.6 release, then you must buy the new Cradle-7.6 release.

Help with Cradle-7.6

In conclusion, we’re pleased with the new capabilities in Cradle-7.6 most importantly hope you will benefit by upgrading. If you are not already a customer and would like more information about Cradle, you can download the software and a free evaluation licence, find more on our website, or request a webinar. If you would like to read some independent reviews feel free to use your favourite search engine or take a look at Capterra.

Article Updated

02/01/2021 – Additional information regarding changes and clarification for single user products.

08/02/2021 – Typo. Added link to special discount offer 7.5 – 7.6 Single user products