What Is Key Difference between Software Smoke Testing and Sanity Testing?

Victor Font (Senior IT Executive) Smoke testing and sanity testing are often confused. Smoke tests are used as acceptance tests before introducing a new build to the primary testing process. In other words, a smoke test determines whether it is possible to continue testing. If the smoke test fails, it is impossible to conduct a sanity test.

The sanity test determines whether it is reasonable to proceed with further testing by testing the smallest subset of application functions needed to determine whether the application logic is generally functional and correct. If the sanity test fails, it is not reasonable to attempt more rigorous testing.

Both sanity and smoke tests avoid wasting time and effort by quickly determining whether an application is too flawed to merit further rigorous testing. Many companies run automated sanity and unit tests as part of their development process.

Roy Monica (Head of specialized platforms, Sprint) Smoke testing and sanity testing are different tests performed to assess the status of a particular release. One key difference is the reason that the testing is being performed, and another is what team is performing the test.

Smoke testing is usually performed by the development team to see if the build or release 'comes together' as planned. When making complex changes, there are usually hurdles and/or gates that must be passed before handing off code off from the development team to QA or test. Smoke testing gives the development team a sense that the release has accomplished the basic goal desired from a coding change, and that the release is worth to send to QA or test.

Sanity testing is often the first thing the QA or test team will perform when receiving a release from the development team. Again, commencement of the testing process is usually an expensive endeavor from a resource perspective, and a 'sanity' test gives the testing team confidence that the release is worth investing in. Sanity testing can be a small regression test or a subset of critical functionality.

Robin Goldsmith (President, Go Pro Management, Inc.) Because smoke and sanity tests are both preliminary types of tests aimed at avoiding wasted further test effort, they can be confused. However, they have different purposes and are performed differently.

A smoke test is run to confirm that code is sufficiently stable to warrant further testing. A smoke test does not check the accuracy of the code. Testers often run smoke tests on code when they receive it from the developer. The smoke test usually exercises basic functionality from beginning to end. If the code can’t at least stay up that long, there is no point in subjecting it to more demanding tests; and the tester sends it right back to the developer to be fixed without wasting further test time. Proactive testers give the smoke tests to the developers so they can find their own problems and thus avoid added delays from sending code versions that testers will just have to send back; but even so, the testers still run the smoke test when they receive the code just in case…

A sanity test is run early on newly-received code to confirm that the code is reasonably accurate. That is, a sanity check focuses on basic functionality and especially business rules. If the basics are not correct, there is little reason to expect more complex functions to be correct; and thus again the code is sent right back to the developer rather than spending time on further tests which invariably will fail. Unlike a smoke test, which developers also can perform, sanity tests must be performed by a knowledgeable, independent party, because they are checking the developer’s content understanding. A separate view is needed because a developer has no way to tell by him/herself whether s/he correctly understands the content s/he is automating.