IT6206 — Software Quality Assurance

Software (IEEE): Computer programs, procedures, and possibly associated documentation and data pertaining to operation of a computer system.

Software (ISO, 4 components): Computer Programs | Procedures | Documentation | Data necessary for operating the system.

Relationship: QC is a subset of QA. Testing is a Quality Control activity (not QA directly — QA supports good testing).

Example chain: Developer writes if (age > 18) instead of if (age >= 18) (Error) → the age check module now incorrectly rejects 18-year-olds (Fault) → when an 18-year-old registers, they are wrongly blocked (Failure).

Key statistic: Example — 17 errors → 8 faults → 3 failures.

ISO 9000-3: Based on 8 quality management principles (Customer Focus, Leadership, Involvement of People, Process Approach, Continual Improvement etc.). Certification involves: (1) Develop SQA system → (2) Implement system → (3) Undergo certification audit.

CMM / CMMI: Carnegie Mellon initiative (1986); 5 maturity levels: Initial → Managed → Defined → Quantitatively Managed → Optimising. CMMI adds Agile principles; improves dev process, configuration management & quality management.

Bootstrap: Alternative — measures maturity on 31 quality attributes across 3 classes: Process, Organisation, Technology.

Benefit of certification: Ability to obtain independent professional quality audits; demonstrates commitment to quality to customers and stakeholders; opens doors to contracts requiring ISO certification.

Work products is the generic name given to any form of documentation, informal communication or artefact used in testing.

Traceability: The degree to which a relationship can be established between two or more work products. Bi-directional traceability links requirements ↔ test conditions ↔ test cases ↔ test results.

Benefits (any 3):

• Analysing the impact of changes in requirements or design
• Making testing auditable and measuring test coverage
• Meeting IT governance criteria
• Improving coherence of progress/summary reports to stakeholders
• Relating technical aspects to stakeholders in understandable terms
• Assessing product quality & project progress vs business goals

Agile Advantages: (1) Focus on working software → simpler design, easier to test. (2) Test-driven development (TDD) — testing is the starting point, driving code quality.

Agile Challenge: Constant time pressure; less time available for new-feature testing; test design is less formal and subject to change.

White-box coverage techniques (any 4):

• Statement Coverage — every executable statement run at least once
• Decision / Branch Coverage — every TRUE and FALSE outcome executed
• Condition Coverage — all individual conditions within decisions covered
• Path Coverage — every possible execution path covered (most thorough)
• Multiple Condition Coverage — all combinations of condition outcomes tested
• FSM (Finite State Machine) Coverage — how many states are visited/transited

Key rule: 100% decision coverage → guarantees 100% statement coverage. But NOT vice versa.

With 2 binary conditions → 2² = 4 rules:

Why useful: Systematically covers ALL combinations of conditions → finds gaps/ambiguities in specs that other techniques miss. Works well combined with Equivalence Partitioning.

(i) 4 Parts of a State Transition Model:

• States — conditions the software may occupy (e.g. open/closed)
• Transitions — movement from one state to another (not all transitions are valid)
• Events — what CAUSES a transition (e.g. inserting a card, entering a PIN)
• Actions — RESULT of a transition (e.g. error message, cash dispensed)

(ii) State Transition Table:

Dashes (—) = INVALID transitions — important for designing negative test conditions.

Formulae:

• Statement Coverage = (Statements exercised ÷ Total statements) × 100
• Decision Coverage = (Decision outcomes exercised ÷ Total decision outcomes) × 100

Statement Coverage — Test01 (X=5, Y=3):
X > Y is TRUE → Z=0 executes. 5 statements executed out of 5 total. 5/5 × 100 = 100%

Decision Coverage — Test01 + Test02:

• Test01 (X=5, Y=3): X>Y = TRUE → 5 outcomes. Decision Cov. = 5/5 × 100 = 100%
• Test02 (X=–2, Y=8): X>Y = FALSE → 4 outcomes. Decision Cov. = 4/5 × 100 = 80%

KEY RULE: 100% Decision Coverage guarantees 100% Statement Coverage — but NOT vice versa. Statement coverage provides LESS coverage than decision coverage.

Static Testing: Examining software work products manually or with tools WITHOUT executing the code.

Primary Objective: Improve quality by helping engineers recognise and fix their OWN defects early in the lifecycle.

Work products (any 4): Source code | Design documents/models | Requirement specifications | Test plans | User manuals | Traceability matrices

Typical CIT defects: incorrect sequencing/timing of interface calls, wrong assumptions about data meaning. Typical SIT defects: inconsistent message structures between systems, failure to comply with security regulations.

Why critical in Agile: Each sprint delivers working software quickly → changes are frequent → without automated regression testing, new features constantly risk breaking existing functionality. Regression is the safety net that lets teams move fast with confidence.

1. Requirements Analysis — identify testing scope, prepare RTM, automation feasibility
2. Test Planning — prepare Test Plan, effort estimation, resource plan
3. Test Case Design & Development — scenario ID, test case writing, test data preparation
4. Test Environment Setup — configure hardware/OS/software; verify with smoke test
5. Test Execution — run tests, compare actual vs expected, log defects
6. Test Closure — archive artefacts, sign-off, lessons learned, closure report

Entry Criteria: Conditions or documents REQUIRED to begin a particular STLC phase. If absent, the task will not be performed.

Exit Criteria: Items/documents/actions/tasks required to be completed BEFORE concluding an STLC phase and moving to the next. A set of expectations from a particular phase.

Test Closure Phase:

Flow: New → Assigned → Active → Test/Fixed → Verified → Closed
Side paths from Active: → Rejected | → Deferred; and Closed → Reopened

Priority = QA team's view — the ORDER in which defects should be RESOLVED (Low / Medium / High / Urgent). Example: Submit button colour mismatch — Low Priority (cosmetic).

Severity = Dev team's view — the IMPACT on the system and complexity to fix (S4 Low → S3 Medium → S2 Major → S1 Critical). Example: App crashes every login → S1 Critical Severity.

These are independent: a logo on a homepage wrong colour = High Priority but Low Severity. A crash in a rarely-used feature = Low Priority but High Severity.

• Monitor the test plan, schedule and key project parameters; keep on track
• Conduct progress and milestone reviews to determine actual status; re-plan as appropriate
• Monitor risks and take appropriate action
• Analyse issues and change requests and take appropriate action
• Monitor resources and manage resource issues
• Report the test status to management

Schedule Timeliness: Refers to elapsed calendar time (months) — compares planned vs actual elapsed calendar months for each phase/milestone. It is a lagging measure.

Effort Timeliness: Refers to the planned number of person-months required to carry out the work, vs the actual number of person-months it actually took. NOT calendar time.

Key difference: Schedule is about TIME (calendar); Effort is about PEOPLE/HOURS (person-months). Both are needed for accurate project reporting.

Responsibility: The Test Manager presents status, key risks and issues to management, explaining how issues are being managed.

Automation Testing is the use of software tools to develop and execute tests.

Tools used in test automation can:

• Enter test data into the application under test
• Compare expected and actual results
• Generate detailed reports automatically

"Software test automation is not always possible" — TRUE. Examples where automation is NOT suitable: exploratory testing, usability/UX testing, one-off tests, tests requiring human observation.

Advantages (3):

• Speed — Test execution is faster compared to manual testing
• Reliable — Automated tests perform the same operation precisely each time, eliminating human error
• Repeatable — Tests can be re-run easily across different versions
• Also: Coverage (wider test coverage) | Reusable (across app versions)

Risks (2):

• Incompatibility of automation tools with the test environment
• Incorrect evaluation of time and effort required (tool selection, training, maintenance)
• Also: Overlooking human-tester capability | Budget/vendor issues

Key Factors (any 2):

• Platform Compatibility — OS types, application types, mobile platforms supported
• User-Friendliness — Supports keyword testing or requires scripting expertise
• Ecosystem Integration — Integrates with CI/CD pipeline, defect management, source control

XPath = //tagname[@attribute='value']

Example: //input[@id='username'] — finds all <input> elements anywhere in the DOM where id equals 'username'.

Additional axes: following (all nodes after), preceding-sibling (siblings before), descendant (all children/grandchildren).

Is POM required? FALSE — POM is a best-practice design pattern, not a requirement. Selenium is effective without POM.

// i. Create ChromeDriver instance
WebDriver driver = new ChromeDriver();

// ii. Open URL
driver.get("http://ucsc.lk");

// iii. Click button by XPath
driver.findElement(By.xpath("//input[@id='btn']")).click();

// iv. Select dropdown option
WebElement dropdown = driver.findElement(By.xpath("//select[@id='country']"));
Select select = new Select(dropdown);
select.selectByVisibleText("Sri Lanka");

// v. Close all windows
driver.quit();  // Use driver.close() to close only the current window

JUnit and NUnit

Annotation order: BeforeSuite → BeforeTest → BeforeClass → BeforeMethod → @Test → AfterMethod (repeats per @Test) → AfterClass → AfterTest → AfterSuite

Note: @Test with no priority = priority 0 (runs FIRST). @Test(priority=1) runs second.

C
B
A
Test One
A
A
Test Two
A
B
C

CI — Continuous Integration: A method of continuously verifying the condition of a codebase through automated testing. Best achieved by integrating CI with version control. Developers merge code frequently; automated builds & tests run on each commit.

CD — Continuous Delivery: Approach to regularly deploying artefacts that successfully pass the CI phase, to ensure confidence around deployments. Deployments become quick, frequent, and reliable.

(A) sh 'make check || true'
Executes the shell command "make check" within the Jenkins environment. The || true part ensures that even if "make check" fails (returns a non-zero exit status), the pipeline does NOT fail. The pipeline continues regardless of the command's outcome.

(B) junit '**/target/*.xml'
Publishes JUnit test results by specifying the path to the JUnit XML report files. The glob pattern **/target/*.xml matches XML files in ANY subdirectory of the "target" directory. Jenkins uses these results to mark the build as UNSTABLE (yellow) if tests fail, and captures them for the build report.

# A. Initialise repository
git init bitproject

# B. Stage all files and commit
git add .
git commit -m "Initial Commit"

# C. Create branch and switch to it
git branch feature
git checkout feature
# OR shorthand:
git checkout -b feature

# D. Switch to master and merge
git checkout master
git merge feature

# E. View commit history
git log

UIAutomator: A native Android UI automation framework that allows testers to run JUnit test cases directly on the device via command line using Java. Appium allows it to be driven from any WebDriver-supported language. It provides APIs to interact with UI elements (click buttons, enter text, navigate screens, verify elements).

Appium Architecture: Appium is an HTTP server written in Node.js. Client-server flow:

1. Test script sends commands (JSON over HTTP/WebDriver protocol) to the Appium server
2. Appium initiates a session and returns a session ID
3. Appium translates commands to platform-specific frameworks
4. Returns execution status to the test script

Additional Frameworks:

• BDD (Behaviour Driven): Tests written DESCRIPTIVELY in plain English (Gherkin — Given/When/Then) so business stakeholders can understand them. Tests in terms of user-focused behaviours/scenarios.
• Module Based: Application tested in smaller parts (functions/modules/sub-systems); separate test scripts per module; combined for system-level tests.

Test Summary Report: A document summarising all test activities and final results of a testing project. Helps stakeholders assess product quality and decide whether to release.

Key sections: Test Objective | Areas Covered | Areas NOT Covered | Testing Approach | Defect Report | Platform Details | Overall Summary/Recommendations | Exit Criteria & Sign-Off

10-Step Creation Process:

1. Capture purpose of the document
2. Capture overview of the product under test
3. Capture testing scope (in/out, untested areas + reasons)
4. Capture metrics (planned/executed/passed/failed test case counts)
5. Capture types of testing performed (incl. multiple rounds)
6. Capture test environment & tools (App URL, DB version, bug tool)
7. Document learnings during test activity (issues & solutions)
8. Write recommendations for next cycle
9. Define exit criteria (all planned cases run; critical issues closed)
10. SIGN OFF — testing team approves go-live if exit criteria fulfilled