CleanMM/Docs/Execution/Manual-Test-SOP.md

Manual Test SOP

Goal

Provide a repeatable manual test procedure for Atlas for Mac beta validation on a real macOS machine.

Intended Tester

• Internal QA
• Product owner
• Developer performing release candidate validation

Test Environment Preparation

Machine Requirements

• macOS 14 or newer
• Ability to grant Accessibility permission to the terminal and Xcode when UI automation is used
• Access to dist/native artifacts from the current candidate build

Clean Start Checklist

• Quit all running Atlas for Mac instances
• Remove old local install if testing a clean DMG install:
  • ~/Applications/Atlas for Mac.app
• Clear temporary validation folders if needed:
  • .build/atlas-native/
  • .build/atlas-dmg-verify/
• Ensure the build under test is freshly produced

Preflight Commands

Run these before starting manual validation:

swift test --package-path Packages
swift test --package-path Apps
./scripts/atlas/package-native.sh
./scripts/atlas/verify-bundle-contents.sh
KEEP_INSTALLED_APP=1 ./scripts/atlas/verify-dmg-install.sh
./scripts/atlas/verify-app-launch.sh
./scripts/atlas/ui-automation-preflight.sh || true
./scripts/atlas/run-ui-automation.sh || true

Manual Test Logging Rules

For every issue found, record:

• build timestamp
• artifact used (.app, .dmg, .pkg)
• screen name
• exact steps
• expected result
• actual result
• screenshot or screen recording if possible
• whether it blocks the beta exit criteria

Scenario SOP

SOP-01 Launch and Navigation

1. Open Atlas for Mac.app.
2. Confirm the main window appears.
3. Confirm sidebar routes are visible:
   • Overview
   • Smart Clean
   • Apps
   • History
   • Permissions
   • Settings
4. Switch through all routes once.
5. Confirm no crash or blank screen occurs.

Pass condition

• All routes render and navigation remains responsive.

SOP-02 Smart Clean Workflow

Reference: Docs/Execution/Smart-Clean-Manual-Verification-2026-03-09.md for disposable local fixtures and rescan/restore verification.

1. Open Smart Clean.
2. Click Run Scan.
3. Wait for summary and progress to update.
4. Click Refresh Preview.
5. Review Safe, Review, and Advanced sections.
6. Click Execute Preview.
7. Open History.
8. Confirm a new execution record exists.
9. Confirm Recovery shows new entries.

Pass condition

• Scan, preview, and execute complete without crash and leave history/recovery evidence.

SOP-03 Apps Workflow

1. Open Apps.
2. Click Refresh App Footprints.
3. Pick one app and click Preview.
4. Review the uninstall preview.
5. Click Uninstall for the selected app.
6. Open History.
7. Confirm an uninstall task run exists.
8. Confirm Recovery includes the app recovery entry.

Pass condition

• Preview and uninstall flow complete through worker-backed behavior.

SOP-04 Recovery Restore Workflow

1. Open History.
2. In Recovery, choose one item.
3. Click Restore.
4. Confirm the item disappears from the recovery list.
5. Return to the relevant screen (Smart Clean or Apps) and confirm state reflects the restore.

Pass condition

• Recovery restore succeeds and updates visible state.

SOP-05 Permissions Workflow

1. Open Permissions.
2. Click Refresh Permission Status.
3. Confirm cards render for:
   • Full Disk Access
   • Accessibility
   • Notifications
4. If you enable Full Disk Access, fully quit and reopen Atlas, then confirm Refresh Permission Status can reflect the new state.
5. Confirm the page does not hang or crash if some permissions are missing.

Pass condition

• Best-effort permission inspection returns a stable UI state, and Full Disk Access guidance matches the real macOS relaunch requirement.

SOP-06 Settings Workflow

1. Open Settings.
2. Change recovery retention.
3. Toggle notifications.
4. Quit the app.
5. Relaunch the app.
6. Return to Settings.
7. Confirm both values persisted.
8. Confirm acknowledgement and third-party notices text are visible.

Pass condition

• Settings persist across relaunch and trust content is visible.

SOP-07 DMG Install Path

1. Double-click dist/native/Atlas-for-Mac.dmg.
2. Drag Atlas for Mac.app into Applications.
3. Launch the installed app from Applications.
4. Confirm the main window appears.

Pass condition

• DMG install path behaves like a normal user install.

SOP-08 PKG Verification Path

1. Run pkgutil --expand dist/native/Atlas-for-Mac.pkg dist/native/pkg-expand-manual.
2. Confirm the package expands without error.
3. Run pkgutil --check-signature dist/native/Atlas-for-Mac.pkg.
4. Record whether the current build is Developer ID signed, ad hoc signed, or unsigned.

Pass condition

• PKG structure is valid and signing state is explicitly recorded.

SOP-09 Native UI Smoke

1. Run ./scripts/atlas/ui-automation-preflight.sh.
2. If the machine is trusted for Accessibility, run ./scripts/atlas/run-ui-automation.sh.
3. Confirm the UI smoke test suite passes.

Pass condition

• Native UI smoke passes on a machine with proper macOS automation permissions.

Failure Classification

P0

• App does not launch
• Primary workflow crashes
• Smart Clean or Apps core flow cannot complete
• Recovery restore fails consistently

P1

• Non-blocking but severe UX issue
• Persistent visual corruption on a core screen
• Packaging/install issue with a documented workaround

P2

• Minor UI issue
• Copy inconsistency
• Non-core polish regression

Final Tester Output

At the end of a run, summarize:

• build under test
• artifact path used
• scenarios executed
• pass/fail for each scenario
• P0 / P1 / P2 issues found
• recommendation:
  • Pass
  • Pass with Conditions
  • Fail