How NativeTest Securely Validates Native Apps — Step-by-Step

Troubleshooting NativeTest: Common Issues and Fixes

Overview

NativeTest is a tool for validating native applications. This article covers common issues you may encounter and precise fixes to get tests running reliably.

1. Installation fails or hangs

• Symptom: Installer exits with errors or progress stalls.
• Causes: Missing dependencies, insufficient permissions, corrupted installer.
• Fixes:
  1. Check permissions: Run installer with admin/root privileges.
  2. Install dependencies: Ensure required runtimes (e.g., specific JDK, .NET, or platform SDK) are present.
  3. Verify checksum: Re-download installer and confirm checksum matches the vendor-provided value.
  4. Use verbose logs: Re-run installer with verbose/debug flag and inspect logs for missing files.

2. Tests fail to launch on device or emulator

• Symptom: Test runner cannot connect to device/emulator or app fails to start.
• Causes: Device not visible, incompatible OS version, missing app build, platform tooling mismatch.
• Fixes:
  1. Confirm device connectivity: Use platform tools (adb devices or idevice_id) to verify the device is detected.
  2. Match OS versions: Ensure app build supports the device OS; rebuild if necessary.
  3. Restart tooling: Restart adb/daemon or emulator; reboot device if needed.
  4. Check app installation: Manually install the app bundle and verify it launches outside NativeTest.

3. Flaky or nondeterministic tests

• Symptom: Tests intermittently pass or fail without code changes.
• Causes: Timing issues, race conditions, environment variability, resource contention.
• Fixes:
  1. Stabilize waits: Replace fixed sleeps with explicit waits for UI elements or conditions.
  2. Isolate tests: Run tests in single-threaded mode to detect shared-state issues.
  3. Reset state: Ensure each test starts with a clean app state (clear caches, reset DB, reinstall app).
  4. Increase resource allocation: Allocate more CPU/memory to emulators or CI runners.

4. Network-dependent tests failing

• Symptom: Tests that rely on network calls fail in CI or lab environments.
• Causes: No network access, flaky test endpoints, DNS issues, proxy/firewall constraints.
• Fixes:
  1. Mock network: Use stubs or local mock servers for external APIs during tests.
  2. Verify connectivity: From the test environment, ping the endpoint or run curl to confirm access.
  3. Configure proxy: Set HTTP(S)_PROXY and related environment variables if a proxy is required.
  4. Use deterministic test data: Seed predictable responses to avoid variability.

5. Permissions and signing errors (mobile)

• Symptom: App installation or automation fails due to signing or permission denials.
• Causes: Invalid code signing, missing entitlements, runtime permission prompts blocking automation.
• Fixes:
  1. Validate signing: Ensure code signing certificates and provisioning profiles are correct and unexpired.
  2. Grant permissions programmatically: Use platform tooling to grant runtime permissions before tests run.
  3. Disable prompts: Use test build flags or automation-friendly settings to bypass first-run dialogs.

6. Slow test execution

• Symptom: Test suite runs much longer than expected.
• Causes: Heavy UI operations, large test dataset, emulator overhead, synchronous waits.
• Fixes:
  1. Parallelize suites: Run independent tests in parallel where safe.
  2. Use headless emulators: Use faster headless or cloud device farms optimized for CI.
  3. Trim tests: Focus on critical tests for fast feedback; move long-running scenarios to nightly runs.
  4. Profile hotspots: Measure where time is spent and optimize setup/teardown or expensive operations.

7. Log and artifact collection problems

• Symptom: Missing logs, screenshots, or crash dumps after failures.
• Causes: Misconfigured artifact paths, insufficient permissions, retention policies.
• Fixes:
  1. Configure paths: Ensure NativeTest output directory is writable and paths are correctly set.
  2. Capture on failure: Enable automatic screenshot and log capture on test failure.
  3. Increase retention: Adjust CI/job retention to keep artifacts long enough to inspect.

8. Integration issues with CI/CD

• Symptom: Tests pass locally but fail or behave differently in CI.
• Causes: Environment differences, missing device drivers, headless mode, timing.
• Fixes:
  1. Replicate CI locally: Mirror CI environment using containers or VM images to reproduce issues.
  2. Pin tool versions: Lock NativeTest, SDK, and platform tool versions in CI configuration.
  3. Add retries with backoff: For transient CI-related failures, add controlled retries and better logging.

Troubleshooting checklist (quick)

• Confirm environment matches required versions.
• Verify device/emulator visibility and tooling (adb, emulators).
• Reproduce failures locally with verbose logging.
• Isolate and run a single failing test.
• Reset app state and dependencies before test runs.
• Capture logs, screenshots, and crash dumps for each failure.

When to contact support

If you’ve tried the above and still see issues, collect:

• NativeTest version and logs (verbose),
• Platform SDK version and device/emulator details,
• Repro steps and a minimal failing test. Provide these to your support or engineering team for deeper analysis.

End of article.