Automatically Import Android CTS Failures into Kiwi TCMS

Running the Android Compatibility Test Suite (CTS) is essential for validating Android devices, but parsing 100,000+ test results manually? That’s not scalable.

This lightweight Python script helps by importing only failed test cases into Kiwi TCMS, keeping your test management focused, clean, and ready for real debugging.

GitHub Repo: github.com/maksonlee/import-android-cts-to-kiwi-tcms

What This Script Does

This tool parses test_result.xml from an Android CTS run and syncs only failed test cases into Kiwi TCMS.

Key Features

• Parses Android CTS test_result.xml
• Only imports test cases with FAILED, BLOCKED, or ERROR status
• Automatically creates:
  • Product: Android Phone
  • Version: e.g., 14_r7
  • Test Plan: Android CTS
  • Category (per test package)
• Tracks previously failed test cases and updates their status if they pass later
• Completely ignores test cases that have always passed

Installation

Clone the repository:

git clone https://github.com/maksonlee/import-android-cts-to-kiwi-tcms.git
cd import-android-cts-to-kiwi-tcms

Install dependencies:

pip install tcms-api lxml

• Requires Python 3.10+
  Depends on: tcms-api, lxml

Configuration

Set the following environment variables:

export TCMS_URL=https://tcms.yourdomain.com/xml-rpc/
export TCMS_USERNAME=your_username
export TCMS_PASSWORD=your_password

export PRODUCT_NAME="Android Phone"
export VERSION_NAME="14_r7"
export TEST_PLAN_NAME="Android CTS"

Place your test_result.xml file in the same directory as the script, then run:

python3 import_cts_failures.py

Sample output:

📋 Created Test Run ID: 42
✅ Created TestCase: com.android.cts.SomeTest#testSomething()
🔁 Updated result for com.android.cts.AnotherTest#testAgain(): PASSED
✔️ Imported 15 failed tests, 3 updated results

Why Only Failures?

CTS generates over 100,000 test results per run. Importing everything into Kiwi TCMS would:

• Bloat your test management database
• Slow down searches and dashboards
• Obscure the real problems

This script uses a fail-first principle:

• Focuses only on test cases that matter
• Keeps Kiwi TCMS lean and actionable
• Tracks historical failures until resolved

Project Structure

import-android-cts-to-kiwi-tcms/
├── import_cts_failures.py   # Main script
└── test_result.xml          # CTS output file

Test Plan Type

By default, the script creates test plans of type:

type_name = "System"

You can customize this in the script if your organization uses other types (e.g., Regression, Certification).