Testing Overview¶
A well-tested CKAN extension relies on a structured, domain-driven testing architecture. Rather than writing monolithic end-to-end tests that attempt to verify an entire feature in one file, tests must be divided by domain and placed in their expected, predictable location.
This guide outlines the testing strategy, layer separation rules, and overall workflow for extension testing.
Domain-Driven Test Placement¶
Every test in your extension has an expected place based on the layer of the application it targets. Testing layers independently ensures fast execution, precise failure diagnosis, and maintainable test suites.
graph TD
subgraph "UI & Browser Layer"
E2E["Playwright / Cypress E2E Tests (tests/e2e/)"]
end
subgraph "HTTP & Presentation Layer"
Views["View Tests (tests/views/)"]
CLI["CLI Tests (tests/cli/)"]
end
subgraph "Logic & Domain Layer"
Actions["Action Tests (tests/logic/test_action.py)"]
Auth["Auth Tests (tests/logic/test_auth.py)"]
Helpers["Helper Tests (tests/test_helpers.py)"]
end
subgraph "Data Layer"
Models["Model Tests (tests/test_models.py)"]
end
E2E --> Views
Views --> Actions
CLI --> Actions
Actions --> Auth
Actions --> Models
Where Tests Belong¶
| Application Layer | Test Location | Responsibility | Guide |
|---|---|---|---|
| API & Business Logic | tests/logic/test_action.py |
Validates Action data schemas, business rules, return values, and side effects. | Action Tests |
| Authorization | tests/logic/test_auth.py |
Validates access control rules, user permissions, and anonymous access restrictions. | Auth Function Tests |
| Database & Models | tests/test_models.py |
Validates SQLAlchemy v2 schemas, DB constraints, table queries, and model methods. | Model Tests |
| HTTP & Views | tests/views/ |
Validates Flask route mapping, HTTP status codes, headers, and redirects via app client. |
View Tests |
| CLI Commands | tests/cli/ |
Validates Click command execution, arguments, and terminal stdout output. | CLI Tests |
| Helpers & Utilities | tests/test_helpers.py |
Validates Jinja2 template helpers and isolated utility functions in pure isolation. | Helper Tests & Unit Tests |
| UI & E2E Workflows | tests/e2e/ |
Validates full browser interactions, DOM updates, and complex visual workflows. | Playwright E2E & Cypress E2E |
Why Layered Testing Matters¶
Dividing tests by domain delivers key engineering benefits:
- Fast Execution Loops: Action and model tests execute in milliseconds using direct Python calls. Reserving browser automation strictly for E2E tests keeps your continuous integration suite fast.
- Immediate Failure Diagnostics: When a database constraint fails,
test_models.pypinpoints the exact table issue immediately—eliminating the need to debug vague UI element timeouts. - Decoupled Refactoring: You can redesign template HTML or update Flask routes without breaking Action logic tests, provided the underlying Action API contract remains unchanged.
The Testing Process¶
When adding a feature or refactoring code, follow this testing progression:
- Test Data Persistence: Write model tests for custom database entities and methods (Model Tests).
- Test Core Logic: Write Pytest functions verifying Action execution and Auth rules (Action Tests and Auth Function Tests).
- Test Presentation Endpoints: Verify HTTP responses and status codes via Flask test client (View Tests and CLI Tests).
- Test Critical User Journeys: Use Playwright or Cypress strictly for interactive browser flows that require real DOM or JavaScript execution (Playwright E2E).
- Leverage Shared Infrastructure: Use Pytest Fixtures (
clean_db,ckan_config,app) and Test Factories (sysadmin,user,package) to prepare clean state before each test.
Self-Documenting Tests & Docstrings
Every test function is recommended to have a short docstring explaining what scenario or assertion it checks in a human-readable manner. Concise test docstrings make it effortless to audit covered functionality, understand test suite scope at a glance, and quickly spot missing use-cases or untested edge cases.
Section Map¶
Refer to individual testing guides for detailed code examples, Pytest fixture usage, and implementation details:
- Fixtures — Built-in CKAN Pytest fixtures (
clean_db,ckan_config,app, etc.). - Test Factories — Factory Boy definitions for generating test users, datasets, and organizations.
- Unit Tests — Unit testing principles, directory structure, and Pytest configuration.
- Action Tests — Testing action logic, validation schemas, and error handling.
- Auth Function Tests — Testing authorization logic, sysadmin bypasses, and anonymous rules.
- View Tests — Testing Flask Blueprints, web routes, and HTTP response codes.
- CLI Tests — Testing Click CLI command invocation and output parsing.
- Helper Tests — Testing custom Jinja2 template helpers.
- Model Tests — Testing database models, SQLAlchemy queries, and Alembic migrations.
- Playwright E2E — End-to-end browser automation testing using Playwright.
- Cypress E2E — End-to-end integration testing using Cypress.