Testing Envoy
Testing Envoy
Envoy is composed of three distinct services, each with its own testing suite tailored to its responsibilities and technology stack. This section outlines how to run and understand the tests for the Agent, Tools, and Portal services.
Agent Testing
The agent/ service, built with Python and FastAPI, uses pytest for its testing. Tests cover API endpoints, core business logic, database interactions, and integrations with external services (like the tools/ service or LLMs).
How to run Agent tests:
Navigate to the agent/ directory and run pytest:
cd agent/
pytest
Key aspects of Agent tests:
- API Endpoint Testing: Uses FastAPI's
TestClientto simulate HTTP requests to the Agent's API endpoints. This verifies request/response schemas, status codes, and basic functional behavior.- Example:
agent/tests/test_api_jobs.pyandagent/tests/test_api_profile_interview.pydemonstrate this for job listings and profile interview flows.
- Example:
- External Service Mocking: For interactions with the
tools/service or external LLMs (like OpenAI), tests often employ mocking libraries:respx.mockis used to intercept and mock HTTP requests made from the Agent to thetools/service, ensuring that external dependencies don't require actual browser launches during Agent tests.- LLM calls (
app/services/profile_interview_ai.py) are typically mocked or configured to use dummy models during testing to avoid costly and slow external API calls.
- Database Isolation: Tests ensure data isolation by configuring SQLite to run in an in-memory database (
:memory:) usingmonkeypatch.setenv("SQLITE_PATH", ":memory:"). This means each test run starts with a clean database. - Environment Configuration:
pytestfixtures (e.g.,clientfixture inagent/tests/test_api_jobs.py) are used to set up environment variables and application settings dynamically, ensuring a consistent and isolated test environment.
Tools Testing
The tools/ service, built with Node.js and Playwright, focuses on browser automation and HTML parsing. While specific test files were not provided in the context, a typical testing approach for such a service involves:
How to run Tools tests (inferred):
Navigate to the tools/ directory and use a Node.js test runner (e.g., Jest or Vitest):
cd tools/
npm test
# or
yarn test
Key aspects of Tools tests (inferred):
- Browser Automation Scenarios: Tests launch real (headless or headed) browser instances via Playwright to navigate pages, interact with elements, and verify the outcomes of browser actions (e.g., filling forms, clicking buttons).
- HTML Parsing and Data Extraction: Tests will assert that the parsing logic correctly extracts expected data from specific HTML structures, often using snapshot testing or direct assertions on extracted objects.
- Stateless Operation Verification: Given that the
tools/service is stateless, tests will confirm that operations do not depend on or persist state across calls, aside from the active browser session. - Provider-Specific Logic: Tests will cover provider-specific implementations, such as those for SEEK, ensuring that their unique application flows are handled correctly (e.g.,
src/providers/seek/apply.ts).
Portal Testing
The portal/ service, a React application, uses a JavaScript testing framework (commonly Vitest or Jest) combined with React Testing Library to test its user interface components and client-side logic.
How to run Portal tests (inferred):
Navigate to the portal/ directory and use a Node.js test runner:
cd portal/
npm test
# or
yarn test
Key aspects of Portal tests (inferred):
- Component Rendering and Interaction: Tests verify that React components render correctly under various states and that user interactions (clicks, input changes) trigger the expected UI updates.
- API Integration Mocking: For components that interact with the
agent/API (e.g.,portal/src/pages/ApplyPage.tsx), tests will mock the API calls using tools likeMSW (Mock Service Worker)or direct mocks forreact-querymutations. This isolates the UI logic from the backend. - Routing Logic: Tests ensure that navigation and route parameters are handled correctly by
react-router-dom. - State Management: Tests verify how local and global state (e.g., using
useStateorreact-query) is managed and propagates through the application.