Revamping Invitation Tests For Better User Flow

by Alex Johnson 48 views

Invitation E2E workflow tests are crucial for ensuring a smooth user experience, particularly for features like project collaboration. However, the current tests were plagued with issues, primarily due to a mismatch between the test setup and the actual application architecture. This article delves into the problems, the fixes, and, most importantly, the recommended solution: a redesigned test architecture that aligns with how users interact with the invitation process.

The Problem: Mismatched Test and App Architecture

The core issue revolved around a fundamental disconnect. The existing tests were built on assumptions that didn't hold true in the real-world application. This led to frequent test failures, making it difficult to gauge the reliability of the invitation workflow. Let's explore the specific pain points.

Current Test Flow: A Broken Approach

The previous test flow, as outlined, suffered from significant flaws. It involved the following steps:

  1. Backend Invitation Creation: The test would create an invitation via the backend API, targeting a specific email address like accept-test@example.com. This step was designed to simulate the initial invitation from the system.
  2. UI Interaction with Invitation: The test would then attempt to add the invitation through the UI, using a function named addInvitationToken(token). This function was meant to mimic the user's action of inputting an invitation token.
  3. User Mismatch: This is where the issues truly began. The invitation was created for a different user (accept-test@example.com) than the user running the test. This meant that the current user couldn't see the invitation.
  4. Incomplete Data: Consequently, the invitation data appeared incomplete, often showing "NaN" days and an empty project name. This was a direct result of the mismatch.
  5. Timeout Frustration: Finally, the test would often time out, waiting indefinitely for the invitation row to appear in a table. This indicated that the test was unable to find the expected data, further highlighting the problem.

Root Causes: Unpacking the Issues

The problems extended beyond the superficial failures. Several root causes contributed to the test's instability:

  • User Mismatch: The primary culprit was the user mismatch. The backend API created invitations for a specific email, while the application's /api/v1/invitations/my endpoint filtered invitations based on the current user's email. Since the test user and the invitation recipient were different, the invitation never appeared in the test user's "My Invitations" section.
  • State Management Mismatch: The application loaded invitations from the /api/v1/invitations/my endpoint upon page load. The handleAddInvitation function then added invitations to the React state via token lookup. Tests accumulated many old invitations, and clearing local storage helped a bit, but didn't solve the core user mismatch.
  • Test Infrastructure vs. App Design: The tests were trying to create and receive invitations with a single user, which conflicted with the app's design. The actual user workflow involves an admin creating an invitation, sending a token, and a recipient adding the token. The tests didn't match real-world usage patterns.

What We Fixed: Initial Improvements

Before tackling the architectural issues, initial steps were taken to improve the test infrastructure. Commit 90277c073 brought about enhancements that addressed some of the symptoms, but not the root cause.

  • Deterministic Waits: The addInvitationToken() function now used deterministic waits, waiting for a specific project row to appear. This eliminated some of the timing-related issues.
  • Visibility Waits: The acceptInvitation() function was updated to explicitly wait for the invitation to become visible before proceeding.
  • Clean Slate: beforeEach was implemented to clear localStorage and reload the page, ensuring a clean state for each test run, mitigating the accumulation of stale data.
  • Reduced Timeout Reliance: The reliance on brittle, timeout-based waiting was minimized.

These were all good improvements, but they didn't fully resolve the fundamental user mismatch issue.

Solution: A Redesigned Test Architecture

The most effective approach involves redesigning the test architecture to align with the actual user flow. Several options were considered, each with its strengths and weaknesses.

Option A: Multi-User Test Flow

This option tests the actual admin-to-recipient flow, offering a comprehensive approach. It involves the following steps:

  1. Admin Context: Create an admin context using the browser.newContext({ storageState: 'admin-auth.json' }) method, simulating an authenticated admin user.
  2. Project Creation: Using the admin context, create a project.
  3. Invitation Sending: The admin sends an invitation to a test user using sendInvitation(). The test gets a token.
  4. User Context: Create a user context simulating an authenticated test user.
  5. Token Addition: The test user adds the invitation via token, using the token generated in the previous step.
  6. Invitation Acceptance: The test user accepts the invitation.
  7. Verification: Verify that the user is now a project member.

Option B: Single-User Self-Invitation

This option focuses on a simpler test flow, ideal for iterative testing. The following steps are involved:

  1. User Email Retrieval: Retrieve the current user's email using a hypothetical getCurrentUserEmail() method within the application.
  2. Project Creation: Create a project using createProject().
  3. Self-Invitation: Send an invitation to the current user's email address using sendInvitation(). This simulates the user inviting themselves.
  4. Invitation Navigation: Navigate to the invitation page, possibly reloading the page to fetch updated invitations.
  5. Invitation Visibility: Verify that the invitation appears automatically, without requiring the addInvitationToken() step, since the invitation is already associated with the current user.
  6. Acceptance: Accept the invitation.

Option C: Mock Backend Responses

This option uses Playwright's route mocking feature, offering a faster testing experience by simulating the backend responses.

  1. Mocking the API: Use page.route to intercept requests to /api/v1/invitations/my and return a mock response containing a test invitation.
  2. Navigation and Interaction: Navigate to the invitation page, then test the UI interactions, such as accepting the invitation.

Recommended Approach: Single-User Self-Invitation

After careful consideration, Option B (Single-User Self-Invitation) is the most suitable approach for the following reasons:

  • Simplicity: It streamlines the test setup by eliminating the need for multi-user authentication, making the tests easier to write and maintain.
  • API Integration: The tests directly interact with the actual API, which ensures that the tests accurately reflect the application's behavior.
  • Real-World Matching: The test flow directly mirrors how invitations are loaded from /api/v1/invitations/my. This reduces the chances of issues arising from different data loading approaches.
  • Token Input Elimination: The invitations appear automatically, reducing the complexity of the test and removing the need for UI interaction via a token.
  • React State Testing: The tests effectively validate React state management, as they ensure that invitations are correctly loaded and displayed based on the current user's context.

Implementation Checklist: Steps to Success

To ensure a smooth transition to the new test architecture, the following steps must be completed:

  • Implement getCurrentUserEmail(): Add a getCurrentUserEmail() method to the SafePrismAPI to retrieve the current user's email address.
  • Update Test Helpers: Update existing test helpers to send invitations to the current user, rather than a hardcoded email.
  • Remove addInvitationToken(): Eliminate the addInvitationToken() function from the tests, simplifying the invitation process.
  • Add Page Reload: Add an explicit page reload after sending the invitation to ensure that the updated invitations are fetched.
  • Update Failing Tests: Update all 19 failing invitation tests with the new single-user pattern.
  • Test Cleanup: Implement test cleanup to delete test invitations after each test run, preventing data accumulation and maintaining a clean state.
  • Documentation: Update the TESTING.md file to document the new invitation testing pattern. This will help developers understand the new process.

Related Files: Key Areas of Concern

The following files are crucial for the implementation and understanding of the new test architecture:

  • cmd/prism-gui/frontend/tests/e2e/invitation-workflows.spec.ts: The primary test file, where the invitation tests are defined.
  • cmd/prism-gui/frontend/tests/e2e/pages/ProjectsPage.ts:665: Contains the test helpers used in the invitation tests.
  • cmd/prism-gui/frontend/src/App.tsx:4968: The handleAddInvitation() method, responsible for adding invitations. This file will be impacted by the changes.

Session Summary: A Recap

Commit: 90277c073 focused on test infrastructure improvements, introducing deterministic waits. The tests still failed because of a mismatch.

Key Discovery: The primary issue isn't about timeouts or UI issues, but a fundamental mismatch between the test's approach and the application's design.

Impact: 19 out of 28 invitation tests were failing due to this fundamental user-mismatch.

By adopting the recommended single-user self-invitation approach and making the required code changes, these tests can be refactored to align with the application architecture and provide reliable validation of the invitation workflow.

To learn more about Playwright and testing best practices, visit the Playwright documentation.