Email Testing in CI/CD Pipelines

Testing email functionality in CI/CD pipelines has traditionally been painful. You either skip email tests entirely, mock them (and miss real bugs), or deal with flaky tests against real email providers. Docmailer changes this.

The Problem with Email Testing in CI

Common approaches and their issues:

• Mocking emails — Doesn't catch real delivery issues
• Using test Gmail accounts — Rate limits, flaky, hard to automate
• Mailhog/Mailcatcher — Requires infrastructure, doesn't test real SMTP
• Skipping email tests — Bugs slip through to production

Docmailer Approach

Docmailer provides real, disposable email addresses with a simple API. Your CI pipeline can:

1. Create a fresh inbox before each test
2. Use that email in your signup/notification flow
3. Query the API to verify email arrival
4. Extract content (verification codes, links, etc.)
5. Clean up after the test

GitHub Actions Example

name: E2E Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Run E2E tests
        env:
          DOCMAILER_TOKEN: ${{ secrets.DOCMAILER_TOKEN }}
        run: |
          npm install
          npm run test:e2e

Test File Example (Playwright)

import { test, expect } from '@playwright/test';

const DOCMAILER_API = 'https://docmailer.org/api';
const TOKEN = process.env.DOCMAILER_TOKEN;

test('user signup with email verification', async ({ page }) => {
  // 1. Create test inbox
  const inboxRes = await fetch(`${DOCMAILER_API}/inbox`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${TOKEN}` }
  });
  const { inbox } = await inboxRes.json();
  
  // 2. Sign up with test email
  await page.goto('/signup');
  await page.fill('#email', inbox.email);
  await page.fill('#password', 'TestPass123!');
  await page.click('#submit');
  
  // 3. Wait for verification email
  await page.waitForTimeout(3000);
  
  const emailsRes = await fetch(
    `${DOCMAILER_API}/inbox/${inbox.id}/emails`,
    { headers: { 'Authorization': `Bearer ${TOKEN}` }}
  );
  const { emails } = await emailsRes.json();
  
  expect(emails.length).toBeGreaterThan(0);
  expect(emails[0].subject).toContain('Verify');
  
  // 4. Extract and use verification link
  const verifyLink = emails[0].html.match(/href="([^"]+verify[^"]+)"/)[1];
  await page.goto(verifyLink);
  
  await expect(page.locator('.success')).toBeVisible();
});

GitLab CI Example

e2e-tests:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0
  variables:
    DOCMAILER_TOKEN: $DOCMAILER_TOKEN
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    paths:
      - playwright-report/

Best Practices

1. One Inbox Per Test

Create a fresh inbox for each test to avoid cross-contamination:

test.beforeEach(async () => {
  inbox = await createDocmailerInbox();
});

test.afterEach(async () => {
  await deleteDocmailerInbox(inbox.id);
});

2. Use Retry Logic

Email delivery isn't instant. Build in retries:

async function waitForEmail(inboxId, maxAttempts = 10) {
  for (let i = 0; i < maxAttempts; i++) {
    const emails = await getEmails(inboxId);
    if (emails.length > 0) return emails[0];
    await new Promise(r => setTimeout(r, 1000));
  }
  throw new Error('Email not received');
}

3. Store Token Securely

Never commit your API token. Use CI secrets.