Files
supabase-postgres-best-prac…/packages/evals/docker-entrypoint.sh
Pedro Rodrigues 9b08864e94 feat(evals): replace mock CLIs with real Supabase instance per eval run
Start a shared local Supabase stack once before all scenarios and reset
the database (drop/recreate public schema + clear migration history) between
each run. This lets agents apply migrations via `supabase db push` against a
real Postgres instance instead of mock shell scripts.

- Add supabase-setup.ts: startSupabase / stopSupabase / resetDB / getKeys
- Update runner.ts to start/stop Supabase and inject keys into process.env
- Update agent.ts to point MCP config at the local Supabase HTTP endpoint
- Update preflight.ts to check supabase CLI availability and Docker socket
- Update scaffold.ts to seed workspace with supabase/config.toml
- Add passThreshold support (test.ts / results.ts / types.ts) for partial pass
- Delete mock shell scripts (mocks/docker, mocks/psql, mocks/supabase)
- Update Dockerfile/docker-compose to mount Docker socket for supabase CLI

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 14:39:54 +00:00

25 lines
755 B
Bash
Executable File

#!/usr/bin/env bash
# Entrypoint for the eval Docker container.
# Validates environment, then runs the given command.
set -euo pipefail
export IN_DOCKER=true
# Validate required env
if [[ -z "${ANTHROPIC_API_KEY:-}" ]]; then
echo "ERROR: ANTHROPIC_API_KEY is not set." >&2
echo "Pass it via: docker run -e ANTHROPIC_API_KEY=sk-ant-... ..." >&2
exit 1
fi
echo "=== Eval Environment ==="
echo " Node: $(node --version)"
echo " Claude: $(claude --version 2>/dev/null || echo 'n/a')"
echo " Supabase: $(supabase --version 2>/dev/null || echo 'n/a')"
echo " Docker: $(docker --version 2>/dev/null || echo 'n/a')"
echo " Model: ${EVAL_MODEL:-default}"
echo " Scenario: ${EVAL_SCENARIO:-all}"
echo "========================"
exec "$@"