diff --git a/packages/ui/docs/places-llm-filters.md b/packages/ui/docs/places-llm-filters.md index 9a65261e..bdda7846 100644 --- a/packages/ui/docs/places-llm-filters.md +++ b/packages/ui/docs/places-llm-filters.md @@ -31,7 +31,9 @@ This document captures the current codebase touchpoints and a proposed architect - **On**: full `competitors` list + **seeded MUI column filters** on `types` via `seedExcludeTypeFilters={excludedTypes}` (`PlacesGridView` / `useGridColumns` custom operator `excludesAnyOf`). - **Saved excludes** are loaded/stored via **`/api/me/secrets`** as `gridsearch_exclude_types` (see `client-gridsearch.ts` and `PlacesLibrary.postPlacesGridSearchExpand` reading `getUserSecrets`). -**Implication:** “AI Filters” should **compose** with this: same DataGrid, extra **dynamic columns** whose `field` names match **target fields** stored under `meta`. +**Target UX:** Introduce a **`gridFiltersOpen`** (name TBD) collapsible **below** the main toolbar row—same structural pattern as **`aiContextOpen`** + violet panel (see ~`{aiContextOpen && (…textarea…)}`). **Grid filters** becomes the parent affordance: expanded panel holds **type-exclude / MUI hint copy** and the **AI filter editor** (not a duplicate top-level “AI” entry). Today the **Sparkles** control sits beside view modes; plan is to **merge** grid-related AI into this **Grid filters** section. + +**Implication:** “AI Filters” **compose** with existing grid behavior: same DataGrid, extra **dynamic columns** whose `field` names match **target fields** stored under `meta`. ### 2.2 MUI grid columns (`useGridColumns.tsx`, `PlacesGridView.tsx`) @@ -41,8 +43,8 @@ This document captures the current codebase touchpoints and a proposed architect ### 2.3 Place LLM today (`server/src/products/places/llm.ts`, `place-info.md`) -- `GET` handler resolves **`runId`** / manual **`context`**, builds `${grid_search_context}`, runs kbot completion, returns JSON including **`potentialCustomer`** (1–5) when context exists. -- **Planned product change:** drop `potentialCustomer` from this response and prompt; scoring moves to **per-filter** outputs stored in **`meta`**, not the summary tab’s single score. +- `GET` handler resolves **`runId`** / manual **`context`**, builds `${grid_search_context}`, runs kbot completion, returns summary JSON for the place detail tab. +- **Product direction:** campaign fit moves to **per-filter** outputs in **`meta.llm_grid_filters`**, not a dedicated **`potentialCustomer`** field on the summary response (removed from prompt/UI per §7). ### 2.4 `places.meta` (`server/src/products/places/db-places.ts`, `places.ts`) @@ -118,6 +120,8 @@ meta.llm_grid_filters = { ## 4. Server API sketch +**Home for handlers:** implement in **`server/src/products/places/llm.ts`** (re-export or register from `PlacesProduct` like existing LLM routes). Keeps kbot tasks, Zod `format`, and prompt path helpers in one place. + | Endpoint | Purpose | |----------|---------| | `POST /api/places/llm-filters/run` | Body: `{ placeIds: string[], filterIds: string[], jobId?: string }`. Enqueues work, returns `{ runId }`. | @@ -203,18 +207,22 @@ This avoids proliferating files while keeping **consistent** privacy rules and * --- -## 6. Frontend refactor (reduce `GridSearchResults.tsx` size) +## 6. Frontend (after server): Grid Filters panel + components + +**Pattern:** Reuse the **Context** editor UX: a **toolbar button** toggles open state; when open, a **full-width panel** under the toolbar (bordered card, compact labels) holds forms. Apply the same for **AI filters** inside the **Grid filters** region—**not** a separate floating section next to view modes. Suggested new files under `src/modules/places/gridsearch/`: | Module | Responsibility | |--------|------------------| +| `GridFiltersPanel.tsx` (or split) | Collapsible content for **Grid filters**: type-exclude explanation, **AI filter list** (add/remove/enable), prompts, target field, run actions. Consumed when `gridFiltersOpen` (or merged state with `applySavedExcludesInGrid`). | | `aiGridFiltersState.ts` | Load/save defs from secrets; enabled set per `jobId`; localStorage helpers if used. | | `useAiGridColumns.ts` | Build `GridColDef[]` from enabled defs + `valueGetter` from `meta.llm_grid_filters`. | -| `AiGridFiltersToolbar.tsx` | Add/remove/enable filters, “Run on selection / run on all” actions. | | `PlacesGridView.tsx` (or wrapper) | Accept **optional** `extraColumns: GridColDef[]` merged after static columns. | -`GridSearchResults.tsx` keeps: layout, view mode, wiring **props** into `CompetitorsGridView`. +`GridSearchResults.tsx` stays layout-only: view modes, **`GridFiltersPanel`** slot, **`CompetitorsGridView`** props. + +**Phase order:** build **Grid Filters** collapsible + editor **after** endpoints exist (mock with static data if needed); add **MUI dynamic columns** once `meta.llm_grid_filters` flows from the server. --- @@ -232,17 +240,16 @@ Suggested new files under `src/modules/places/gridsearch/`: - **MUI single-filter constraint:** adding many AI columns may interact badly with **community** grid limits; may need to **lift** filter encoding (documented in `useGridColumns`) when AI columns need filtering simultaneously with types. - **Meta size:** cap string length per cell; strip `raw` in list views if large. - **Cost / abuse:** rate limits per user; optional confirmation before “run on all.” - -yeah, we take care of those but good point, we may disable certain columns for public shared searches +- **Public / shared grid searches:** hide or disable **AI filter** columns and editor affordances for viewers when `is_public` and `!isOwner` (same spirit as hiding sensitive tools); only owner runs LLM fills that write `meta`. --- -## 9. Suggested implementation phases +## 9. Suggested implementation phases (matches § order at top) -1. **Schema & types:** secrets shape + `meta.llm_grid_filters` TypeScript interfaces; **`AiGridFilterResultSchema`** (Zod) + **`zodResponseFormat`** wired into kbot `run()` tasks (pm-pics + shared if exported). -2. **Server:** single-place single-filter endpoint (debug) using **`format`** → validate with **`.parse()`** → batch + SSE → `updatePlaceMeta` merge. -3. **Client:** secrets editor + dynamic columns + run button; subscribe SSE and patch row state / refetch place. -4. **Cleanup:** remove `potentialCustomer` from place-info pipeline; align docs : done +1. **Server (`llm.ts` + routes):** Zod **`AiGridFilterResultSchema`** + **`zodResponseFormat`**; single-place single-filter handler (debug); **`POST …/run`** + **`GET …/stream`** (or EventBus bridge); **`updatePlaceMeta`** merge with **`promptHash`** checks. +2. **Client — Grid Filters UI:** **`GridFiltersPanel`** (or equivalent); filter editor **like Context** (collapsible under **Grid filters**); wire secrets **`PATCH /api/me/secrets`** for definitions; run actions calling batch API + SSE. +3. **Client — MUI:** **`useAiGridColumns`** + **`CompetitorsGridView`** `extraColumns`; row refresh on SSE / refetch. +4. **Product cleanup:** `potentialCustomer` removal from place-info pipeline — **done** where noted in §1 / §7. --- @@ -253,7 +260,8 @@ yeah, we take care of those but good point, we may disable certain columns for p | Grid toggle & toolbar | `src/modules/places/gridsearch/GridSearchResults.tsx` | | Column defs | `src/modules/places/useGridColumns.tsx`, `PlacesGridView.tsx` | | Secrets (exclude types) | `src/modules/places/client-gridsearch.ts`, `server/.../places/places.ts` | -| Place LLM | `server/src/products/places/llm.ts`, `server/data/products/places/llm/place-info.md` | +| Place LLM & **new grid-filter LLM** | `server/src/products/places/llm.ts` (handlers + kbot tasks), `server/data/products/places/llm/place-info.md`, `grid-filter.md` (TBD) | +| Grid Filters UI / Context pattern | `src/modules/places/gridsearch/GridSearchResults.tsx` (toolbar; collapse under Grid filters per §6) | | SSE | `server/src/products/places/index.ts` (`handleGetPlacesGridSearchStream`), `server/src/products/places/routes.ts` | | Meta persistence | `server/src/products/places/db-places.ts` (`updatePlaceMeta`) | | Shared place type | `packages/ui/shared/src/competitors/schemas.ts` (`PlaceSchemaFull`) | @@ -263,4 +271,4 @@ yeah, we take care of those but good point, we may disable certain columns for p --- -*Draft for review; no implementation committed in this step.* +*Living plan — implementation follows server (`llm.ts`) → Grid Filters panel + editor → MUI columns.* diff --git a/packages/ui/src/modules/places/LocationDetail.tsx b/packages/ui/src/modules/places/LocationDetail.tsx index 7eaf4bf0..e9d94c20 100644 --- a/packages/ui/src/modules/places/LocationDetail.tsx +++ b/packages/ui/src/modules/places/LocationDetail.tsx @@ -15,15 +15,12 @@ export const LocationDetailView = React.memo(({ onClose, livePhotos, gridSearchRunId, - potentialCustomerContext, }: { competitor: PlaceFull; onClose?: () => void; livePhotos?: any; - /** When set (e.g. grid search job id), server loads grid_search_runs.request for campaign fit scoring */ + /** When set (e.g. grid search job id), server loads grid_search_runs.request for place summary context */ gridSearchRunId?: string | null; - /** When set, overrides grid run JSON for Potential customer scoring (user “Context” in grid search UI) */ - potentialCustomerContext?: string | null; }) => { const [lightboxOpen, setLightboxOpen] = useState(false); const [lightboxIndex, setLightboxIndex] = useState(0); @@ -61,7 +58,6 @@ export const LocationDetailView = React.memo(({ fetchPlaceLlmSummary(competitor.place_id, { lang, runId: gridSearchRunId || undefined, - context: potentialCustomerContext?.trim() || undefined, }) .then((data) => { if (!cancelled) setSummaryData(data); @@ -75,7 +71,7 @@ export const LocationDetailView = React.memo(({ return () => { cancelled = true; }; - }, [activeTab, competitor.place_id, gridSearchRunId, potentialCustomerContext]); + }, [activeTab, competitor.place_id, gridSearchRunId]); // Prefer prop-injected, then dynamically fetched, then DB-cached const photoSource = livePhotos || fetchedPhotos || competitor.raw_data?.google_media; diff --git a/packages/ui/src/modules/places/PlacesGridView.tsx b/packages/ui/src/modules/places/PlacesGridView.tsx index e5ec6050..62518110 100644 --- a/packages/ui/src/modules/places/PlacesGridView.tsx +++ b/packages/ui/src/modules/places/PlacesGridView.tsx @@ -47,6 +47,8 @@ function rowHasLeadEmail(row: any): boolean { export type CompetitorsGridViewHandle = { /** Row IDs (place ids) that pass the current grid filter model and quick filter. */ getFilteredPlaceIds: () => string[]; + /** Currently selected row IDs (checkbox selection). */ + getSelectedPlaceIds: () => string[]; }; interface PlacesGridViewProps { @@ -63,6 +65,8 @@ interface PlacesGridViewProps { seedExcludeTypeFilters?: string[]; /** When false, filter/sort/pagination changes are not written to the URL (avoids collisions with multiple `types` filters). */ persistFiltersInUrl?: boolean; + /** Appended after base columns (e.g. AI grid filters). */ + extraColumns?: GridColDef[]; } const CustomToolbar = ({ selectedCount }: { selectedCount: number }) => { @@ -97,6 +101,7 @@ export const CompetitorsGridView = forwardRef filteredRowsLookup[id] !== false) .map((id) => String(id)); }, - }), []); + getSelectedPlaceIds: () => [...selectedRows], + }), [selectedRows]); // Sync local highlighted state with global selectedPlaceId useEffect(() => { @@ -209,20 +215,66 @@ export const CompetitorsGridView = forwardRef [...baseColumns, ...(extraColumns || [])], + [baseColumns, extraColumns], + ); + + /** Apply min/full preset only when `preset` actually changes — not on remount (keeps URL column visibility / order). */ + const presetVisibilityBootstrappedRef = useRef(false); + const lastPresetRef = useRef(null); useEffect(() => { + if (!presetVisibilityBootstrappedRef.current) { + presetVisibilityBootstrappedRef.current = true; + lastPresetRef.current = preset; + return; + } + if (lastPresetRef.current === preset) return; + lastPresetRef.current = preset; const presetModel = getPresetVisibilityModel(preset); - setColumnVisibilityModel(prev => ({ - ...prev, - ...presetModel - })); + setColumnVisibilityModel((prev) => ({ ...prev, ...presetModel })); }, [preset]); + /** Keep filter model in sync when switching “saved excludes as grid filters” without remounting (preserves columns). */ + const filterSyncKey = React.useMemo(() => searchParams.toString(), [searchParams]); + useEffect(() => { + if (seedExcludeTypeFilters?.length) { + const list = seedExcludeTypeFilters.map((t) => String(t).trim()).filter(Boolean); + setFilterModel({ + items: list.length + ? [ + { + field: 'types', + operator: GRID_TYPES_FILTER_EXCLUDES_ANY_OF, + value: JSON.stringify(list), + id: 'seed-excludes-types', + }, + ] + : [], + logicOperator: GridLogicOperator.And, + }); + return; + } + if (!persistFiltersInUrl) { + setFilterModel({ items: [], logicOperator: GridLogicOperator.And }); + return; + } + const fromUrl = paramsToFilterModel(searchParams); + let items = [...(fromUrl.items || [])]; + if (items.length === 0 && !searchParams.has('nofilter')) { + const shouldHideEmptyEmails = !isPublic || isOwner; + if (shouldHideEmptyEmails) { + items.push({ field: 'email', operator: 'isNotEmpty', id: 'default-email' }); + } + } + setFilterModel({ items, logicOperator: GridLogicOperator.And }); + }, [seedExcludeTypeFilters, persistFiltersInUrl, filterSyncKey, isPublic, isOwner, searchParams]); + // Update URL when filter model changes const handleFilterModelChange = (newFilterModel: GridFilterModel) => { setFilterModel(newFilterModel); diff --git a/packages/ui/src/modules/places/client-gridsearch.ts b/packages/ui/src/modules/places/client-gridsearch.ts index f652d627..d696aa88 100644 --- a/packages/ui/src/modules/places/client-gridsearch.ts +++ b/packages/ui/src/modules/places/client-gridsearch.ts @@ -248,6 +248,52 @@ export const saveGridSearchExcludeTypes = async (types: string[]): Promise } }; +/** User-defined AI grid filters (stored in user_secrets.settings.places_ai_grid_filters). */ +export interface PlacesAiGridFilterDef { + id: string; + label: string; + prompt: string; + targetField: string; + enabled?: boolean; + /** How to coerce LLM output for the grid (`auto` = infer numeric strings + optional LLM `value_type`). */ + valueType?: 'auto' | 'number' | 'string'; +} + +export const getPlacesAiGridFilters = async (): Promise => { + try { + const res = await apiClient('/api/me/secrets'); + const raw = res?.places_ai_grid_filters; + return Array.isArray(raw) ? raw : []; + } catch { + return []; + } +}; + +export const savePlacesAiGridFilters = async (filters: PlacesAiGridFilterDef[]): Promise => { + await apiClient('/api/me/secrets', { + method: 'PUT', + body: JSON.stringify({ places_ai_grid_filters: filters }), + }); +}; + +export const postLlmFiltersRun = async (body: { + placeIds: string[]; + filterIds: string[]; + lang?: string; +}): Promise<{ runId: string }> => { + const res = await apiClient<{ data: { runId: string } }>('/api/places/llm-filters/run', { + method: 'POST', + body: JSON.stringify(body), + }); + return res.data; +}; + +export const getLlmFiltersStreamUrl = (runId: string, token?: string | null): string => { + const q = new URLSearchParams({ runId }); + if (token) q.set('token', token); + return `${serverUrl}/api/places/llm-filters/stream?${q.toString()}`; +}; + export const getPlacesTypes = async (): Promise => { try { const res = await apiClient<{ data: string[] }>('/api/places/types'); diff --git a/packages/ui/src/modules/places/gridsearch/GridFiltersPanel.tsx b/packages/ui/src/modules/places/gridsearch/GridFiltersPanel.tsx new file mode 100644 index 00000000..4b433490 --- /dev/null +++ b/packages/ui/src/modules/places/gridsearch/GridFiltersPanel.tsx @@ -0,0 +1,286 @@ +import React, { useCallback, useEffect, useState } from 'react'; +import { Loader2 } from 'lucide-react'; +import { getAuthToken } from '@/lib/db'; +import { + fetchCompetitorById, + getPlacesAiGridFilters, + postLlmFiltersRun, + getLlmFiltersStreamUrl, + savePlacesAiGridFilters, + type PlacesAiGridFilterDef, +} from '../client-gridsearch'; +import type { CompetitorsGridViewHandle } from '../PlacesGridView'; +import { T, translate } from '@/i18n'; + +function newFilter(): PlacesAiGridFilterDef { + return { + id: crypto.randomUUID(), + label: '', + prompt: '', + targetField: `ai_${crypto.randomUUID().slice(0, 8)}`, + enabled: true, + valueType: 'auto', + }; +} + +type GridFiltersPanelProps = { + gridExportRef: React.RefObject; + getPlaceIdsAll: () => string[]; + onPlaceLlmUpdated: (placeId: string) => void; + /** When false (e.g. public viewer), hide run + editor. */ + canEdit: boolean; + /** After definitions are saved to secrets, parent can refresh column defs. */ + onFiltersPersisted?: () => void; +}; + +export const GridFiltersPanel: React.FC = ({ + gridExportRef, + getPlaceIdsAll, + onPlaceLlmUpdated, + canEdit, + onFiltersPersisted, +}) => { + const [defs, setDefs] = useState([]); + const [loading, setLoading] = useState(true); + const [saving, setSaving] = useState(false); + const [running, setRunning] = useState(false); + const [runStatus, setRunStatus] = useState(null); + + useEffect(() => { + let cancelled = false; + (async () => { + try { + const d = await getPlacesAiGridFilters(); + if (!cancelled) setDefs(d); + } finally { + if (!cancelled) setLoading(false); + } + })(); + return () => { + cancelled = true; + }; + }, []); + + const persistDefs = useCallback( + async (next: PlacesAiGridFilterDef[]) => { + setSaving(true); + try { + await savePlacesAiGridFilters(next); + setDefs(next); + onFiltersPersisted?.(); + } catch (e) { + console.error(e); + setRunStatus(String((e as Error)?.message || e)); + } finally { + setSaving(false); + } + }, + [onFiltersPersisted], + ); + + const addFilter = () => { + void persistDefs([...defs, newFilter()]); + }; + + const removeFilter = (id: string) => { + void persistDefs(defs.filter((d) => d.id !== id)); + }; + + const patchFilter = (id: string, patch: Partial) => { + setDefs((prev) => prev.map((d) => (d.id === id ? { ...d, ...patch } : d))); + }; + + const saveDraft = () => { + void persistDefs(defs); + }; + + const enabledFilterIds = defs.filter((d) => d.enabled !== false).map((d) => d.id); + + const runBatch = async (placeIds: string[]) => { + if (!canEdit || placeIds.length === 0 || enabledFilterIds.length === 0) return; + setRunning(true); + setRunStatus(null); + let es: EventSource | null = null; + try { + const { runId } = await postLlmFiltersRun({ + placeIds, + filterIds: enabledFilterIds, + lang: typeof navigator !== 'undefined' ? navigator.language?.slice(0, 2) : undefined, + }); + const token = await getAuthToken(); + const url = getLlmFiltersStreamUrl(runId, token); + es = new EventSource(url); + + es.addEventListener('message', (ev: MessageEvent) => { + try { + const p = JSON.parse(ev.data) as { + kind?: string; + placeId?: string; + filterId?: string; + error?: string; + }; + if (p.kind === 'cell' && p.placeId) { + onPlaceLlmUpdated(p.placeId); + } + if (p.kind === 'error') { + setRunStatus(p.error || 'Error'); + } + } catch { + /* ignore */ + } + }); + + es.addEventListener('complete', () => { + setRunStatus(translate('AI filters run finished')); + es?.close(); + es = null; + setRunning(false); + }); + + es.onerror = () => { + setRunning(false); + es?.close(); + es = null; + }; + } catch (e) { + console.error(e); + setRunStatus(String((e as Error)?.message || e)); + setRunning(false); + es?.close(); + } + }; + + const handleRunAll = () => { + void runBatch(getPlaceIdsAll()); + }; + + const handleRunSelected = () => { + const api = gridExportRef.current; + const selected = api?.getSelectedPlaceIds?.() ?? []; + const ids = selected.length > 0 ? selected : getPlaceIdsAll(); + void runBatch(ids); + }; + + if (!canEdit) { + return ( +

+ AI grid filters are available to the search owner. +

+ ); + } + + if (loading) { + return ( +
+ + Loading filters… +
+ ); + } + + return ( +
+
+ + + + +
+ + {runStatus &&

{runStatus}

} + +
+ {defs.map((d) => ( +
+
+ + patchFilter(d.id, { label: e.target.value })} + placeholder={translate('Label')} + /> + + patchFilter(d.id, { + targetField: e.target.value.replace(/[^a-z0-9_]/gi, '_').toLowerCase(), + }) + } + placeholder="ai_field" + /> + + +
+