AI and ML engineering
Building with models: LLM apps, RAG, evals, agents, training and MLOps.
Download all 102
Start here 1
The general, start-from-zero entry for a need in this category: it works from a blank start, asks for what it needs and hands off to the specific entries. Example: Plan a trip.
- Add an AI feature to an app
Turns an AI feature idea into a buildable design, choosing the simplest approach that works and planning evals, guardrails, cost and build steps. Use when you want to add AI or an LLM to your product.
Specific 77
One situation, audience, tool or format inside the category. Example: Plan a road trip with children.
- Plan a machine-learning experiment
Plans a machine-learning experiment before any training code exists: framing, baselines, leak-proof splits, metrics, ablations and a stop rule. Use when starting a new model or modelling spike.
- Write a model card
Writes a model card with intended use, training data, metrics by slice, limitations and ethical considerations from training notes and eval results, flagging gaps. Use before releasing a model.
- Review a training dataset sample
Audits a sample of a labelled dataset for label noise, leakage, duplicates, class imbalance and representation gaps, and gives a concrete fix for each problem. Use before training or fine-tuning.
- Build an MCP server
Implements a Model Context Protocol server exposing the given tools and resources, with input validation, least privilege and error messages a model can act on. Use to connect a system to AI clients.
- Design a RAG pipeline
Designs a retrieval-augmented generation pipeline from a corpus and its real questions, covering chunking, hybrid retrieval, reranking, citations and evals. Use before building or rebuilding RAG.
- Build an LLM structured extraction step
Builds an LLM step that turns documents into schema-valid JSON, with the schema, prompt, validation and repair loop, null handling and an eval set. Use when automating invoices, forms or emails.
- Design an LLM agent architecture
Designs an LLM agent system, deciding first whether an agent is needed, then single or multi-agent, tools, memory, guardrails, human checkpoints, evals and cost limits.
- Reduce LLM costs and latency
Cuts an LLM feature's cost and latency through prompt trimming, caching, model routing, batching and output limits, each paired with the quality check that proves nothing regressed.
- Choose between rules, ML and an LLM
Recommends rules, classical ML, a hosted LLM or a fine-tuned model for a problem, comparing accuracy, cost, latency and maintenance with the reasoning shown. Use before committing to an approach.
- Implement LLM tool calling
Implements tool calling in an LLM feature with tool schemas, a dispatch loop, argument validation, timeouts, limits and safe error handling. Use when wiring a model to functions or APIs.
- Evaluate retrieval quality
Builds a labelled query set and a script that measures a search or RAG retriever's recall@k, MRR and nDCG separately from answer quality, with per-query-type breakdowns and failure lists.
- Design tool definitions for an LLM agent
Designs tool or function definitions for an LLM agent, with names, descriptions, JSON Schema parameters and error returns that models call reliably. Use when exposing an API or capability to an agent.
- Write an eval suite for an LLM feature
Writes an eval set for an LLM feature with golden, edge and adversarial cases, graders matched to each criterion, and pass thresholds. Use before shipping or changing a model, prompt or pipeline.
- Build a recommendation system
Builds a recommender from a popularity baseline to item-item or learned models, with time-based offline evaluation, cold-start handling and business rules, before any online test.
- Choose an embedding model
Chooses an embedding model for search or RAG by shortlisting candidates on hard constraints, then testing them on the user's own queries for recall, latency, cost and storage.
- Choose a vector database
Picks a vector store (a Postgres extension, a search engine's vector support, a managed service or a dedicated database) from data size, filtering, update rate, ops skills and cost.
- Debug poor RAG answers
Diagnoses bad answers from a RAG system by checking each stage in order (corpus, chunking, retrieval, ranking, context assembly, generation) on the failing questions, with a targeted fix per cause.
- Implement structure-aware document chunking
Implements chunking for PDF, HTML and Markdown documents that splits on structure, keeps heading paths and metadata on each chunk, handles tables and code, and is tested on real documents.
- Implement streaming LLM responses
Implements streaming LLM responses end to end, from provider stream to server-sent events to UI, with cancellation, timeouts and mid-stream errors. Use when replies feel slow to start.
- Prepare a fine-tuning dataset
Converts raw examples into a clean chat-format fine-tuning dataset with deduplication, quality and safety filters, token-length checks, leakage-free splits and a data card.
- Fine-tune an image classifier
Fine-tunes a pretrained image classifier on a custom dataset with leak-free splits, suitable augmentation, class balance, per-class metrics and a misclassification review.
- Train a text classifier
Trains a text classifier starting from a TF-IDF and logistic regression baseline up to a fine-tuned transformer only if needed, with per-class metrics and an error review.
- Analyze LLM failures into a taxonomy
Reads a sample of failing LLM traces or eval outputs, codes each failure, groups them into a failure taxonomy with counts and picks the fixes worth making first.
- Build an AI chat interface
Builds the frontend of an LLM chat feature - message model, composer, safe streaming Markdown, stop, retry, edit and regenerate, scroll behaviour, conversation list and error states.
- Build image similarity and duplicate search
Builds image similarity or near-duplicate search with perceptual hashes and learned embeddings, a vector index, and a threshold calibrated on labelled pairs.
- Build a scikit-learn preprocessing pipeline
Builds a scikit-learn Pipeline with ColumnTransformer for missing values, encoding and scaling that is fitted only on training data, saved as one object and tested on messy input.
- Build a speech-to-text transcription pipeline
Builds a speech-to-text pipeline with Whisper-class models: audio normalisation, silence-aware chunking, diarisation, word timestamps, hallucination guards and a word error rate check.
- Choose a classifier's decision threshold
Picks a classifier's decision threshold from the cost of each error or a capacity limit, using held-out predictions, calibration checks and per-segment results, instead of defaulting to 0.5.
- Clean up an automatic speech transcript
Cleans an automatic speech transcript by fixing punctuation, casing, obvious misrecognitions and speaker labels while keeping the words faithful and marking every uncertain fix.
- Decompose a complex question into sub-questions
Breaks a multi-hop, comparative or aggregate question into ordered sub-questions with dependencies and a composition step, so a retrieval or agent system can answer each one first.
- Grade a response against a rubric
Grades a response against a rubric one criterion at a time, quotes the evidence behind each score, and returns structured scores with an overall pass or fail. Use for LLM evals and automated marking.
- Implement chat history and context management
Implements chat history storage and context-window assembly for an LLM chat feature, with token budgets, trimming, summarisation triggers, per-user isolation and retention rules.
- Implement hybrid keyword and vector search
Implements hybrid retrieval that runs keyword (BM25) and vector search, merges them with reciprocal rank fusion and shared filters, and proves on a query set that it beats each alone.
- Moderate user content against your policy
Classifies user-generated content against a platform policy the operator supplies, returning violated clauses, severity, quoted evidence and a recommended action. Use as an LLM moderation step.
- Move prompts into versioned templates
Moves LLM prompts hard-coded in application strings into versioned template files with declared variables, a loader, render tests and a prompt version logged on every call.
- Rerank retrieved passages by relevance
Scores retrieved passages for how well they answer a query and returns a ranked list with graded relevance and a one-line reason, flagging when none are relevant. Use as an LLM reranker in RAG.
- Review a production prompt change
Reviews a pull request that edits a production LLM prompt for behaviour regressions, broken output contracts, injection exposure, cost changes and missing eval evidence, with a merge verdict.
- Set up ML experiment tracking
Adds MLflow or Weights and Biases tracking to training code so every run records parameters, metrics, code commit, data version, artifacts and registered models.
- Suggest follow-up questions after an answer
Suggests short follow-up questions a user might ask next, grounded in the last answer and the app's scope, avoiding repeats and out-of-scope topics. Use for suggestion chips in chat apps.
- Set up a hyperparameter search
Sets up a hyperparameter search with Optuna or the project's tuner: a sane search space, pruning, a fixed budget, resumable storage and a held-out check that the gain is real.
- Write a PyTorch training loop
Writes a PyTorch training loop with mixed precision, gradient clipping, checkpointing that can resume, seeding, validation and early stopping, plus an overfit-one-batch test.
- Summarise with increasing density
Produces a series of same-length summaries of one document, each adding salient entities the previous one missed while staying faithful, so an application can pick the density it needs.
- Check an answer's faithfulness to its sources
Splits an answer into atomic claims, labels each as supported, contradicted or not found in the sources, and returns a faithfulness score with the unsupported claims. Use to catch hallucinations.
- Compress a conversation into a carry-over state
Compresses a long chat history into a compact state summary of goals, decisions, constraints, open questions and user facts, to carry into a fresh context window without losing what matters.
- Convert a model to ONNX and verify it
Exports a PyTorch or scikit-learn model to ONNX with dynamic shapes, verifies numerical parity on real inputs with ONNX Runtime, and benchmarks inference speed.
- Convert a question into safe read-only SQL
Turns a natural-language question into one bounded, read-only SQL query for a given schema, asking for clarification on ambiguity and refusing writes or unbounded scans. Use in text-to-SQL features.
- Critique a draft against requirements and revise it
Critiques a generated draft against stated requirements, lists concrete defects by severity, then produces a revision that fixes them without adding unsupported content. Use as a self-refine step.
- Decide whether an assistant answer needs a human
Decides whether an assistant's draft answer can be sent or must go to a human, checking policy triggers, risk and confidence, and returns a decision with a reason code the app can log.
- Estimate the cost of an LLM feature
Estimates the monthly token cost of a planned LLM feature from traffic, prompt sizes, calls per action, caching and model choice, with the arithmetic shown and a sensitivity table.
- Send images and PDFs to a multimodal LLM
Implements sending user images or PDFs to a vision-capable LLM API, with format and size checks, resizing for token cost, prompt layout, injection defences and validated structured output.
Showing 50 of 77. See them all in search
Advanced 24
Expert or deep work that assumes the user can judge expert output. Example: Plan croissant lamination.
- Machine-learning engineer
Acts as a machine-learning engineer who starts from the data and a baseline, insists on evals and reproducibility, and distrusts any gain a simpler model explains.
- Plan a fine-tuning project
Decides whether fine-tuning beats prompting or retrieval for a task and, if it does, plans the data, splits, training settings, evaluation against a prompt baseline, and cost.
- Add LLM provider fallback
Adds failover from a primary LLM provider or model to a backup, with error-based triggers, a circuit breaker, per-model prompt adaptation and an eval that proves the fallback output is acceptable.
- Evaluate multi-step agent trajectories
Builds an eval for a multi-step LLM agent that runs scripted scenarios with simulated tools and scores task success, tool choice, step count and unsafe actions across repeated runs.
- Self-host an open-weight LLM
Sets up an open-weight LLM behind an OpenAI-compatible API with vLLM, Ollama or llama.cpp, sizing GPU memory, quantisation, context length and concurrency, with load and quality checks.
- Add human approval to agent actions
Adds human-in-the-loop approval to an LLM agent's risky tool calls, with a risk policy per tool, action previews, durable pause and resume, timeouts and an audit log.
- Add guardrails to an LLM feature
Adds layered guardrails to an LLM feature with input checks, schema-validated output, content and grounding checks, refusal handling, fallbacks and monitoring. Use before real users see it.
- Build a voice agent pipeline
Builds a real-time voice agent pipeline (speech-to-text, LLM, text-to-speech) with streaming at every stage, turn detection, barge-in, a latency budget and graceful failure handling.
- Judge two responses side by side
Compares two candidate responses to the same prompt against stated criteria, reasons per criterion before deciding, and returns A, B or tie. Built to be run twice with the order swapped.
- Run a tool-using agent loop
System prompt for a tool-using agent that plans the next step, calls one declared tool at a time, checks each result, and stops with an answer, a request for approval or a request for help.
- Scale PyTorch training to multiple GPUs
Converts a single-GPU PyTorch training script to DDP or FSDP with correct sampling, per-rank seeding, rank-0 logging and checkpointing, and a test run on two processes.
- Write a LoRA fine-tuning script
Writes a LoRA or QLoRA fine-tuning script with Hugging Face PEFT and TRL that fits the user's GPU, uses the model's chat template, evaluates against the base model and exports the adapter.
- Shadow or canary test a new model version
Runs a new model version in shadow or canary next to the current one, compares outputs, latency and errors, and applies promotion and rollback rules written before the test.
- Add a semantic cache to an LLM feature
Adds a semantic cache in front of an LLM call, with an exact-match layer, cache keys scoped by tenant, prompt and model version, a threshold tuned on labelled pairs, invalidation and a shadow rollout.
- Calibrate an LLM judge against human labels
Measures how well an LLM-as-judge agrees with human labels, finds its biases (length, position, self-preference, leniency), revises it and re-measures on a held-out set.
- Debug an LLM agent that loops or stalls
Diagnoses an LLM agent you built that loops, repeats tool calls, picks the wrong tool or stops early, by reading its traces, then fixes tool design, prompts or stop rules.
- Debug a neural network that will not train
Diagnoses neural network training that diverges, stalls, produces NaN or overfits, checking in order: data, labels, loss setup, learning rate, initialisation and precision.
- Detect prompt injection in untrusted content
Classifies untrusted content such as a web page, email, file or tool output for attempts to override instructions, exfiltrate data or trigger tools, and returns a risk level with the suspicious spans.
- Fine-tune an embedding model for retrieval
Fine-tunes an open embedding model on domain query-passage pairs with hard negatives, then proves a retrieval gain over the base model on a held-out query set before it is used.
- Implement long-term memory for an assistant
Implements a user-scoped long-term memory store for an AI assistant, with write rules, deduplication, relevance retrieval, expiry, user-visible editing and deletion, plus tests.
- Find data leakage in ML training code
Reviews ML training code for target leakage, preprocessing fitted on test data, time and group leakage and duplicates across splits, with the line, evidence and fix for each.
- Keep a RAG index in sync with its sources
Builds incremental re-indexing for a RAG corpus that handles new, edited, deleted and moved documents, permission changes and embedding-model upgrades, with a reconciliation check.
- Plan a multi-step task for an agent
Turns a user goal into an executable agent plan of steps with tools, inputs, success checks, approval gates and replanning triggers, for plan-then-execute agent architectures.
- Build a model retraining pipeline with a promotion gate
Builds a scheduled or triggered retraining pipeline with data validation, reproducible training, a champion-versus-challenger gate and safe promotion with rollback.
Not: prompts for using an assistant (prompting domain); coding-agent setup (meta).