Learn to automate the cataloging of archival photos, videos, and audio. This workflow uses Python, the OpenAI Vision API, and vector embeddings to create a structured, semantically searchable media database.
with Claire Vo
Use an AI-first code editor like Cursor to write a Python script. The script should take a local image file and submit it to the OpenAI Vision API to generate a general visual description.
Write me a script that submits the jpeg at the root of this workspace to open ai for description. I want just a general visual description of what we can see in the image. Uh, any API credentials you need are in a text file at the root of the folder.Modify the script to first extract any available EXIF metadata from the image file (e.g., photographer, date, location). Append this factual metadata to the prompt before sending the image to the AI to act as a guardrail and produce more accurate, fact-based descriptions.
I want you to add a step to this script. I wanna scrape any available metadata from the file first and append that to the prompt.For video files, create a process to sample still frames at regular intervals (e.g., every five seconds) and transcribe the audio in chunks using a model like Whisper. Send the collected frame captions and the full audio transcript to a reasoning model to generate a comprehensive summary of the video clip.
To enable advanced discovery, generate vector embeddings for each asset. Use an image model like CLIP for image thumbnails and a text model for the descriptions. Fuse these embeddings together to create a rich, multi-modal representation of the asset.
Use the generated vector embeddings to power a 'find similar' feature in your database. This allows users to select an asset and instantly find all other visually or thematically related items in the collection, moving beyond simple keyword search.
Follow Sahil Lavingia's method for delegating small but impactful maintenance tasks, like fixing an internal Slack report, to the AI agent Devin. This workflow shows how to use simple, clear prompts to fix code and update system prompts without distracting your engineering team.
Learn Gumroad CEO Sahil Lavingia's process for rapidly prototyping a UI with v0, then using the AI agent Devin to autonomously code, implement, and submit the feature for review, turning a multi-day task into a one-hour workflow.
Improve your writing in real-time by using ChatGPT as a first reader. Get instant feedback on structure, clarity, and flow as you write, allowing you to polish your work section by section.
We've made ChatPRD affordable so everyone from engineers to founders to Chief Product Officers can benefit from an AI PM.