AWS S3 Vector doesn’t work with traditional file formats because it stores structured vector data rather than files. Instead of uploading documents or media files directly, you must first convert your source content into numerical vector embeddings using machine learning models before storing them in S3 Vector. The service accepts vectors as arrays of floating-point numbers (float32 format) with dimensions between 1 and 4,096. Each vector must be accompanied by a unique key identifier and can include optional metadata as key-value pairs supporting string, number, boolean, and list data types.
The process of preparing data for S3 Vector involves using embedding models to transform your source content into vector representations. For text data, you might use models like Amazon Titan Text Embeddings, OpenAI’s text-embedding models, or open-source alternatives like Sentence Transformers to convert documents, articles, or customer reviews into vectors. Image data requires vision models such as Amazon Titan Multimodal Embeddings or CLIP to generate vector representations of visual content. Audio and video content can be processed using specialized models that create embeddings representing acoustic or temporal features. The key requirement is that all vectors within a single vector index must have identical dimensions matching the index configuration.
When ingesting data through the PutVectors API, you structure each vector as a JSON object containing the unique key, the vector data as a float32 array, and optional metadata. For example, when storing document embeddings, you might include metadata like document title, creation date, author, category, or source system to enable filtered searches later. The metadata values can be strings (for titles or categories), numbers (for timestamps or scores), booleans (for flags like “published” or “confidential”), or lists (for tags or multiple authors). While S3 Vector doesn’t directly support traditional file formats, you can use preprocessing pipelines with services like Amazon Bedrock Knowledge Bases to automatically extract text from PDFs, Word documents, or web pages, generate embeddings, and store the results in S3 Vector with appropriate metadata linking back to source files.