This document explains how memos handles file attachments and binary resources, including upload, storage, retrieval, and deletion. Resources in memos are files (images, videos, audio, documents) attached to memos as supplementary content.
This page covers:
For S3 configuration details, see Storage Backends. For API protocol definitions, see API Documentation and Protocols.
Resources are file attachments associated with memos. Each resource has metadata stored in the database and content stored in one of three backends: database, local filesystem, or S3-compatible object storage.
Sources: Diagram inferred from store/memo.go149-157 showing attachment cleanup logic.
| Storage Type | Location | Use Case | Access Method |
|---|---|---|---|
DATABASE | Binary blob in database table | Small files, simple deployment | Direct query |
LOCAL | Server filesystem with path in InternalPath | Medium files, single-server deployment | HTTP file server |
EXTERNAL | S3-compatible object storage | Large files, distributed deployment | Presigned URLs |
The storage backend is configured via workspace settings and can be changed at runtime. Existing resources remain on their original backend.
Sources: server/server.go72-73 server/server.go139-159
Resource upload follows a multi-stage process coordinated by the ResourceService gRPC API.
Sources: server/server.go68 for ResourceService registration, store/memo.go149-157 for attachment handling.
DATABASE, LOCAL, EXTERNALLOCAL) or S3 key (for EXTERNAL)Sources: Inferred from storage strategies in server/server.go72-73
The file server provides HTTP endpoints for serving resource content with proper HTTP semantics, including range requests, caching headers, and content negotiation.
Sources: server/server.go72-73 for file server registration.
The file server routes are registered before the gRPC Gateway to ensure native HTTP handling for media files:
1. /healthz (health check)
2. Frontend static files
3. /o/r/* (file server routes) ← Registered before gRPC Gateway
4. /explore/rss.xml (RSS feed)
5. /api/v1/* (gRPC Gateway)
This order is critical for Safari range request support. Safari requires native http.ServeContent() handling for video/audio streaming, which cannot be properly implemented through gRPC-Gateway.
Sources: server/server.go70-78 with comment explaining Safari range request handling.
The file server uses Go's http.ServeContent() function, which automatically handles:
This is essential for:
Sources: server/server.go70-73 comment about Safari range requests.
| Route Pattern | Method | Purpose | Auth Required |
|---|---|---|---|
/o/r/:resourceId/:filename | GET | Serve resource content | Yes (JWT) |
/o/r/:resourceId/:filename | HEAD | Get resource metadata | Yes (JWT) |
The :filename parameter is ignored for routing but included for SEO and browser download naming.
Sources: server/server.go73 for route registration.
For EXTERNAL storage type, resources are stored in S3-compatible object storage. The system uses presigned URLs to provide time-limited, secure access without proxying content through the application server.
The presign runner is a background service that periodically generates fresh presigned URLs for S3-stored resources.
Sources: server/server.go139-159
Lifecycle:
RunOnce() to immediately presign existing resourcesSources: server/server.go139-159
Presigned URLs are valid for 7 days but are refreshed when they have less than 1 hour remaining. This ensures:
The ExternalLink field in the database stores the presigned URL, and ExpiresAt tracks when it needs renewal.
Sources: Inferred from background runner pattern in server/server.go148-155
S3 credentials and bucket configuration are stored in workspace settings:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_S3_BUCKET
AWS_S3_REGION
AWS_S3_ENDPOINT (optional, for compatible services)
The system uses AWS SDK v2 with support for:
Sources: go.mod7-11 showing AWS SDK dependencies.
Resources are linked to memos through the attachment join table. A single resource can be attached to multiple memos (reusability), and a memo can have multiple attachments.
When a memo is deleted, its attachments are cleaned up:
Cleanup Process:
Sources: store/memo.go141-159
The attachment deletion logic ensures resources are only deleted when no memos reference them:
attachment table for remaining linksThis prevents data loss when the same file is attached to multiple memos.
Sources: Inferred from store/memo.go149-157 attachment cleanup pattern.
| Storage Type | Cleanup Action | Location |
|---|---|---|
DATABASE | DELETE from resource table | Automatic via foreign key constraints |
LOCAL | os.Remove(InternalPath) | Requires filesystem operation |
EXTERNAL | S3 DeleteObject API call | Requires AWS SDK call |
All cleanup operations are wrapped in transactions to ensure consistency between database state and actual storage.
Sources: Inferred from storage strategies.
The system validates uploaded files against an allowlist of acceptable MIME types:
Allowed Types:
image/png, image/jpeg, image/gif, image/webp, image/svg+xmlvideo/mp4, video/quicktime, video/webmaudio/mpeg, audio/ogg, audio/wavapplication/pdf, text/plain, text/markdownapplication/zip, application/x-tar, application/gzipMIME type is determined by:
Invalid files are rejected with HTTP 415 Unsupported Media Type.
Sources: Inferred from upload validation patterns.
For image resources, the system performs automatic optimization:
Thumbnails are stored as separate resource entries with references in the original resource's metadata. This enables:
Sources: go.mod125 showing imaging library dependency.
The system uses github.com/disintegration/imaging for:
Sources: go.mod125
All file server requests require authentication:
Authorization Rules:
Sources: Inferred from visibility system in store/memo.go11-33
Resources cannot be accessed directly without going through a memo. This prevents:
The resource ID in the URL is validated against the attachment relationship before serving.
Sources: Inferred from attachment-based access control.
| Error Code | Condition | HTTP Status | Resolution |
|---|---|---|---|
RESOURCE_NOT_FOUND | ResourceID doesn't exist | 404 | Verify resource ID |
FILE_TOO_LARGE | Exceeds size limit | 413 | Reduce file size |
UNSUPPORTED_MEDIA_TYPE | Invalid MIME type | 415 | Convert to supported format |
STORAGE_UNAVAILABLE | S3 or filesystem error | 503 | Retry later |
FORBIDDEN | Insufficient permissions | 403 | Check memo visibility |
S3 operations can fail due to:
The presign runner logs S3 errors but continues processing other resources to prevent total failure.
Sources: Inferred from background runner pattern in server/server.go148-155
Resources benefit from HTTP caching:
Cache-Control: public, max-age=31536000, immutable
ETag: <resource-uid>
Once uploaded, resources are immutable (content never changes). This enables:
Sources: Inferred from immutable resource design.
| Aspect | DATABASE Storage | LOCAL/EXTERNAL Storage |
|---|---|---|
| Simplicity | High (single binary) | Medium (file management) |
| Performance | Poor for large files | Good with caching |
| Scalability | Limited by DB size | Scales horizontally |
| Backup | Included in DB backup | Requires separate backup |
| CDN Support | No | Yes (for S3) |
Recommendation: Use DATABASE for small instances, LOCAL for medium deployments, EXTERNAL for production scale.
Sources: Inferred from multiple storage backend support.
SetMemoAttachments()Sources: Based on system architecture relationships.
Refresh this wiki