This guide covers the development workflow for Protocol Buffer (protobuf) definitions in Memos, including how to define new services and messages, generate code, and maintain API compatibility. For deployment-related configuration, see Deployment. For backend service implementation details, see API and Service Architecture.
The Memos codebase uses Protocol Buffers v3 to define all API contracts. Proto files are organized by service domain:
proto/
├── api/v1/ # API version 1 definitions
│ ├── memo_service.proto # Memo CRUD and operations
│ ├── user_service.proto # User management and settings
│ ├── attachment_service.proto # Attachment operations
│ ├── activity_service.proto # Activity tracking
│ ├── auth_service.proto # Authentication
│ ├── workspace_service.proto # Workspace configuration
│ ├── shortcut_service.proto # Shortcuts/filters
│ └── common.proto # Shared types
└── store/ # Internal storage types
├── memo.proto # Memo storage schema
├── user_setting.proto # User settings storage
└── workspace_setting.proto # Workspace settings storage
Generated Code Locations:
| Language | Output Directory | Purpose |
|---|---|---|
| Go | proto/gen/api/v1/ | Server-side service implementation |
| TypeScript | web/src/types/proto/api/v1/ | Client-side type definitions |
| OpenAPI | proto/gen/openapi.yaml | REST API documentation |
Sources: proto/api/v1/memo_service.proto1-507 proto/api/v1/user_service.proto1-700
Diagram: Code Generation Pipeline
Sources: proto/gen/api/v1/memo_service.pb.go1-27 proto/gen/api/v1/memo_service_grpc.pb.go1-20 proto/gen/api/v1/memo_service.pb.gw.go1-26 buf.gen.yaml1-30
Code Generation Process:
The codebase uses buf for managing protocol buffer compilation. Generation is configured in buf.gen.yaml and executed via the buf generate command.
Key Generated File Types:
| File Pattern | Generator | Purpose | Example Entities |
|---|---|---|---|
*.pb.go | protoc-gen-go | Message type definitions and serialization | type Memo struct, func (x *Memo) ProtoReflect(), func (x *Memo) GetName() string |
*_grpc.pb.go | protoc-gen-go-grpc | gRPC service interfaces and client stubs | type MemoServiceServer interface, type MemoServiceClient interface, func NewMemoServiceClient() |
*.pb.gw.go | protoc-gen-grpc-gateway | HTTP/JSON to gRPC transcoding | func RegisterMemoServiceHandlerServer(), func request_MemoService_CreateMemo_0(), func local_request_MemoService_CreateMemo_0() |
*_pb.ts | protoc-gen-es | TypeScript message schemas | export const MemoSchema, export type Memo, export enum Visibility |
openapi.yaml | protoc-gen-openapiv2 | REST API documentation | paths: /api/v1/memos, components.schemas.Memo |
buf.yaml - Linting and Breaking Change Detection:
Located at repository root, this file configures proto style rules:
buf.gen.yaml - Code Generation Configuration:
Specifies protoc plugins and output paths for each target:
Key Configuration Points:
| Setting | Purpose | Value |
|---|---|---|
paths=source_relative | Generate Go files relative to proto source | Prevents nested gen/api/v1/api/v1/ paths |
target=ts | Generate TypeScript instead of JavaScript | Enables type checking in frontend |
managed.enabled=true | Auto-manage Go package paths | Ensures consistent imports |
Generate all proto code:
Lint proto files:
Check for breaking changes:
Format proto files:
Sources: proto/gen/api/v1/memo_service.pb.go1-27 proto/gen/api/v1/memo_service_grpc.pb.go1-20 proto/gen/api/v1/memo_service.pb.gw.go1-26 .github/workflows/proto-linter.yml20-40
Services in Memos follow Google's API Improvement Proposals (AIP) conventions. Here's the anatomy of a service definition:
Diagram: Service Definition to Generated Code
Sources: proto/api/v1/memo_service.proto17-48 proto/gen/api/v1/memo_service_grpc.pb.go224-253 proto/gen/api/v1/memo_service.pb.gw.go38-77 server/router/api/v1/memo_service.go24-144
Example Service Definition:
From proto/api/v1/memo_service.proto17-48:
This service definition generates the MemoServiceServer interface in proto/gen/api/v1/memo_service_grpc.pb.go224-253 and HTTP route handlers in proto/gen/api/v1/memo_service.pb.gw.go732-837 The backend implementation in server/router/api/v1/memo_service.go must implement all methods defined in the generated interface.
Key Annotations:
| Annotation | Purpose | Example |
|---|---|---|
google.api.http | Maps gRPC method to HTTP endpoint | post: "/api/v1/memos" |
google.api.method_signature | Defines required parameters | "memo,update_mask" |
google.api.field_behavior | Marks field requirements | REQUIRED, OPTIONAL, OUTPUT_ONLY |
google.api.resource | Defines resource naming patterns | pattern: "memos/{memo}" |
google.api.resource_reference | Links to resource definitions | type: "memos.api.v1/Memo" |
Sources: proto/api/v1/memo_service.proto17-48 proto/gen/api/v1/memo_service_grpc.pb.go224-253 proto/gen/api/v1/memo_service.pb.gw.go732-837 server/router/api/v1/memo_service.go24-144
The codebase uses two types of proto messages:
API Proto Messages (proto/api/v1/):
Memo message with name, creator, content fieldsStore Proto Messages (proto/store/):
payload columnsMemoPayload with property, location, tags fieldsDiagram: Proto Message Usage Flow
Sources: proto/gen/api/v1/memo_service.pb.gw.go40-77 proto/gen/api/v1/memo_service.pb.go215-407 server/router/api/v1/memo_service.go24-144 store/db/sqlite/memo.go16-51 store/db/sqlite/memo.go54-191
Messages use specific field annotations to control validation and serialization:
| Annotation | Meaning | Validation | Example Use Case |
|---|---|---|---|
REQUIRED | Must be set in requests | Client must provide | string content = 7 [(google.api.field_behavior) = REQUIRED] |
OPTIONAL | May be omitted | Can be nil/empty | string email = 4 [(google.api.field_behavior) = OPTIONAL] |
OUTPUT_ONLY | Server-populated only | Ignored in requests | Timestamp create_time = 4 [(google.api.field_behavior) = OUTPUT_ONLY] |
INPUT_ONLY | Never returned by server | Cleared from responses | string password = 8 [(google.api.field_behavior) = INPUT_ONLY] |
IDENTIFIER | Resource identifier field | Used in resource name | string name = 1 [(google.api.field_behavior) = IDENTIFIER] |
Example API Message:
From proto/api/v1/memo_service.proto153-229:
Example Store Message:
From proto/store/memo.proto1-28:
Store Payload Serialization:
The store layer serializes storepb.MemoPayload as JSON in database columns. From store/db/sqlite/memo.go16-39:
Sources: proto/api/v1/memo_service.proto153-232 proto/store/memo.proto1-30 proto/gen/api/v1/memo_service.pb.go215-407 store/db/sqlite/memo.go16-51 server/router/api/v1/memo_service.go24-144
Diagram: Adding New RPC Method - Complete Flow
Sources: proto/api/v1/memo_service.proto17-48 proto/gen/api/v1/memo_service.pb.go1-27 proto/gen/api/v1/memo_service_grpc.pb.go224-253 server/router/api/v1/memo_service.go337-474 store/db/sqlite/memo.go194-233
1. Define the Proto Message and RPC:
Add to proto/api/v1/memo_service.proto:
This follows the pattern of existing methods like DeleteMemo in proto/api/v1/memo_service.proto44-48
2. Run Code Generation:
3. Implement the Service Method:
Add implementation to server/router/api/v1/memo_service.go:
Key Implementation Patterns:
| Pattern | Purpose | Example | Code Reference |
|---|---|---|---|
ExtractMemoUIDFromName() | Parse resource name | "memos/abc123" → "abc123" | server/router/api/v1/common.go |
s.fetchCurrentUser(ctx) | Get authenticated user | Returns *store.User from context | server/router/api/v1/memo_service.go24-31 |
s.Store.GetMemo() | Fetch from database | Uses store.FindMemo filter struct | server/router/api/v1/memo_service.go294-302 |
s.Store.UpdateMemo() | Update database | Uses store.UpdateMemo with field pointers | server/router/api/v1/memo_service.go442-444 |
s.convertMemoFromStore() | Convert to API type | store.Memo → v1pb.Memo | server/router/api/v1/memo_service.go330-334 |
status.Errorf() | Return gRPC error | Maps to HTTP status codes via gateway | server/router/api/v1/memo_service.go27-30 |
4. The Generated Interface:
After running buf generate, the MemoServiceServer interface in proto/gen/api/v1/memo_service_grpc.pb.go automatically includes:
5. Test the New Endpoint:
Sources: proto/api/v1/memo_service.proto17-48 proto/gen/api/v1/memo_service_grpc.pb.go224-253 server/router/api/v1/memo_service.go24-144 server/router/api/v1/memo_service.go337-474
Diagram: Generated Code Structure
Sources: proto/gen/api/v1/memo_service.pb.go215-407 proto/gen/api/v1/memo_service.pb.go28-78 proto/gen/api/v1/memo_service_grpc.pb.go39-71 proto/gen/api/v1/memo_service.pb.gw.go732-837 web/src/types/proto/api/v1/memo_service_pb.ts28-67
Message Definitions (*.pb.go):
The generated Go message types include all fields from the proto definition with appropriate Go types. From proto/gen/api/v1/memo_service.pb.go215-256:
Each message includes getter methods like GetName(), GetContent(), etc., generated by protoc-gen-go.
Service Interface (*_grpc.pb.go):
The generated server interface must be implemented by the backend service. From proto/gen/api/v1/memo_service_grpc.pb.go224-253:
The APIV1Service struct in server/router/api/v1/memo_service.go implements this interface.
gRPC-Gateway Functions (*.pb.gw.go):
The gateway generates HTTP-to-gRPC translation functions. From proto/gen/api/v1/memo_service.pb.gw.go40-77:
The RegisterMemoServiceHandlerServer function in proto/gen/api/v1/memo_service.pb.gw.go732-837 registers all HTTP routes with the gateway mux.
Sources: proto/gen/api/v1/memo_service.pb.go215-407 proto/gen/api/v1/memo_service_grpc.pb.go224-253 proto/gen/api/v1/memo_service.pb.gw.go40-77 proto/gen/api/v1/memo_service.pb.gw.go732-837 server/router/api/v1/memo_service.go24-144
The frontend uses generated TypeScript types for type-safe API calls:
Generated TypeScript Interface:
From web/src/types/proto/api/v1/memo_service.ts85-256:
Frontend Usage Example:
The frontend imports generated TypeScript types and uses the gRPC-web client. While the specific client implementation may vary, the types ensure compile-time safety:
Generated TypeScript Message Schema:
From web/src/types/proto/api/v1/memo_service_pb.ts28-67:
Sources: web/src/types/proto/api/v1/memo_service_pb.ts28-67
The codebase uses buf for proto file validation. From .github/workflows/proto-linter.yml1-34:
Validation Checks:
| Check | Tool | Purpose | When It Runs |
|---|---|---|---|
| Lint | buf lint | Ensures proto style consistency and best practices | On every PR touching proto/** |
| Format | buf format | Validates proper proto formatting | On every PR touching proto/** |
| Breaking Changes | buf breaking | Detects incompatible API changes | Before merging to main |
| Backend Tests | go test | Tests service implementations | On every PR touching **.go |
| Frontend Tests | pnpm lint && pnpm build | Tests TypeScript compilation | On every PR touching web/** |
Run all validations before pushing:
Pre-Push Checklist:
| Step | Command | Purpose |
|---|---|---|
| Format | buf format -w proto | Auto-fix proto formatting |
| Lint | buf lint proto | Check style violations |
| Breaking | buf breaking proto --against '.git#branch=main' | Ensure backward compatibility |
| Generate | buf generate proto | Update generated code |
| Compile Go | go build ./server/... | Verify Go compilation |
| Compile TS | cd web && pnpm typecheck | Verify TypeScript types |
| Test | go test ./server/... && cd web && pnpm test | Run all tests |
Common Linting Errors:
| Error | Cause | Fix |
|---|---|---|
ENUM_ZERO_VALUE_SUFFIX | Enum 0 value doesn't end with _UNSPECIFIED | Rename to TYPE_UNSPECIFIED = 0 |
FIELD_LOWER_SNAKE_CASE | Field name not in snake_case | Rename userId to user_id |
RPC_REQUEST_STANDARD_NAME | Request message name doesn't match RPC | Rename to {RpcName}Request |
SERVICE_SUFFIX | Service name doesn't end with Service | Rename to MemoService |
Sources: .github/workflows/proto-linter.yml1-34 .github/workflows/backend-tests.yml1-48 .github/workflows/frontend-tests.yml1-48
All resources follow the pattern: {collection}/{resource_id}
Examples:
users/101 - User by numeric IDusers/steven - User by usernamememos/abc123 - Memo by UIDusers/101/settings/general - Nested resourceFrom proto/api/v1/memo_service.proto162-164:
Update operations use google.protobuf.FieldMask to specify which fields to modify:
From proto/api/v1/memo_service.proto300-307:
Implementation Example:
From server/router/api/v1/memo_service.go327-398:
List operations use cursor-based pagination:
From proto/api/v1/memo_service.proto251-289:
Implementation:
From server/router/api/v1/memo_service.go158-188:
Sources: proto/api/v1/memo_service.proto251-289 server/router/api/v1/memo_service.go105-244
The API uses Common Expression Language (CEL) for filtering. From proto/api/v1/memo_service.proto273-276:
Filter Processing:
From server/router/api/v1/memo_service.go128-133:
Store Layer Translation:
From store/db/sqlite/memo.go44-51:
Supported Filter Examples:
creator_id == 101visibility in ["PUBLIC", "PROTECTED"]creator_id == 5 || visibility in ["PUBLIC", "PROTECTED"]Sources: server/router/api/v1/memo_service.go128-148 store/db/sqlite/memo.go42-51 store/db/mysql/memo.go50-59 store/db/postgres/memo.go42-50
DO:
optionalreserved for deprecated field numbersDON'T:
| Scenario | Behavior |
|---|---|
| User must provide value | REQUIRED |
| Server computes value | OUTPUT_ONLY |
| Never return to client | INPUT_ONLY (e.g., passwords) |
| May be omitted | OPTIONAL |
Always use the pattern: {collection}/{id}
From proto/api/v1/user_service.proto188-190:
From proto/api/v1/memo_service.proto108-113:
The 0 value should always be UNSPECIFIED to handle default cases safely.
Sources: proto/api/v1/memo_service.proto108-113 proto/api/v1/user_service.proto223-233
Refresh this wiki