Media Operations

Overview

Media operations handle binary file uploads (images, PDFs, documents) and retrievals using Google Cloud Storage (GCS) as the backend. Files are stored in multi-tenant isolated paths and referenced in dimension tables via metadata.

Key Characteristics:

Base64-encoded uploads via JSON
Automatic GCS storage with tenant isolation
Path structure: {tenantId}/{env}/{dim}/{field}/{year}/{month}/{filename}
Private and public bucket support
Signed URLs for temporary access
MIME type detection

Upload Workflow

How Upload Works

Client encodes file to base64
POST to CoreWrite with base64 data in JSON body
CoreWrite processes:
- Decodes base64
- Generates unique MD5 filename
- Uploads to GCS bucket
- Stores metadata in dimension table field
Response includes bucket metadata and URL

Storage Path Structure

{tenantId}/{env}/{dim}/{field}/{year}/{month}/{filename}

Example:

abc123-tenant/app.q01.io/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg

Components:

{tenantId}: Extracted from JWT (automatic tenant isolation)
{env}: Environment resource name (e.g., "app.q01.io")
{dim}: Dimension code (e.g., PRD, ART)
{field}: Field code (e.g., XPRD12)
{year}/{month}: Upload date (YYYY/MM format)
{filename}: MD5 hash + original extension (ensures uniqueness)

Upload Example

JavaScript Implementation

async function uploadProductImage(productId, fileInput) {
  // Read file as base64
  const file = fileInput.files[0];
  const reader = new FileReader();

  return new Promise((resolve, reject) => {
    reader.onload = async (e) => {
      const base64Data = e.target.result;

      // Create product with image
      const response = await fetch('/api/v4/core/PRD', {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${token}`,
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({
          XPRD01: 'Premium Widget',
          XPRD02: 49.99,
          XPRD12: {
            data: base64Data,
            name: file.name,
            type: file.type,
            public: false // private bucket
          },
          source: 'productCreate'
        })
      });

      if (!response.ok) {
        throw new Error('Upload failed');
      }

      const result = await response.json();
      resolve(result[0]);
    };

    reader.onerror = reject;
    reader.readAsDataURL(file);
  });
}

Request Body Format

{
  "XPRD01": "Premium Widget",
  "XPRD02": 49.99,
  "XPRD12": {
    "data": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD...",
    "name": "product-photo.jpg",
    "type": "image/jpeg",
    "public": false
  },
  "source": "productCreate"
}

Field Properties:

data: Base64-encoded file with data URI prefix
name: Original filename
type: MIME type
public: Boolean - store in public bucket (true) or private bucket (false)

Response Format

[{
  "code": 201,
  "insertedId": "123",
  "body": {
    "XPRD01": "Premium Widget",
    "XPRD02": 49.99,
    "XPRD12": {
      "data": "",
      "name": "product-photo.jpg",
      "type": "image/jpeg",
      "bucket": {
        "name": "a3f2e8d9c1b4.jpg",
        "dim": "PRD",
        "field": "XPRD12",
        "year": "2025",
        "month": "12",
        "tenantId": "abc123-tenant",
        "env": "app.q01.io",
        "public": ""
      },
      "url": "/media/getImage/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg"
    }
  }
}]

Response Notes:

data field is cleared (empty string) - file stored in GCS
bucket object contains metadata for retrieval
url is the retrieval endpoint path
Public uploads include tenantId and env in URL

Download Operations

Private Image Retrieval

Endpoint: GET /api/v4/core/media/getImage/{dim}/{field}/{year}/{month}/{file}

Authentication: Required (Bearer token)

Use Case: Retrieve images from private bucket (default)

Example:

curl -X GET "https://coreservice.q01.io/api/v4/core/media/getImage/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg" \
  -H "Authorization: Bearer eyJhbGc..."

Response:

Binary image data
Content-Type: Detected from file extension
Content-Disposition: inline; filename="a3f2e8d9c1b4.jpg"
Cache-Control: private

JavaScript Example:

async function displayProductImage(imageMetadata) {
  const { dim, field, year, month, name } = imageMetadata.bucket;

  const url = `/api/v4/core/media/getImage/${dim}/${field}/${year}/${month}/${name}`;

  const response = await fetch(url, {
    headers: { 'Authorization': `Bearer ${token}` }
  });

  if (!response.ok) {
    throw new Error('Failed to load image');
  }

  const blob = await response.blob();
  const imageUrl = URL.createObjectURL(blob);

  // Display in img tag
  document.getElementById('productImage').src = imageUrl;
}

Public Image Retrieval

Endpoint: GET /api/v4/core/media/getImage/{tenantId}/{env}/{dim}/{field}/{year}/{month}/{file}

Authentication: Not required

Use Case: Publicly accessible images (e.g., product catalog, marketing)

Example:

<!-- Direct image tag - no authentication needed -->
<img src="https://coreservice.q01.io/api/v4/core/media/getImage/abc123-tenant/app.q01.io/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg"
     alt="Product Image" />

Use Case:

Product catalog images
Public avatars
Marketing materials
Content accessible without login

Batch Retrieval with Signed URLs

Private Batch Retrieval

Endpoint: POST /api/v4/core/media/getImages

Authentication: Required

Purpose: Get temporary signed URLs for multiple images

Request Body:

[
  {
    "dim": "PRD",
    "field": "XPRD12",
    "year": "2025",
    "month": "12",
    "file": "a3f2e8d9c1b4.jpg"
  },
  {
    "dim": "PRD",
    "field": "XPRD12",
    "year": "2025",
    "month": "12",
    "file": "b2e8f1c3d5a9.jpg"
  }
]

Response:

[
  {
    "url": "https://storage.googleapis.com/q01-media/abc123-tenant/app.q01.io/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=...",
    "filename": "a3f2e8d9c1b4.jpg"
  },
  {
    "url": "https://storage.googleapis.com/q01-media/abc123-tenant/app.q01.io/PRD/XPRD12/2025/12/b2e8f1c3d5a9.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=...",
    "filename": "b2e8f1c3d5a9.jpg"
  }
]

Signed URL Properties:

Validity: 15 minutes
Direct access: No additional authentication required during validity period
Method: GET only
Version: v4 (Google Cloud Storage standard)

JavaScript Example:

async function loadProductGallery(images) {
  // Prepare batch request
  const imageRequests = images.map(img => ({
    dim: 'PRD',
    field: 'XPRD12',
    year: img.year,
    month: img.month,
    file: img.filename
  }));

  // Request signed URLs
  const response = await fetch('/api/v4/core/media/getImages', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${token}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify(imageRequests)
  });

  const signedUrls = await response.json();

  // Render gallery with signed URLs
  const gallery = document.getElementById('gallery');
  signedUrls.forEach(item => {
    const img = document.createElement('img');
    img.src = item.url;
    img.alt = item.filename;
    gallery.appendChild(img);
  });
}

Public Batch Retrieval

Endpoint: POST /api/v4/core/media/getPublicImages

Authentication: Not required

Request Body:

[
  {
    "tenantId": "abc123-tenant",
    "env": "app.q01.io",
    "dim": "PRD",
    "field": "XPRD12",
    "year": "2025",
    "month": "12",
    "file": "a3f2e8d9c1b4.jpg"
  }
]

Response: Same format as private batch (signed URLs with 15 min expiry)

Storage Configuration

Environment Variables

CoreQuery (for retrieval):

BUCKET_NAME_MEDIA="q01-media"              # Private bucket
BUCKET_NAME_MEDIA_PUBLIC="q01-media-public"  # Public bucket

CoreWrite (for upload):

BUCKET_NAME_MEDIA="q01-media"
BUCKET_NAME_MEDIA_PUBLIC="q01-media-public"

GCS Authentication

Both services require bucket.json (service account key) in root directory:

/Users/mwpo/q01sdk/mscorequery/dkcorequery/src/bucket.json
/Users/mwpo/q01sdk/mscorewrite/dkcorewrite/src/bucket.json

Permissions Required:

storage.objects.create (upload)
storage.objects.get (download)
storage.objects.delete (cleanup)

Supported File Types

MIME Type Detection

CoreQuery automatically detects MIME types for proper Content-Type headers:

Extension	MIME Type	Use Case
jpg, jpeg	image/jpeg	Photos
png	image/png	Graphics
gif	image/gif	Animations
pdf	application/pdf	Documents
doc, docx	application/msword	Word documents
xls, xlsx	application/vnd.ms-excel	Spreadsheets
zip	application/zip	Archives
mp3	audio/mpeg	Audio files

Fallback: application/octet-stream for unknown types

Best Practices

✅ DO

Use base64 encoding for uploads:

const reader = new FileReader();
reader.readAsDataURL(file); // Returns data:mime;base64,... format

Store metadata in dimension table:

{
  "XPRD12": {
    "bucket": {
      "name": "a3f2e8d9c1b4.jpg",
      "dim": "PRD",
      "field": "XPRD12",
      "year": "2025",
      "month": "12"
    },
    "url": "/media/getImage/PRD/XPRD12/2025/12/a3f2e8d9c1b4.jpg"
  }
}

Use signed URLs for galleries:

Reduces authentication overhead
Browser can cache images
Faster page load

Choose correct bucket:

Private: User photos, invoices, sensitive documents
Public: Product catalog, marketing content, public profiles

❌ DON'T

Don't upload without base64 encoding:

// ❌ Wrong
body: JSON.stringify({ XPRD12: fileObject })

// ✅ Correct
body: JSON.stringify({ XPRD12: { data: base64String } })

Don't hardcode tenant paths:

// ❌ Wrong - tenant extracted from JWT automatically
const path = `/tenant123/app.q01.io/PRD/...`

// ✅ Correct - let backend construct path
const metadata = result[0].body.XPRD12.bucket;

Don't skip file validation:

// ✅ Validate before upload
const MAX_SIZE = 5 * 1024 * 1024; // 5MB
if (file.size > MAX_SIZE) {
  throw new Error('File too large');
}

const ALLOWED_TYPES = ['image/jpeg', 'image/png', 'image/gif'];
if (!ALLOWED_TYPES.includes(file.type)) {
  throw new Error('Invalid file type');
}

Don't forget signed URL expiry:

// Signed URLs expire after 15 minutes
// Refresh if needed:
async function refreshGallery() {
  const newUrls = await fetchSignedUrls(images);
  updateGalleryImages(newUrls);
}

setInterval(refreshGallery, 14 * 60 * 1000); // Refresh every 14 min

Error Handling

Common Errors

500 - Unable to connect to bucket:

{
  "code": 500,
  "message": "Unable to connect to bucket",
  "errors": []
}

Fix: Verify bucket.json exists and has correct permissions

404 - File not found:

{
  "code": 404,
  "message": "File not found in bucket",
  "errors": []
}

Fix: Verify path components (dim, field, year, month, filename)

413 - Payload too large:

{
  "code": 413,
  "message": "Request entity too large"
}

Fix: Compress images or reduce file size before upload

Use Case: Product Image Upload

class ProductImageManager {
  constructor(apiBase, token) {
    this.apiBase = apiBase;
    this.token = token;
    this.MAX_SIZE = 5 * 1024 * 1024; // 5MB
  }

  async uploadImage(file, isPublic = false) {
    // Validate
    if (file.size > this.MAX_SIZE) {
      throw new Error('File exceeds 5MB limit');
    }

    // Read as base64
    const base64 = await this.readFileAsBase64(file);

    // Create product with image
    const response = await fetch(`${this.apiBase}/api/v4/core/PRD`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.token}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        XPRD01: 'New Product',
        XPRD12: {
          data: base64,
          name: file.name,
          type: file.type,
          public: isPublic
        },
        source: 'productCreate'
      })
    });

    if (!response.ok) {
      const error = await response.json();
      throw new Error(error.message);
    }

    const result = await response.json();
    return result[0].body.XPRD12;
  }

  async getImageUrl(imageMetadata) {
    const { bucket } = imageMetadata;

    if (bucket.public === 'public') {
      // Public URL - no auth needed
      return `${this.apiBase}/api/v4/core/media/getImage/${bucket.tenantId}/${bucket.env}/${bucket.dim}/${bucket.field}/${bucket.year}/${bucket.month}/${bucket.name}`;
    } else {
      // Private URL - auth required
      return `${this.apiBase}/api/v4/core/media/getImage/${bucket.dim}/${bucket.field}/${bucket.year}/${bucket.month}/${bucket.name}`;
    }
  }

  readFileAsBase64(file) {
    return new Promise((resolve, reject) => {
      const reader = new FileReader();
      reader.onload = (e) => resolve(e.target.result);
      reader.onerror = reject;
      reader.readAsDataURL(file);
    });
  }
}

Summary

Media operations provide:

✅ Secure file storage with tenant isolation
✅ Automatic path generation and uniqueness
✅ Private and public bucket support
✅ Signed URLs for temporary access
✅ MIME type detection
✅ Base64 upload via JSON (no multipart/form-data)

Key Takeaways:

Upload via POST with base64-encoded data in JSON body
Files stored in GCS with path: {tenantId}/{env}/{dim}/{field}/{year}/{month}/{filename}
Metadata stored in dimension table field
Private retrieval requires authentication
Public retrieval is unauthenticated
Batch endpoints return signed URLs (15 min validity)
Always validate file size and type before upload

Next: Sort Operations →

Write Operations - Creating records with media
Field Visibility - COD_ON_OFF for image fields
Context Chain - Environment and tenant isolation

Media Operations

Overview​

Upload Workflow​

How Upload Works​

Storage Path Structure​

Upload Example​

JavaScript Implementation​

Request Body Format​

Response Format​

Download Operations​

Private Image Retrieval​

Public Image Retrieval​

Batch Retrieval with Signed URLs​

Private Batch Retrieval​

Public Batch Retrieval​

Storage Configuration​

Environment Variables​

GCS Authentication​

Supported File Types​

MIME Type Detection​

Best Practices​

✅ DO​

❌ DON'T​

Error Handling​

Common Errors​

Use Case: Product Image Upload​

Summary​

Related Concepts​

Overview

Upload Workflow

How Upload Works

Storage Path Structure

Upload Example

JavaScript Implementation

Request Body Format

Response Format

Download Operations

Private Image Retrieval

Public Image Retrieval

Batch Retrieval with Signed URLs

Private Batch Retrieval

Public Batch Retrieval

Storage Configuration

Environment Variables

GCS Authentication

Supported File Types

MIME Type Detection

Best Practices

✅ DO

❌ DON'T

Error Handling

Common Errors

Use Case: Product Image Upload

Summary

Related Concepts