Content Moderation

Related docs: Hosting Infrastructure · SDLC/DESIGN · Microservices

1. Overview

All image uploads on Pakashop — product images, shop logos, shop banners, and KYC documents — pass through an asynchronous content moderation pipeline powered by Sightengine via the pakashop-moderation microservice (Python, port 3110).

The moderation service runs as an independent microservice deployed on AWS EC2, listening on port 3110.

2. Architecture

3. Moderation Service API Contract

The pakashop-moderation service exposes a single endpoint:

`POST /moderate`

Request body:

{
  "imageUrl": "https://res.cloudinary.com/pakashop/image/upload/...",
  "assetId": "uuid",
  "assetType": "PRODUCT_IMAGE" | "SHOP_LOGO" | "SHOP_BANNER" | "KYC_DOCUMENT"
}

Response:

{
  "approved": true,
  "scores": {
    "nudity_safe": 0.98,
    "violence": 0.01,
    "offensive": 0.02
  }
}

The Express API calls this endpoint non-blockingly using a fire-and-forget pattern:

// Non-blocking moderation trigger
moderationClient.post('/moderate', { imageUrl, assetId, assetType })
  .catch(err => logger.warn({ err }, 'Moderation service unreachable'));

4. Sightengine Configuration

Model	Purpose	Threshold
`nudity`	NSFW detection	`nudity.safe < 0.85` → flag
`violence`	Gore / violent content	`violence.prob > 0.30` → flag
`offensive`	Hate symbols, offensive text in images	`offensive.prob > 0.30` → flag

Environment variables (on pakashop-moderation service):

SIGHTENGINE_API_USER=your_api_user
SIGHTENGINE_API_SECRET=your_api_secret
SIGHTENGINE_BASE_URL=https://api.sightengine.com/1.0/check.json

5. Violation Actions

Severity	Action
Flagged (automated)	Asset hidden from public display; `ModerationFlag` record created; admin notified via in-app notification
Reviewed & approved (admin)	Asset restored; `ModerationFlag.status = APPROVED`
Reviewed & rejected (admin)	Asset permanently removed from Cloudinary; associated product/shop suspended
KYC document flagged	Seller application paused pending manual review

6. Retry Logic

If the Sightengine API is unavailable:

The moderation service retries up to 3 times with exponential back-off (1 s, 2 s, 4 s).
If all retries fail, the asset is marked moderationStatus = PENDING and queued for a scheduled re-check job (runs every 15 minutes via pakashop-scheduler).
Assets in PENDING status are visible by default (optimistic approval) unless the asset type is KYC_DOCUMENT, which defaults to hidden until explicitly approved.

7. Admin Review Dashboard

Flagged assets are visible in the admin dashboard under Content Moderation → Flagged Items. Admins can:

View the flagged image alongside the Sightengine confidence scores.
Approve — removes the flag; asset becomes public.
Reject — deletes the asset from Cloudinary; associated product/shop is suspended.
Escalate — marks for senior review without taking action.

All actions are logged in ModerationFlag.reviewedAt, reviewedBy, and reviewAction.

8. Asset Types and Handling

Asset Type	Visibility Default	Admin Review Required	Special Handling
`PRODUCT_IMAGE`	Visible (optimistic)	Yes if flagged	Product hidden if rejected
`SHOP_LOGO`	Visible (optimistic)	Yes if flagged	Shop page affected if rejected
`SHOP_BANNER`	Visible (optimistic)	Yes if flagged	Shop page affected if rejected
`KYC_DOCUMENT`	Hidden (pessimistic)	Always required	Seller application blocked until approved

For internal use only. Do not distribute outside Pakashop engineering.

1. Overview​

2. Architecture​

3. Moderation Service API Contract​

POST /moderate​

4. Sightengine Configuration​

5. Violation Actions​

6. Retry Logic​

7. Admin Review Dashboard​

8. Asset Types and Handling​