Making background media processing verifiable.

Media processing takes longer than a normal web request and depends on several components working together. A job can be accepted by an API yet still encounter a storage, worker or encoder problem. The processing path needs to be checked beyond the request itself.

Conceptual flow, simplified for this case study.

My contributions include worker configuration, storage integration, encoder validation and deployment infrastructure. This overview focuses on those areas of the processing system.

• Celery worker and queue configuration
• Storage integration around processing jobs
• Video-encoder capability validation and deployment work

Validate with an encoding operation

A configured encoder is not sufficient evidence that a processing workload will run. The validation uses an actual encoding operation to check capability before relying on the encoder in the workflow.

Treat worker configuration as part of the system

Concurrency and queue behaviour affect how processing jobs reach an encoder. They need to be considered together with the workload and available resources, then verified along the processing path.

This work demonstrates investigation across application and infrastructure boundaries: following a job through dispatch, workers, storage and encoding instead of stopping at a successful API response.

The system is in development. Current engineering work centres on processing-flow verification, encoder validation and worker configuration.