Dalet Media Cortex API

The Dalet Media Cortex API allows you to submit media analysis jobs managed by the Dalet Media Cortex service. Media analysis jobs include for example speech-to-text, face recognition, content moderation, OCR or brand detection.

The Dalet Cortex API is a REST API with typed schema for the payload.

In order to start using the Dalet Cortex service, you need to obtain a valid subscription. Start here to get your credentials for Dalet Media Cortex.

Architecture

Dalet Media Cortex job processing is performed on the cloud via dynamic combination of microservices. Dalet Media Cortex adopts the EBU MCMA architecture. The key objectives of this architecture are to support:

• Job management and monitoring
• Long running transactions
• Event based communication pattern
• Service registration and discovery
• Horizontal scalability in an elastic manner

The Dalet Media Cortex architecture is implemented using the serverless approach - relying on independent microservices accessible through well documented REST endpoints and sharing a common object model.

Roles

The following services are involved in the processing of Dalet Media Cortex jobs exposed through the Dalet Media Cortex API:

• Mediator: this is the main entry point to the architecture; this API endpoint supports:

1. Checking authentication using an API key and a token mechanism
2. Verifying quota restrictions before accepting a submitted job
3. Keeping track of usage so that job processing can be tracked and billed
4. Keeping track of jobs metadata as a job repository

• Job Processor: once a job request is accepted by the mediator, it is assigned to a Job Processor. The Job Processor dispatches the job to an appropriate Job Worker (depending on the job profile and other criteria such as load on the system and cost of operation). It then keeps track of the progress of the job and its status until completion and possible failures and timeout. It reports progress to the Mediator through notifications.
• Job Worker: The Job Worker performs the actual work on the media object, for example, AI metadata extraction (AME). It reports progress to the Job Processor through notifications.
• Service Registry: The Service Registry keeps track of all active services in the architecture. It is queried by the Mediator and by Processors to discover candidate services to perform jobs. It is updated whenever a new service is launched or stopped. The Service Registry also stores the list of all job profiles supported by one of the Job Workers deployed in the architecture. The Dalet Cortex API abstracts away from the complexity of this orchestration and provides a simple endpoint to submit long running jobs and monitor the progress of their execution. It serves as a facade for the additional technical services for authentication, usage monitoring and service registry.

When integrated with Dalet Galaxy, the Dalet Media Cortex API is invoked by a dedicated on-premise Media Cortex Indexer agent, which interacts with Dalet Media Cortex and orchestrates the exchange of media on cloud storage and the storage of the returned metadata in the Dalet Galaxy MAM repository.

Getting Started

Subscription

In order to start using the Dalet Mediator services, you need to obtain a valid subscription which provides personal client authorization keys. Start here to get one for Dalet Media Cortex.

We provide client SDK libraries to ease the interaction with the Dalet Media Cortex API in different programming languages (Python, TypeScript, Java). For each language, we also provide a sample application that demonstrates how to perform a media indexing job end to end. In the sample apps below, the credential keys are accessed under the names:

• clientKey
• secretKey

The sample apps reads these keys from a configuration file in a JSON format, for example:

{
      "clientKey": "clientKey1234",
      "clientSecret": "clientSecret1234",
      "projectServiceId": "projectServiceId1234",
      "bucketName": "my-bucket",
      "localPath": ".\Samples",
      "inputFile": "sample.mp4",
      "outputFile": "sample-output.json"
}

Installation

We describe below how to install the Dalet Media Cortex client library for each supported programming language:

Java

If you are using Maven, add the following to your pom.xml file:
<dependency>
      <groupId>com.dalet.mediator</groupId>
      <artifactId>cortex-client</artifactId>
      <version>1.2.0</version>
</dependency>

If you are using Gradle, add the following to your dependencies:
compile 'com.dalet.mediator:cortex-client:1.2.0'

If you are using SBT, add the following to your dependencies:
libraryDependencies += "com.dalet.mediator" % "cortex-client" % "1.2.0"

Python

For more information, see Setting Up a Python Development Environment.

Using pip:
pip install --upgrade git+https://github.com/daletcoreil/cortex-client-python-sdk.git

Using setuptools:

• Download client SDK
• Extract SDK zip file content
• Enter SDK folder cd cortex-client-python-sdk-master
• Run python setup.py install

TypeScript

Using NPM repository:
npm install --save git+https://github.com/daletcoreil/cortex-client-tsnode-sdk.git

Using downloaded SDK:

• Download client SDK
• Extract SDK zip file content
• Enter SDK folder cd cortex-client-tsnode-sdk-master
• Run npm install
• In your project folder, run npm install <Downloaded SDK Folder Path> --save

Usage

Once you have a subscription and your platform is setup with a Dalet Media Cortex client SDK, you can start using Dalet Media Cortex services.

A typical Dalet Media Cortex session to submit and track a job follows these steps:

• Authenticate as an authorized user (this returns a security token used in the following calls)
• Create a Job according to your requirement (this builds a complex data object according to the API data model)
• Submit your job to the Dalet Media Cortex service
• Track progress of the job (tracks status and eventually read the output of your job)

See Sample Application for more details.

Sample Application

The following complete API examples are provided in Java, Python and TypeScript and demonstrate end-to-end interactions with the Dalet Media Cortex API, from job preparation, job submission, job monitoring, up to displaying the received results.

Download the full sample files here:   Java  Python  TypeScript

Upload a media

Upload a media file to your AWS S3 bucket

TransferManager tm = TransferManagerBuilder
    .standard()
    .withS3Client(s3Client)
    .build();
Upload upload = tm.upload(bucketName, bucketKey, mediaFile);
upload.waitForCompletion();

s3 = boto3.client('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, region_name='us-east-1', config=Config(s3={'addressing_style': 'path'}, signature_version='s3v4'))
s3.upload_file(inputFile, bucketName, bucketKey)

const credentials = new AWS.Credentials(aws_access_key_id, aws_secret_access_key);
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    signatureVersion: 'v4',
    s3ForcePathStyle: true,
    credentials: credentials
});
const read: ReadStream = fs.createReadStream(inputMediaFilePath);
const params: AWS.S3.PutObjectRequest = {
        Bucket: bucketName,
        Key: ibucketInKey,
        Body: read
    };
await s3.upload(params).promise();

aws s3 cp FILE_PATH s3://BUCKET_NAME/

Acquire a signed URL

Acquire a signed URL which provides access to the media file located on the S3 bucket

 GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucket, key)
        .withMethod(HttpMethod.GET)
        .withExpiration(expiration));
 URL url = s3Client.generatePresignedUrl(request);

s3 = boto3.client('s3')
s3.generate_presigned_url(ClientMethod='get_object', Params={'Bucket': bucketName,'Key': bucketKey})

const credentials = new AWS.Credentials(aws_access_key_id, aws_secret_access_key);
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    signatureVersion: 'v4',
    s3ForcePathStyle: true,
    credentials: credentials
});
const url: string = s3.getSignedUrl('getObject', {
        "Bucket": BucketName,
        "Key": BucketKey
    });

aws s3 presign s3://BUCKET_NAME/BUCKET_KEY

Prepare job

Prepare a Dalet Media Cortex job requesting the annotation of the media file using the Speech to Text service in the backend

Locator jsonFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_json)
        .httpEndpoint(outputSignedUrl_json);

Locator ttmlFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_ttml)
        .httpEndpoint(outputSignedUrl_ttml);

Locator textFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_text)
        .httpEndpoint(outputSignedUrl_text);

SpeechToTextOutput outputLocation = new SpeechToTextOutput().jsonFormat(jsonFormat).ttmlFormat(ttmlFormat).textFormat(textFormat);

JobInput spJobInput = new SpeechToTextInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language("Spanish")
        .title("1234");

Job aiJob = new Job()
        .jobType(Job.JobTypeEnum.AIJOB)
        .jobProfile(Job.JobProfileEnum.MEDIACORTEXSPEECHTOTEXT)
        .jobInput(spJobInput);

JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId(project_service_id)
        .quantity(file_duration)
        .tracking("tracking")
        .job(aiJob);

input_file: Locator = Locator(bucketName, inputKey, signed_url_input)
jsonFormat: Locator = Locator(bucketName, outputKey_json, signed_url_otput_json)
ttmlFormat: Locator = Locator(bucketName, outputKey_ttml, signed_url_otput_ttml)
textFormat: Locator = Locator(bucketName, outputKey_text, signed_url_otput_text)
output_location: SpeechToTextOutput = SpeechToTextOutput(jsonFormat, ttmlFormat, None, textFormat)
job_input: SpeechToTextInput = SpeechToTextInput(input_file, output_location)
job: Job = Job(job_type='AIJob', job_profile='MediaCortexSpeechToText', job_input=job_input)
jobMediatorInput: JobMediatorInput = JobMediatorInput(project_service_id=project_service_id, quantity=30, job=job)

let inputFile: Locator  = new Locator();
inputFile.awsS3Bucket   = s3Bucket;
inputFile.awsS3Key      = inputMediaFile.name;
inputFile.httpEndpoint  = s3InputSignedUrl;

let jsonFormat: Locator  = new Locator();
jsonFormat.awsS3Bucket   = s3Bucket;
jsonFormat.awsS3Key      = outputFileName_json;
jsonFormat.httpEndpoint  = s3OutputSignedUrl_json;

let ttmlFormat: Locator  = new Locator();
ttmlFormat.awsS3Bucket   = s3Bucket;
ttmlFormat.awsS3Key      = outputFileName_ttml;
ttmlFormat.httpEndpoint  = s3OutputSignedUrl_ttml;

let textFormat: Locator  = new Locator();
textFormat.awsS3Bucket   = s3Bucket;
textFormat.awsS3Key      = outputFileName_text;
textFormat.httpEndpoint  = s3OutputSignedUrl_text;

let outputLocation: SpeechToTextOutput  = new SpeechToTextOutput();
outputLocation.jsonFormat = jsonFormat;
outputLocation.ttmlFormat = ttmlFormat;
outputLocation.textFormat = textFormat;

let jobInput: SpeechToTextInput = new SpeechToTextInput();
jobInput.jobInputType   = SpeechToTextInput.name;
jobInput.inputFile      = inputFile;
jobInput.outputLocation = outputLocation;

let job: Job  = new Job();
job.jobType     = Job.JobTypeEnum.AIJob;
job.jobProfile  = Job.JobProfileEnum.MediaCortexSpeechToText;
job.jobInput    = jobInput;

let jobMediatorInput: JobMediatorInput  = new JobMediatorInput();
jobMediatorInput.projectServiceId   = projectServiceId;
jobMediatorInput.quantity           = inputMediaFile.duration;
jobMediatorInput.job                = job;

Acquire an access token

Given your secret keys, acquire an access token to interact with the Dalet Media Cortex API

AuthApi apiInstance = new AuthApi(Configuration.getDefaultApiClient());
Token token = apiInstance.getAccessToken(client_id, client_secret);

api_instance: AuthApi = mediator_client.AuthApi(mediator_client.ApiClient(Configuration()))
api_instance.get_access_token(client, secret)

const authApi: AuthApi = new AuthApi();
await authApi.getAccessToken(client, secret)).body

curl -X POST \
  https://CORTEX_ADDRESS/auth/access-token \
  -H 'client: CORTEX_CLIENT_KEY' \
  -H 'secret: CORTEX_CLIENT_SECRET'

Submit job

Submit the job to the Dalet Media Cortex API

ApiClient apiClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) apiClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey(token.getAuthorization());

JobsApi jobsApi = new JobsApi(apiClient);
jobsApi.createJob(jobMediatorInput);

api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
api_instance.create_job(job_mediator_input)

const submittedJob: MediatorJob = await jobsApi.createJob(jobInput)).body

curl -X POST \
  https://CORTEX_ADDRESS/jobs \
  -H 'Authorization: BEARER_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "projectServiceId":"PROJECT_SERVICE_ID",
  "quantity": 30,
  "tracking": "EXTRA_CLIENT_INFORMATION",
  "job":   {
    "jobType": "AIJob",
    "jobProfile": "MediaCortexSpeechToText",
    "jobInput": {
      "jobInputType": "SpeechToTextInput",
      "inputFile": {
        "awsS3Bucket": "INPUT_BUCKET_NAME",
        "awsS3Key": "INPUT_BUCKET_KEY",
        "httpEndpoint": "INPUT_SIGNED_URL"
      },
      "outputLocation": {
         "jsonFormat": {
            "awsS3Bucket": "OUTPUT_BUCKET_NAME",
            "awsS3Key": "OUTPUT_BUCKET_KEY",
            "httpEndpoint": "OUTPUT_SIGNED_URL"
         }
      }
    }
  }
}
'

Monitor job status

Monitor the status of the job while it is progressing.
We recommend to perform this call at a frequency of no more than once every 30 seconds.

MediatorJob mediatorJob = jobsApi.getJobById(jobId);
JobMediatorStatus jobStatus = mediatorJob.getStatus();

api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
mediator_job: MediatorJob = api_instance.get_job_by_id(job_id)
mediator_status: JobMediatorStatus = mediator_job.status

const job: MediatorJob = await jobsApi.getJobById(job.id)).body;
let mediatorStatus: JobMediatorStatus = job.status;

curl -X GET \
  https://CORTEX_ADDRESS/jobs/JOB_ID \
  -H 'Authorization: BEARER_TOKEN' \
  -H 'Content-Type: application/json'

Download results

Download the results of the media annotation after the job has completed

File localFile_json = new File(outputFile_json);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_json), localFile_json);

File localFile_ttml = new File(outputFile_ttml);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_ttml), localFile_ttml);

File localFile_text = new File(outputFile_text);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_text), localFile_text);       

s3 = boto3.client('s3')
s3.download_file(bucketName, outputKey_json, outputFile_json)
s3.download_file(bucketName, outputKey_ttml, outputFile_ttml)
s3.download_file(bucketName, outputKey_text, outputFile_text)

const params: AWS.S3.GetObjectRequest = {
    Bucket: bucketName,
    Key: bucketKey
};
const result = await s3.getObject(params).promise();
fs.writeFileSync(inputMediaFile.folder + outputFileName, result.Body);

aws s3 cp s3://OUTPUT_BUCKET_NAME/OUTPUT_BUCKET_KEY FILE_PATH

Delete results

Delete the JSON result file and the media on the S3 bucket after you have safely downloaded them

s3Client.deleteObject(bucketName, bucketKey);

s3 = boto3.client('s3')
s3.delete_object(Bucket=bucketName, Key=inputKey)

const inputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: inBucketKey
};
const outputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: outBucketKey
};
await Promise.all([
    s3.deleteObject(inputParams).promise(),
    s3.deleteObject(outputParams).promise()
]);

aws s3 rm s3://INPUT_BUCKET_NAME/INPUT_BUCKET_KEY
aws s3 rm s3://OUTPUT_BUCKET_NAME/OUTPUT_BUCKET_KEY

Authentication

To use the Dalet Media Cortex API - you must obtain an APIKey from Dalet. This key comes in the form of two parameters:

• client ID
• secret

Given these two parameters, a client program must first obtain an access token (POST /auth/access-token) and then associate this token to all subsequent calls in a header. Dalet Cortex Media API expects the API key to be included in all API requests to the server in a header:

Authorization: meowmeowmeow

When the token expires, the API will return a 401 error code. In this case, the client must request a new token and resubmit the request.

Get authentication token

To get authentication token, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.api.AuthApi;
import com.dalet.mediator.cortex.model.Token;

ApiClient defaultClient = Configuration.getDefaultApiClient();
AuthApi apiInstance = new AuthApi(defaultClient);
Token token = apiInstance.getAccessToken('client_id', 'client_secret');

import cortex_client
from cortex_client.rest import ApiException

def main():
    try:
        api_instance = cortex_client.AuthApi(cortex_client.ApiClient())
        token = api_instance.get_access_token('client_id', 'client_secret')     
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)

import {
    AuthApi,
    Token
} from 'mediator-cortex_client'

const client_id: string = 'myClientId';
const client_secret: string = 'mySecret';

const getAccessToken = async (): Promise<Token> => {
    return (await authApi.getAccessToken(client_id, client_secret)).body;
};

Make sure to replace client_id and client_secret with your client and secret values.

This endpoint retrieves the authentication token.

HTTP Request

POST /auth/access-token

Header Parameters

Parameter	Description
client	Authentication client
secret	Authentication secret

Response

Response	Description	Body
200	Success response	Token

Example:
{
      "Authorization": "Bearer eyJraWQiOiIwZ2FsdS1SQktGUFpPYlRmWFI0b05YVjZaVEt4dFEw",
      "Expires": 86400
}

Errors

Error	Description
401	Invalid secret or client

Jobs

Submit a new job

To submit a new Dalet Media Cortex job, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

String bucket       = "my-bucket-name";
String inKey        = "00000C7B.mp4";
String inSignedUrl  = "https://my-bucket-name.s3.us-east-1.amazonaws.com/media/00000C7B.mp4";
String outputKey_json       = "00000C7B.json";
String outputKey_ttml       = "00000C7B.ttml";
String outputKey_text       = "00000C7B.txt";
String outSignedUrl_json = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.json";
String outSignedUrl_ttml = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.ttml";
String outSignedUrl_text = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.txt";

Locator jsonFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_json)
        .httpEndpoint(outputSignedUrl_json);

Locator ttmlFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_ttml)
        .httpEndpoint(outputSignedUrl_ttml);

Locator textFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_text)
        .httpEndpoint(outputSignedUrl_text);

SpeechToTextOutput outputLocation = new SpeechToTextOutput().jsonFormat(jsonFormat).ttmlFormat(ttmlFormat).textFormat(textFormat);      

JobInput spJobInput = new SpeechToTextInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language("Spanish")
        .title("1234");

Job aiJob = new Job()
        .jobType(Job.JobTypeEnum.AIJOB)
        .jobProfile(Job.JobProfileEnum.MEDIACORTEXSPEECHTOTEXT)
        .jobInput(spJobInput);

JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId(project_service_id)
        .quantity(file_duration)
        .tracking("tracking")
        .job(aiJob);

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");
JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob job = apiInstance.createJob(jobMediatorInput);          

import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

bucketName = 'com-dalet-cortex'
inputKey = 'upload_sample.mp4'
inputURL = 'https://com-my-bucket.s3.us-east-1.amazonaws.com/media/00000C7B.mp4'
outputKey_json      = '00000C7B.json'
outputKey_ttml      = '00000C7B.ttml'
outputKey_text      = '00000C7B.txt'
outSignedUrl_json = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.json'
outSignedUrl_ttml = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.ttml'
outSignedUrl_text = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.txt'

def main():
    try:
        input_file: Locator = Locator(bucketName, inputKey, signed_url_input)
        jsonFormat: Locator = Locator(bucketName, outputKey_json, signed_url_otput_json)
        ttmlFormat: Locator = Locator(bucketName, outputKey_ttml, signed_url_otput_ttml)
        textFormat: Locator = Locator(bucketName, outputKey_text, signed_url_otput_text)

        output_location: SpeechToTextOutput = SpeechToTextOutput(jsonFormat, ttmlFormat, None, textFormat)
        job_input: SpeechToTextInput = SpeechToTextInput(input_file, output_location)

        job = Job(job_type='AIJob', job_profile='MediaCortexSpeechToText', job_input=job_input)
        job_mediator_input = JobMediatorInput(project_service_id='my project service id', \
        tracking='tracking', quantity=30, job=job)

        api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
        mediator_job = api_instance.create_job(job_mediator_input)  
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)       

import {
    JobMediatorInput,
    Locator,
    SpeechToTextInput,
    AIJob,
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

const submitJob = async (jobInput: JobMediatorInput): Promise<MediatorJob> => {
    return (await jobsApi.createJob(jobInput)).body;
};

Make sure to replace input and output parameters by your values.

This endpoint submits a job to Dalet Media Cortex.

HTTP Request

POST /jobs

Header Parameters

Parameter	Description
Authorization	Authentication token

Request body

JSON object of type JobMediatorInput describing the request to be executed.

Response

Response	Description	Body
201	Success response	MediatorJob

Errors

Error	Description
400	Bad request
401	Authorization request fail
502	Failed request. Reason is most likely quota violation.

Get job status

To get a job status, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");

JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob mediatorJob = apiInstance.getJobById(job_id);

import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

mediator_config = Configuration()
# set real authentication endpoint path
mediator_config.host = 'https://u9itw7ttvj.execute-api.us-east-1.amazonaws.com/test'

def wait_for_complete(job_id):
    mediator_job = get_mediator_job(job_id)
    status = mediator_job.status.status
    while status not in ["COMPLETED", "FAILED"]:
        time.sleep(30)
        mediator_job = get_mediator_job(mediator_job.id)
        status = mediator_job.status.status
    return mediator_job

def get_mediator_job(job_id):
    # create an instance of the API class
    api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
    return api_instance.get_job_by_id(job_id)

import {
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

job = (await jobsApi.getJobById(job_id)).body;

Make sure to replace job_idwith your posted job id.

This endpoint gets the status of a running job.

HTTP Request

GET /jobs/{jobId}

Path Parameters

Parameter	Description
jobId	ID of the job as returned in JobMediatorEntity

Header Parameters

Parameter	Description
Authorization	Authentication token

Response

Response	Description	Body
200	Success response	MediatorJob

Errors

Error	Description
404	Job with required ID does not exist



Models

The Dalet Media Cortex API uses the following models.

For all objects defined in these models, mandatory fields are marked in bold.

Token

Name	Type	Description
Authorization	string	Autorization token
Expires	integer	Expires interval in seconds default: 86400

An access token is necessary in order to submit requests to the API.

JobMediatorStatus

Name	Type	Description
status	string	Job status. Can be one of: "NEW", "FAILED", "QUEUED", "SCHEDULED", "RUNNING", "COMPLETED"
statusMessage	string	Last message reported by Job Processor handling this job.
progress	integer	Job progress in percent (from 0 to 100)

Current status of the job as reported by the associated Job Processor if the job is committed. Returned by the GET /jobs/{jobId} call.

JobOutput

Name	Type	Description
jobOutputType	string	Type of AI job

Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

Locator

Name	Type	Description
awsS3Bucket	string	Amazon S3 bucket name
awsS3Key	string	Media file name in Amazon S3 bucket
httpEndpoint	string	Signed URL to the media file in Amazon S3 bucket

Generic description of a file location according to the EBU FIMS specification.

SpeechToTextOutput

Name	Type	Description
jsonFormat	Locator	Location of the output file into which metadata will be stored in JSON format. The JSON format provides a timestamp for each transcription
ttmlFormat	Locator	Location of the output file into which the speech transcription will be stored as subtitles encoded in EBU-TT (TTML) format. See here
srtFormat	Locator	Location of the output file into which the speech transcription will be stored as subtitles encoded in SRT format. See here
textFormat	Locator	Location of the output file into which the text of the speech transcription will be stored in simple format with no timestamps

Specifies where the captions computed by the Speech-to-text service are to be stored. Different formats can be requested (EBU-TT, JSON or text).

MediatorJob

Name	Type	Description
id	string	Job ID computed by the Dalet Media Mediator Job registry. Uniquely identifies the job after it has been accepted by the mediator.
dateCreated	string	Datetime when the job was accepted by the Mediator
dateModified	string	Datetime when the job was last updated by a Job Processor service reporting progress on this job.
quantity	integer	Verified number of units after the job is completed. Units depend on the job profile (usually, they correspond to the duration of the media). Invoicing is based on this value, which may be different from the one submitted by the client.
status	JobMediatorStatus	Current status of the job as reported by the associated Job Processor if the job is committed.
jobOutput	JobOutput	Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

After a job is submitted to the Dalet Media Mediator service, it is attributed metadata that helps track its lifecycle. In addition to the input field (Job input / Job output), the JobMediatorEntity provides a unique ID, date created and last modified and usage information (verified quantity and usage ID to reconcile with invoice). These additional fields are all computed by the mediator and read-only.

JobInput

Name	Type	Description
jobInputType	string	Job input type

Can be either VideoAnalysisInput or SpeechToTextInput

Job

Name	Type	Description
jobType	string	Job type (should be AIJob)
jobProfile	string	Define what type of job is to be executed on the media object.
jobInput	JobInput	Can be either VideoAnalysisInput or SpeechToTextInput

Specific job description according to the job profile.

JobMediatorInput

Name	Type	Description
job	Job	Specific job description according to the job profile.
projectServiceId	string	Customer Project ID. This ID must be provided by Dalet when a Cortex service is provisioned. It must match the authorization token you have been provided.
quantity	integer	Number of units that will be charged for this job. The unit depends on the job profile. For example, for an AI metadata extraction job, units are seconds of media duration.
tracking	JSON	Client metadata associated to the job. This can include any identification fields provided by the client. It should identify the job in a unique manner and is useful to reconcile usage reports with client metadata.
notificationEndpoint	string	Callback URL endpoint to be called once the stage of the job is changed.

Toplevel description of any job submitted to the Dalet Media Mediator service. Encapsulates a service specific job description with authentication, billing and tracking metadata.

VideoAnalysisInput

Name	Type	Description
jobInputType	string	Job input type (should be VideoAnalysisInput)
inputFile	Locator	Input media file location info
outputLocation	Locator	Output result file location info
dataFilter	string[]	Determine which types of AI analyses are to be indexed. Possible values: audioEffects, brands, faces, keywords, labels, ocr, sentiments, shots, textualContentModeration, transcript, visualContentModeration
language	string	Language of the media file to be indexed according to ISO 639-1 Language Name. See here
startOfMaterial	number	Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6

Describe the input and expected output of the job to be executed by Video Analise service. Files are specified using FIMS Essence Locators.

SpeechToTextInput

Name	Type	Description
jobInputType	string	Job input type (should be SpeechToTextInput)
inputFile	Locator	Input media file location info
outputLocation	SpeechToTextOutput	Output result file location info
language	string	Language of the media file to be indexed according to ISO 639-1 Language Name. See here
title	string	Readable name associated to the job when submitted to Speech-to-text service
startOfMaterial	number	Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6
vocabulary	SpeechToTextVocabulary	Custom dictionary

Describe the input and expected output of the job to be executed by Speech-to-text service. Files are specified using FIMS Essence Locators: which media file is to be indexed by the Speech-to-text service, its language, and a user defined name associated to the job. This job produces captions for the media in EBU-TT, JSON and textual formats

Supported languages: Arabic (AR), Bulgarian (BG), Catalan (CA), Croatian (HR), Czech (CS), Danish (DA), Dutch (NL), English (EN), Finnish (FI), French (FR), German (DE), Greek (EL), Hindi (HI), Hungarian (HU), Italian (IT), Japanese (JA), Korean (KO), Latvian (LV), Lithuanian (LT), Malay (MS), Mandarin (CMN), Polish (PL), Portuguese (PT), Romanian (RO), Russian (RU), Slovak (SK), Slovenian (SL), Spanish (ES), Swedish (SV), Turkish (TR)

Optimized segmentation is available for marked in blue

SpeechToTextVocabulary

Name	Type	Description
content	string	content
sounds_like	string[]	list of custom words or phrases that should be recognized

This is a list of custom words or phrases that should be recognized. Custom Dictionary Sounds is an extension to this to allow alternative pronunciations to be specified in order to aid recognition, or provide for alternative transcriptions

Legal Notice

Dalet Media Cortex API documentation
Revision: 1.9.0
Creation: 2020-09-17
This API document explains the basic use of the Dalet Media Cortex API for the management of automatic media indexing jobs.
(c) 2019 - 2020, Dalet, all rights reserved

THIS API SPECIFICATION IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE API OR THE USE OR OTHER DEALINGS IN THE API.