Voysis

The Voysis Developer Hub

Welcome to the Voysis developer hub. You'll find comprehensive guides and documentation to help you start working with Voysis as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

JavaScript

Introduction

MIT License

The JavaScript library is made available under the MIT License

To get started, include the JavaScript library on your pages:

<script src="https://cdn.voysis.io/js/voysis.js"></script>

HTTPS Only

Your web page must be loaded using HTTPS, and you must include the JavaScript library using HTTPS. Most modern browsers restrict access to the microphone unless the page (and JS lib) are loaded over HTTPS.

Initialising the Voysis Session

Before you start sending queries, you need to create a VoysisSession. This session can be created once and be used for the lifetime of your webpage. The 2 pieces of information that you must give are your Voysis endpoint and the audio profile identifier (see the section in general concepts for more information on what this value means).

var voysisSession = new VoysisSession({
    host: 'mycompany.voysis.io',
    audioProfileId: 'f8338e44-9d48-11e7-abc4-cec278b6b50a',
    refreshToken: '1.eyJpc3N1ZWRCeSI6IjBkZmI4OGMyLTQ2ZTctNGNjZC05YmZiLTQ3OTQ5NTJmN2I5YyIsImlzc3VlZEF0IjoiMjAxOC0wNC0yN1QwODo0NjozOS40NTRaIiwiZXhwaXJlc0F0IjoiMjAxOC0wNC0yN1QwOTowMTozOS40NTRaIiwiZ2VuZXJhdGlvbiI6MSwicGVybWlzc2lvbnMiOlsiY2VyYmVydXM6bGlvbmVsOmFjY2VzcyJdfQ==.53822ae0527256950896a36570797e3122a1b127e7c3ba85d648a7fce9cc36cd45fa829455579ad6c2ad5ec0ebf958d9bb1680a1e1b322357758c1a70e11b2e6'
});

If you are new to the Voysis JavaScript library, the easiest way to get started after creating the Voysis session is to use the all-in-one convenience method sendAudioQuery.

Asynchronous and Promises

All of the methods available on VoysisSession are executed asynchronously. What is returned from each of the methods is a JavaScript Promise object that will resolve when the asynchronous methods completes successfully, or reject if there is a failure.

Microphone Permissions

All browsers require a user to approve access to their microphone before a webpage can start using it. The JavaScript library will prompt the user for permission the first time they try to stream audio from their browser. Note that the prompt will not appear until streaming audio is initiated; loading the JavaScript library and creating a VoysisSession do not trigger the permission prompt.

Versions

The versioning of the JavaScript library uses the standard MAJOR.MINOR.PATCH semantic versioning schema (for example, voysis-1.1.4.js). In short:

  • MAJOR version changes when there are incompatible API changes with the previous version.
  • MINOR version changes when there is new functionality added in a backwards-compatible manner
  • PATCH version changes when there are backwards-compatible bug fixes.

Minified

A minified version of all JavaScript files will be available with the extension .min.js

Latest Version

voysis.js will always be an alias for the latest library version, so you should use that unless you need a specific version.

Supported Browsers

Not every browser on every operating system supports access to the microphone in order to capture audio.
Below are the list of browser which we have tested that do and don't allow streaming audio from the microphone, ordered by Operating System.

Chrome

:white-check-mark+:

:white-check-mark+:

:x+:

:white-check-mark+:

:white-check-mark+:

Firefox

:white-check-mark+:

:white-check-mark+:

-

-

:white-check-mark+:

Safari

:white-check-mark+:

-

:white-check-mark+:

-

-

Internet Explorer

-

:x+:

-

-

-

Microsoft Edge

-

:white-check-mark+:

-

-

-

Some additional information can be viewed at https://caniuse.com/#feat=stream

Notes

- in the above table means the browser was not available/not tested on the platform specified.

Internet Explorer

Microsoft have stated that web audio streaming won't be added to Internet Explorer. The latest version available at the time of testing was 11.

iOS

Only iOS 11 or later supports web audio streaming - no browser on any prior version of iOS will work.
At the time of testing, only Safari supported web audio streaming. Chrome (61.0) does not support audio streaming.

JavaScript Method

A method is available in the Voysis JavaScript library, isStreamingAudioSupported, to help developers determine if the current browser will support streaming audio from a microphone:

if (!VoysisSession.isStreamingAudioSupported()) {
    alert('Browser not supported');
}

Create Audio Query

Before you start streaming audio to Voysis, you need to create an Audio Query entity that is used to represent the query. (See General Concepts for a description on conversations and queries)

You create an new audio query by calling createAudioQuery on VoysisSession and passing in:

  1. locale: (mandatory) A string representing the locale of the query
  2. context: (optional) An object representing the current context under which the query is being performed.
  3. conversationId: (optional) A string value of the identifier of the conversation that this query will be part of.
  4. audioContext: (optional) An AudioContext instance that will be used to stream audio from.

For example:

var locale = 'en-US';
var context = {
    "keywords": [
        "sprinting",
        "sneakers"
    ],
    "attributes": {
        "color": "red"
    }
};
var conversationId = previousQueryResults.conversationId
voysisSession.createAudioQuery(locale, context, conversationId).then(function (createdAudioQuery) {
    console.log('Created audio query: ' + createdAudioQuery.id);
}).catch(function (error) {
    console.log('Problem creating audio query: ' + JSON.stringify(error));
});

The object passed into to the callback is the created audio query. For example:

{
  "id": "173f19f2-cb86-4bb5-8dc0-fe472d3e9b5e",
  "queryType": "audio",
  "audioQuery": {
    "mimeType": "audio/pcm;bits=16;rate=16000"
  },
  "_links": {
    "self": {
      "href": "/conversations/4827d6f2-bd53-4c95-b094-1561f0776b70/queries/173f19f2-cb86-4bb5-8dc0-fe472d3e9b5e"
    },
    "audio": {
      "href": "/conversations/4827d6f2-bd53-4c95-b094-1561f0776b70/queries/173f19f2-cb86-4bb5-8dc0-fe472d3e9b5e/audio"
    },
    "conversation": {
      "href": "/conversations/4827d6f2-bd53-4c95-b094-1561f0776b70"
    }
  },
  "_embedded": {}
}

Stream Audio

After you have created an Audio Query, you can start streaming data.

You create a new conversation by calling streamAudio on VoysisSession.

Assuming the user can given permissions for the website to access their microphone, the JavaScript library will automatically start streaming audio from the user's microphone to your Voysis endpoint.

The library will automatically detect when the user has stopped speaking, and stop streaming audio.

The streamAudio method takes 2 parameters:

  1. audioQueryResponse: (optional) The response from createAudioQuery
  2. vadStopCallback: (optional) A callback which will be triggered when we have detected that the user has stopped speaking.

When the query has been processed, the query result will be sent to your callback:

voysisSession.streamAudio().then(function (queryResults) {
    console.log('The results to the query were: ' + queryResults);
}).catch(function (error) {
    console.log('Problem streaming audio: ' + JSON.stringify(error));
});

The object passed to the callback will be the results of the query.

See Query Responses for details on the structure and meaning of the fields in the query response.

Below is an example response:

{
  "id": "8a145526-9044-4218-9b6e-066cfef95e79",
  "locale": "en-US",
  "conversationId": "986e79de-6f4e-49dc-a128-41ff41661f3d",
  "queryType": "audio",
  "textQuery": {
    "text": "show me red men's sneakers"
  },
  "audioQuery": {
    "mimeType": "audio/pcm;bits=16;rate=16000"
  },
  "context": {
    "attributes": {
      "color": [
        "red"
      ],
      "gender": [
        "men"
      ]
    },
    "keywords": [
      "sneakers"
    ],
    "price": {},
    "sortBy": ""
  },
  "intent": "newSearch",
  "reply": {
    "text": "Here's what I found"
  },
  "entities": {
    "products": []
  },
  "_links": {
    "self": {
      "href": "/queries/8a145526-9044-4218-9b6e-066cfef95e79"
    },
    "audio": {
      "href": "/queries/8a145526-9044-4218-9b6e-066cfef95e79/audio"
    }
  },
  "_embedded": {}
}

Convenience Method

The sendAudioQuery method on VoysisSession is a convenience method which will create a new audio query and start streaming audio, without you having to do each step individually.

To function takes the same parameters as createAudioQuery, and the callback gets the same query results as streamAudio. For example:

voysisSession.sendAudioQuery('en-us').then(function (queryResult) {
    console.log('You said: ' + queryResult['textQuery']['text']);
}).catch(function (error) {
    console.log("ERROR: " + JSON.stringify(error));
});

The objects passed to the callback will be the result of the query:

{
  "id": "6bd98943-caaa-4058-a32b-9c540cc0f8db",
  "queryType": "audio",
  "textQuery": {
    "text": "show me freshly squeezed orange juice"
  },
  "audioQuery": {
    "mimeType": "audio/pcm;bits=16;rate=16000"
  },
  "intent": "newSearch",
  "reply": {
    "text": "Here's what I found"
  },
  "entities": {
    "keywords": [
      "freshly",
      "squeezed",
      "orange",
      "juice"
    ],
    "products": [],
    "queryString": "freshly squeezed orange juice",
    "sortBy": ""
  },
  "_links": {
    "self": {
      "href": "/conversations/bb30b7de-ae01-4868-9b6a-bf16ecd95126/queries/6bd98943-caaa-4058-a32b-9c540cc0f8db"
    },
    "audio": {
      "href": "/conversations/bb30b7de-ae01-4868-9b6a-bf16ecd95126/queries/6bd98943-caaa-4058-a32b-9c540cc0f8db/audio"
    },
    "conversation": {
      "href": "/conversations/bb30b7de-ae01-4868-9b6a-bf16ecd95126"
    }
  },
  "_embedded": {}
}

Finish Streaming

After calling streamAudio, the response will normally be returned once it is detected that the user has stopped speaking.

You can manually trigger the end of the streaming by calling finishStreaming on VoysisSession.

This will cause streamAudio to immediately work on the response with the audio that has been streamed up to this point in time.

The result will still be sent to the callback from streamAudio.

voysisSession.finishStreaming();

This will also stop the streaming after calling the convenience method sendAudioQuery.

However, as this handles the creation of the conversation and audio query first, it is not known when streaming has begun, and calling finishStreaming before the streaming has actually started will have no effect.

So it is not recommended to use this method to stop sendAudioQuery.

Download

The library can be downloaded from here.

JavaScript