Capturing Audio on BREW Handsets

Capturing Audio on BREW Handsets

oday’s wireless handsets are veritable multimedia powerhouses; sporting more I/O options than the average desktop PC a mere decade ago. Because all users are comfortable talking to a wireless handset (it is, after all, first and foremost a phone!), having the ability to capture audio paves the way for a wide variety of applications, including voice command and speech recognition, audio note-taking, and multi-user chat and messaging in either stand-alone applications or integrated as part of a larger application. Qualcomm BREW makes it easy to capture audio, either as an audio recording you can package up, replay, and send to a remote server, or in real time by directly accessing the handset’s vocoder. This article shows you both methods and arms you with the information you need to know to select which path is right for you.

Using the IMedia Interface to Capture Audio
The IMedia interface?which was covered in in the recent article “Playing Multimedia Using BREW’s IMedia“?not only plays media, but can record media as well. Using the IMedia interface, you can capture audio to a file for storage, playback, and transmission to other hosts. The interface supports a variety of audio formats including PCM, although most handsets currently only support QCELP. (QCELP is a highly optimized format used by the CDMA network itself, and is very well suited for recording the human voice for playback to the human ear, but may not be suitable for other applications that require little or no compression artifacts.) By definition, you can’t use the IMedia interface to directly capture audio off the microphone in real-time; to do this, you use the IVocoder interface, which is the subject of the next section.

Using the IMedia interface to record audio is simple:

  1. Select an appropriate subclass of IMedia based on the desired format for the captured audio (for example, AEECLSID_MEDIAQCP) and create an instance of the interface.
  2. Set the IMedia instance’s callback so that the instance can pass information about the media back to your application using IMEDIA_RegisterNotify.
  3. Configure any recording options using IMEDIA_SetParm.
  4. Begin and control media recording, using handset or programmatic events (key presses, game actions, or so forth) to trigger IMedia methods such as IMEDIA_Record, IMEDIA_Pause, IMEDIA_Resume, and IMEDIA_Stop.
  5. Monitor the values sent to your application callback for information such as errors or the termination of recording and handle those notifications appropriately.
  6. When recording is complete, release the IMedia instance and any other resources you’ve consumed.

Setup of an IMedia interface for recording is the same as for playback:

#include "AEEMedia.h"AEEMediaData sMediaData;char szFilename[10] = "record.qcp";ISHELL_CreateInstance(pIShell, AEECLSID_MEDIAQCP, (void **)&pMe->pIMedia);sMediaData.clsData = MMD_FILE_NAME;sMediaData.pData = (void *)&szFilename;sMediaData.dwSize = 0;IMEDIA_SetMediaData(pMe->pIMedia, &sMediaData);IMEDIA_RegisterNotify(pMe->pIMedia, (*PFNMEDIANOTIFY)IMediaEventHandler, pMe);

This code creates an IMedia instance and sets the file name for the resulting audio recording using IMEDIA_SetMediaData. Finally, it registers a callback the IMedia instance invokes to notify your application when its state changes. Don’t forget to set the PL_FILE privilege in your application’s Module Information File, because IMedia will be interacting with the file system.

You can use the callback to update the user interface for events such as the beginning or ending of recording. The structure of the media callback is well-documented, or you can consult my previous article on IMedia for an example.

Starting and stopping recording can be done programmatically or as a result of a key event; you simply need to call IMEDIA_Record to begin recording and IMEDIA_Stop to end recording. (If you’d like to pause and resume recording, simply use IMEDIA_Pause and IMEDIA_Resume.) The only argument these methods take is the IMedia instance you’ve created and initialized:

// Someplace in an event handler…case AVK_SELECT:if (pMe->bRecording){   pMe->bRecording = FALSE;   IMEDIA_Stop( pMe->pIMedia );}else{   pMe->bRecording = TRUE;   IMEDIA_Recording( pMe->pIMedia );}break;case AVK_SOFT1:if (pMe->bRecording && !pMe ->bPaused ) {   pMe ->bPaused = TRUE;   IMEDIA_Pause(pMe->pIMedia );}else{   pMe ->bPaused = FALSE;   IMEDIA_Resume(pMe->pIMedia );}break;

Of course, you should accompany changes in application state (like a change from recording to paused) with user interface changes, such as showing an icon when recording is paused. While it’s tempting to do this in the same place where you trigger the changes in the IMedia object’s state, it’s actually better to do it in the callback you register with your IMedia object. Doing this ensures that the user interface will only change once the state change for the IMedia object has taken place, and that the user interface cannot get out of sync with the object’s state.

Once recording is finished, you must clean up the IMedia object. To do this, first clear the callback registration and then simply release the object:

IMEDIA_RegisterNotify(pThis->pIMedia, NULL, NULL);IMEDIA_Release(pThis->pIMedia);

Clearing the callback isn’t strictly necessary, but given the asynchronous nature of the IMedia implementation, it’s a good idea.

Once you’ve recorded the sound file, you’re free to do with it as you choose, of course, from sending it to a server, keeping it on the local file system, or analyzing it for features for speech recognition or other purposes.

Using the IVocoder Interface to Capture Audio
The IVocoder interface gives you real-time access to the underlying hardware audio encoder and decoder. Like the IMedia interface, it’s bidirectional?you can use it to both record and play audio. However, the interface is significantly more complex to use, because the audio data is passed continually between the interface and the client of the interface. While this makes simple tasks such as recording an audio clip more difficult (a task better suited to an IMedia instance), it’s the only way to create truly interactive audio applications such as a push-to-talk telephony or an audio chat application.

Although all BREW applications must work asynchronously?performing their work in response to either events or function callbacks invoked by interfaces or timers?the real-time nature of the IVocoder interface places additional demands on your application design. This stems from two related reasons: the time-dependent nature of the data from the vocoder and the implementation of the vocoder itself. First, audio data is sensitive to timing; the vocoder returns data to its client in frames, self-contained units of compressed speech. If you drop a frame, you’ve lost a segment of audio (the result sounds like a digital cell call just as it’s about to drop due to poor coverage). Dropping frames can happen either because of insufficient buffering (you’re not reading from the vocoder fast or often enough, or not writing to it fast or often enough) or because the underlying hardware is occupied by other things. Thus the second challenge: because the vocoder shares processing hardware, it’s important that when using the IVocoder interface you keep other processing to a minimum, especially on low-end hardware. As the vocoder collects audio from the microphone, it compresses the data in real time based on the options you specify, using some of the hardware’s processing resources to do so. (Similarly, replaying audio requires processing to decompress the audio data in real time.)

To use IVocoder, you must:

  1. Create an instance of the IVocoder interface.
  2. Configure the instance by selecting the vocoder type along with the compression options and callbacks for data availability and data playback.
  3. Start and stop the vocoder via the interface according to program control.
  4. Release the vocoder interface instance when you’re through using it.

Vocoder configuration is a little trickier than configuring an IMedia interface, due to the nature of the vocoder itself. First, you must choose a vocoder algorithm (the default used by CDMA cellular networks is IS127). In addition to selecting an algorith, you must select a data rate?that is, how much digital data is generated from the audio channel for a given unit of time. You specify the data rate in terms of the bounds your application requires?the minimum and maximum data rates?and the vocoder dynamically chooses the data rate based on the complexity of the audio stream. This process is adaptive; the vocoder will move to a higher data rate for more complex audio, and throttle back to the minimum data rate when encoding near-silence or audio with fewer compression features. In addition, some audio encoders permit you to specify an additional factor which permits the vocoder to further trim the size of each frame, resulting in lower bandwidth (often with only slightly diminished quality) for a given vocoder rate.

Here is a snippet that creates a vocoder interface and configures it for a typical audio session:

if (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_VOCODER,                           (void **)&pMe->pIVocoder) != SUCCESS)   {      DBGPRINTF("Error Creating IVocoder interface");      return FALSE;   }// Set necessary config data, Description in API   pMe->vocConfig.needDataCB = NeedDataCB;   pMe->vocConfig.haveDataCB = HaveDataCB;   pMe->vocConfig.playedDataCB = NULL;   pMe->vocConfig.readyCB = ReadyCB;   pMe->vocConfig.usrPtr = pMe;   pMe->vocConfig.max = HALF_RATE;   pMe->vocConfig.min = EIGHTH_RATE;   pMe->vocConfig.overwrite = TRUE;   pMe->vocConfig.txReduction = 0;   pMe->vocConfig.vocoder = VOC_IS127;   pMe->vocConfig.watermark = 24;   // Configure IVocoder   status = IVOCODER_VocConfigure(pMe->pIVocoder, pMe->vocConfig,                                   &pMe->vocInfo);

The IVocoder interface requires four callbacks set in in the vocoder configuration structure:

  • The needDataCB, which the vocoder invokes when it needs additional data while playing audio.
  • The haveDataCB, which the vocoder invokes when it has additional data while recording audio.
  • The playedDataCB, which the vocoder invokes each time it has played one or more frames.
  • The readyCB, which the vocoder invokes when it is ready to start or stop audio capture or playback.

To specify both the vocoder algorithm and data rates, Qualcomm provides constants in AEEVocoder.h; this listing configures the vocoder to run at half its maximum data rate at maximum, and an eighth its maximum data rate at minimum. The actual data rates depend on the type of codec; in this case the IS127 codec has been selected, with a maximum data rate of approximately 9kB/sec, so the data rates range between about 4.8kB/s and 1 kB/S.

Finally, because the vocoder interface provides some buffering of data in the event that you cannot service a callback as soon the vocoder requires or has data, you must specify the number of frames to accumulate before engaging the audio path, as well as whether when the vocoder’s buffer overflows new or old frames should be discarded. The example just shown specifies that twenty-four frames should be required (the watermark field) and that new frames should be discarded in the event the buffer is full (overwrite is TRUE).

Once the vocoder is configured, it invokes the callback you specify in the configuration’s readyCB field. At that point, you can begin using the vocoder for input or output. For input, simply invoke IVOCODER_VocInStart; for output, invoke IVOCODER_VocOutStart. In a similar vein, to stop the vocoder, use IVOCODER_VocInStop (to stop capture) or IVOCODER_VocOutStop:

static void ReadyCB(void * usrPtr) {   CApp* pMe = (CVocApp*)usrPtr;   // Start Reading in   IVOCODER_VocInStart(pMe->pIVocoder);// Set Vocoder variable to on   pMe->bVocOn = TRUE;}

This callback starts the vocoder immediately once it’s ready, and sets an internal flag used to track the status of audio capture. You can also update your application’s user interface in response to the callback; I like to do this asynchronously, by sending an event to my application using ISHELL_PostEvent indicating that the UI should be updated in accordance with the new state.

If you are going to repeatedly start and stop the vocoder, it’s a good idea to reset it between each invocation to ensure that it has reset its internal buffers and state:

if (pMe->pIVocoder) {    IVOCODER_VocInStop(pMe->pIVocoder);    IVOCODER_VocOutStop(pMe->pIVocoder);   IVOCODER_VocOutReset(pMe->pIVocoder);    IVOCODER_VocInReset(pMe->pIVocoder); }

Once the vocoder starts, it’s your responsibility to keep collecting data from it and processing the data when it invokes the callback you specified in the configuration’s haveDataCB slot. (In a similar vein if you’re playing data you need to be able to keep up with serving data to the vocoder when it invokes the callback you passed in the needDataCB slot.) What you do with this data (or where you get it, depending on whether you’re recording or playing audio) really depends on your application. For example, if you’re just storing the data for playback later, you might write something like this:

static void HaveDataCB(uint16 numFrames, void *p){   CApp* pMe = (CVocApp*)p;   uint16 length;   int i;   int status;   // Data integrity checks   if (!pMe || !pMe->pIVocoder) return;   // Read in each frame and send to network   for (i = 0; i < numFrames; i++)    {      status = IVOCODER_VocInRead(pMe->pIVocoder,                                  &rate, &length, pMe->arbyFrameData);      // If we succesfully read in data, then write to IVocoder      if (status == SUCCESS)       {         IFILE_Write(pMe->pIFile, pMe-> arbyFrameData, length);      }   }}

The HaveDataCB and NeedDataCB callbacks each take the number of frames available (or that can be accepted in the case of audio playback) along with the user data pointer you specified in the vocoder’s configuration. You should do as little processing as necessary in these callbacks, because they will occur quite often. In this example, I’m simply copying each frame as it arrives to a file on the flash file system; I could just as easily be buffering it in a memory buffer for analysis (perhaps run in the background using BREW’s ISHELL_Resume mechanism) or sending it along a network socket to a server. Regardless, note that I’m copying the frame data into a static memory region arbyFrameData in my application structure, rather than creating this buffer with MALLOC each time the vocoder invokes the callback as a simple optimization. (The buffer is sized according to the constant MAXFRAMELEN defined in AEEVocoder.h.)

Once you’re done with the vocoder, you must stop and release the associated interface. (You can, of course stop and restart the vocoder repeatedly without needing to release the interface.) To do this, simply use the appropriate Stop methods and invoke its Release method:

// We stop incoming and outgoing dataif(pMe->pIVocoder) {   IVOCODER_VocInStop(pMe->pIVocoder);   IVOCODER_VocOutStop(pMe->pIVocoder);   IVOCODER_Release(pMe->pIVocoder);   pMe->pIVocoder = NULL;}

You’ve already seen how to use the Stop methods in a previous snippet; the IVOCODER_Release method is just like that for any other interface.

Both Good Solutions
Qualcomm provides both simple and low-level access to the audio hardware on BREW-enabled handsets via the IMedia interface. If you’re looking to record audio, your best bet is IMedia, which provides a simple wrapper to package audio in a standard format for later playback or to exchange with servers or other uses. On the other hand, if you need real-time, low-level access to the real-time audio stream, it’s easy to build an application with the correct callbacks and use an instance of the IVocoder interface.



Share the Post:
USA Web Development

Top Web Development Companies in USA

Looking for the best web development companies in the USA? We’ve got you covered! Check out our top 10 picks to find the right partner

Clean Energy Adoption

Inside Michigan’s Clean Energy Revolution

Democratic state legislators in Michigan continue to discuss and debate clean energy legislation in the hopes of establishing a comprehensive clean energy strategy for the

Chips Act Revolution

European Chips Act: What is it?

In response to the intensifying worldwide technology competition, Europe has unveiled the long-awaited European Chips Act. This daring legislative proposal aims to fortify Europe’s semiconductor

Revolutionized Low-Code

You Should Use Low-Code Platforms for Apps

As the demand for rapid software development increases, low-code platforms have emerged as a popular choice among developers for their ability to build applications with

Global Layoffs

Tech Layoffs Are Getting Worse Globally

Since the start of 2023, the global technology sector has experienced a significant rise in layoffs, with over 236,000 workers being let go by 1,019

USA Web Development

Top Web Development Companies in USA

Looking for the best web development companies in the USA? We’ve got you covered! Check out our top 10 picks to find the right partner for your online project. Your

Clean Energy Adoption

Inside Michigan’s Clean Energy Revolution

Democratic state legislators in Michigan continue to discuss and debate clean energy legislation in the hopes of establishing a comprehensive clean energy strategy for the state. A Senate committee meeting

Chips Act Revolution

European Chips Act: What is it?

In response to the intensifying worldwide technology competition, Europe has unveiled the long-awaited European Chips Act. This daring legislative proposal aims to fortify Europe’s semiconductor supply chain and enhance its

Revolutionized Low-Code

You Should Use Low-Code Platforms for Apps

As the demand for rapid software development increases, low-code platforms have emerged as a popular choice among developers for their ability to build applications with minimal coding. These platforms not

Cybersecurity Strategy

Five Powerful Strategies to Bolster Your Cybersecurity

In today’s increasingly digital landscape, businesses of all sizes must prioritize cyber security measures to defend against potential dangers. Cyber security professionals suggest five simple technological strategies to help companies

Global Layoffs

Tech Layoffs Are Getting Worse Globally

Since the start of 2023, the global technology sector has experienced a significant rise in layoffs, with over 236,000 workers being let go by 1,019 tech firms, as per data

Huawei Electric Dazzle

Huawei Dazzles with Electric Vehicles and Wireless Earbuds

During a prominent unveiling event, Huawei, the Chinese telecommunications powerhouse, kept quiet about its enigmatic new 5G phone and alleged cutting-edge chip development. Instead, Huawei astounded the audience by presenting

Cybersecurity Banking Revolution

Digital Banking Needs Cybersecurity

The banking, financial, and insurance (BFSI) sectors are pioneers in digital transformation, using web applications and application programming interfaces (APIs) to provide seamless services to customers around the world. Rising

FinTech Leadership

Terry Clune’s Fintech Empire

Over the past 30 years, Terry Clune has built a remarkable business empire, with CluneTech at the helm. The CEO and Founder has successfully created eight fintech firms, attracting renowned

The Role Of AI Within A Web Design Agency?

In the digital age, the role of Artificial Intelligence (AI) in web design is rapidly evolving, transitioning from a futuristic concept to practical tools used in design, coding, content writing

Generative AI Revolution

Is Generative AI the Next Internet?

The increasing demand for Generative AI models has led to a surge in its adoption across diverse sectors, with healthcare, automotive, and financial services being among the top beneficiaries. These

Microsoft Laptop

The New Surface Laptop Studio 2 Is Nuts

The Surface Laptop Studio 2 is a dynamic and robust all-in-one laptop designed for creators and professionals alike. It features a 14.4″ touchscreen and a cutting-edge design that is over

5G Innovations

GPU-Accelerated 5G in Japan

NTT DOCOMO, a global telecommunications giant, is set to break new ground in the industry as it prepares to launch a GPU-accelerated 5G network in Japan. This innovative approach will

AI Ethics

AI Journalism: Balancing Integrity and Innovation

An op-ed, produced using Microsoft’s Bing Chat AI software, recently appeared in the St. Louis Post-Dispatch, discussing the potential concerns surrounding the employment of artificial intelligence (AI) in journalism. These

Savings Extravaganza

Big Deal Days Extravaganza

The highly awaited Big Deal Days event for October 2023 is nearly here, scheduled for the 10th and 11th. Similar to the previous year, this autumn sale has already created

Cisco Splunk Deal

Cisco Splunk Deal Sparks Tech Acquisition Frenzy

Cisco’s recent massive purchase of Splunk, an AI-powered cybersecurity firm, for $28 billion signals a potential boost in tech deals after a year of subdued mergers and acquisitions in the

Iran Drone Expansion

Iran’s Jet-Propelled Drone Reshapes Power Balance

Iran has recently unveiled a jet-propelled variant of its Shahed series drone, marking a significant advancement in the nation’s drone technology. The new drone is poised to reshape the regional

Solar Geoengineering

Did the Overshoot Commission Shoot Down Geoengineering?

The Overshoot Commission has recently released a comprehensive report that discusses the controversial topic of Solar Geoengineering, also known as Solar Radiation Modification (SRM). The Commission’s primary objective is to

Remote Learning

Revolutionizing Remote Learning for Success

School districts are preparing to reveal a substantial technological upgrade designed to significantly improve remote learning experiences for both educators and students amid the ongoing pandemic. This major investment, which

Revolutionary SABERS Transforming

SABERS Batteries Transforming Industries

Scientists John Connell and Yi Lin from NASA’s Solid-state Architecture Batteries for Enhanced Rechargeability and Safety (SABERS) project are working on experimental solid-state battery packs that could dramatically change the

Build a Website

How Much Does It Cost to Build a Website?

Are you wondering how much it costs to build a website? The approximated cost is based on several factors, including which add-ons and platforms you choose. For example, a self-hosted

Battery Investments

Battery Startups Attract Billion-Dollar Investments

In recent times, battery startups have experienced a significant boost in investments, with three businesses obtaining over $1 billion in funding within the last month. French company Verkor amassed $2.1

Copilot Revolution

Microsoft Copilot: A Suit of AI Features

Microsoft’s latest offering, Microsoft Copilot, aims to revolutionize the way we interact with technology. By integrating various AI capabilities, this all-in-one tool provides users with an improved experience that not

AI Girlfriend Craze

AI Girlfriend Craze Threatens Relationships

The surge in virtual AI girlfriends’ popularity is playing a role in the escalating issue of loneliness among young males, and this could have serious repercussions for America’s future. A