FreeSWITCH          




Page tree
Skip to end of metadata
Go to start of metadata



About

Advanced Voice Mail Detection module is designed to detect the single-frequency constant-amplitude tone in audio signal. Common usage is to detect "beep" sound at the end of the voicemail or answering machine greetings (similar to this 1000 Hz beep [raw] shown above, see also "Dialplan examples" below). This is useful in cases where you wish to leave a recorded message on the recipient's message system but don't want to wait for a pause to complete that is common when using wait_for_silence, and you are looking for a beep sound to start recording. Avmd may (should) also be used on the outbound calls made by dialer, simultaneously with answering machine detection module (AMD), because the detection of voicemail is strong indication of the call being answered by machine and AMD doesn't detect sounds (though in AMD case you often want to be able to detect machine as soon as possible, more quickly than at the end of the voicemail greetings). 

 

 Click to expand Table of Contents

How it works

AVMD uses DESA-2 algorithm to separate estimations of instantaneous frequency and amplitude from TKEO energy operator of an audio signal. DESA-2 uses three sample derivative estimation. This algorithm is fast, computationally simple and a little inaccurate in presence of noise, though good enough for almost all practical uses.

AVMD offers three modes of detection, so that it can be used to detect three types of sound:

  • detection of constant amplitude
  • detection of constant frequency (this may be used to detect dtmf sound)
  • detection of constant amplitude and frequency (default)

The call to AVMD's start method results in launching of detection threads processing audio frames with different resolutions and/or offsets. The avmd::start event is fired after successful start of the AVMD session. The FreeSWITCH core then calls AVMD on each incoming audio frame and all detection threads process incoming frame according to their resolution, offset and lag. Once the signal has been qualified as meeting the specified criteria (of constant amplitude, frequency or both) the avmd::beep event is raised. Application should then stop AVMD running on the call which results in avmd::stop event being fired.

 

 

There are known also other energy separation algorithms including bot not limited to DESA-1, variance-covariance method, PRONY, etc. These algorithms differ in a way energy of differentiated signal is estimated and/or in other adjustments. They are more computationally-expensive methods which have been reported to yield better results in terms of robustness and accuracy, at least at some conditions, by some researchers. Therefore it is possible avmd will be extended in the future to implement some of them. At this moment, avmd uses DESA-2 and this results in a single digit percentage error in a production environments.

Details

This UML diagram depicts processing of audio frames by each of detection threads running in AVMD session, after thread is signaled to start processing by the AVMD main thread in avmd_process function. 

Features added in version 1.6.11

  • multithreaded detectors and lagged detectors
  • three modes of detection (amplitude only, frequency only, both simultaneously)
  • detection mode setting added to configuration XML
  • number of detection threads and the number of lagged detectors settings added to configuration XML
  • extended event API (more information about detection session and about it's outcome added to the events)

 

Running avmd is CPU intensive. There is a tradeoff between detection quality (which increases with number of detection threads) and CPU usage. Please adjust detectors number in AVMD configuration XML or per session to fit your needs and computing resources.

 

Usage

It is as simple as could only be.

  • You "start" the detection on the particular voice channel.
  • The avmd::beep indicating that beep has been detected is delivered to your ESL socket listener, to command prompt and written to logs. Avmd also sets the channel variable (see "Channel variables").
  • You (optionally) "stop" the detection on the particular voice channel.

 

Therefore you will see output similar to this in the console:

Console log (running avmd on given example answering_machine_sound_effect_8000.raw)

 

API interface

This interface is exposed for calls from mod_event_socket.

 

start

Starts avmd detection on a given channel.

 

Usage: <uuid> start

<uuid> - channel unique identifier

 

Examples:

afcbe4db-cdb3-4e1c-98c5-c12fea8474cc start

 

In all of this documentation we omit "api avmd" (used in calls from ESL, telnet client), "avmd" (used in calls from fs_cli, dialplan). It is however assumed that this prefix is always there. In all of this documentation by <x> we mean parameter that is required for a given command and when command is issued the <x> becomes just x. For example, above uuid is channel unique identifier and <uuid> should be understood as "insert uuid of the channel here" (obligatory parameter) when issuing start/stop commands. Extending this example, given the resulting event shown in "Example event" (see below) the following command must have been executed from ESL:

api avmd afcbe4db-cdb3-4e1c-98c5-c12fea8474cc start

or alternatively this form from fs_cli:

avmd afcbe4db-cdb3-4e1c-98c5-c12fea8474cc start

stop

Stops avmd detection on a given channel.

 

Usage: <uuid> stop

<uuid> - channel unique identifier

 

Examples:

afcbe4db-cdb3-4e1c-98c5-c12fea8474cc stop

 

All below commands are available since 1.6.8 release.

show

Prints to console current default/global avmd settings.

 

Usage: show

 

Examples:

show

reload

Reloads avmd XML configuration settings to what FS has in RAM.

 

Usage: reload

 

Examples:

reload

The settings are loaded from FS memory. This means new data will be a copy of what was loaded to FS from $CONFDIR when it was started or when reloadxml command was called last time. If you change XML file and want avmd to load this, it is enough to call FS command reloadxml. But you can also make avmd to load the XML settings file that was cached by FS when reloadxml was called last time. This avmd's reload method will do it, it always restores settings to what FS has in it's RAM, and what FS has in RAM is the version of XML file corresponding to most recent reloadxml command.

set

Sets detection on internal/external channel.

 

Usage: set inbound | outbound | default

inbound - set detection on internal channel

outboud - set detection on external channel

default - set avmd settings to default factory settings (see "Dynamic configuration of avmd session")

 

Examples:

set inbound
set outbound
set default

load

Loads avmd XML settings from RAM and sets detection on internal/external channel.

 

Usage: load inbound | outbound

inbound - set detection on internal channel

outboud - set detection on external channel

 

Examples:

load inbound
load outbound
 

This command is equivalent to reload + set. load inbound == reload + set inbound, load outbound == reload + set outbound.

Sometimes you may want to start avmd a few seconds after answer. In that case, once you get an answer event, issue this command

api sched_api +3 none avmd <uuid> start

For best results in getting accurate detection as well a saving system resources (CPU & memory), you should do the following:

  • Wait until AFTER the call has been answered before starting AVMD. In many cases you may find it best to wait a few seconds after the call is answered before starting AVMD. Starting AVMD before the call is answered will result in waste of CPU resources.
  • Once you have received the AVMD event, you should explicitly stop AVMD, even though it will not return more events.

APP interface

Commands

This interface is exposed for calls from dialplan and scripts (LUA, js). As from 1.6.8 this version of APP should be used:

Available since 1.6.9 release.

APP (command name)Dynamic parameters?Usage (from dialplan)DescriptionNotes
avmd_startno
<action application="avmd_start"/>

starts avmd session with default parameters

 

avmd_startyes
<action application="avmd_start" data=""/>
starts avmd session with default parametersdata="" is legal, though useless
avmd_startyes
<action application="avmd_start" data= "dynamic parameters: comma separated list of key=value options"/>

(e.g. <action application="avmd_start" data="inbound_channel=0,outbound_channel=1, sample_n_continuous_streak=19,sample_n_to_skip=18,debug=0,detectors_n=3,detectors_lagged_n=1,report_status=1"/>)

starts avmd session with parameters set, value of the parameters which have not been specified here is set to default value from XML config

 
avmd_stopno
<action application="avmd_stop"/>
stops avmd session 

When dynamic passing of parameters is used avmd module will validate the comma separated list (which may be empty, that is data="") of key=value pairs for name and value correctness and will set found keys to specified values if validation succeeds. If it doesn't succeed avmd session parameters are set to currently stored default (which means it is set specified in last call to avmd set/ avmd load or otherwise, if none such call has been made - taken from XML config or from default factory settings if config load failed [there is no config or it is invalid]). 

 

Below APP calls will be deprecated since 1.6.8 version but they will still work until the next release for not to break the backward compatibility. When you call one of these methods the warning "YOU ARE USING DEPRECATED APP INTERFACE"  is printed to the logs.

APP (command name)DescriptionUsage (from dialplan)Dynamic parameters?Notes
 avmd starts/stops avmd session dependant on data argument
<action application="avmd" data="start"/>
<action application="avmd" data="stop"/>
 no

DEPRECATED since 1.6.8 version

 

Dialplan examples

 

Dialplan example 1 (running avmd on given example answering_machine_sound_effect_8000.raw)  Expand source

 

Available since 1.6.9 release.

Dialplan example 2 (running avmd on given example answering_machine_sound_effect_8000.raw)  Expand source
Dialplan example 3 (running avmd on given example answering_machine_sound_effect_8000.raw)  Expand source
Dialplan example 4 (running avmd on given example answering_machine_sound_effect_8000.raw in amplitude only mode)  Expand source
Dialplan example 5 (running avmd on given example answering_machine_sound_effect_8000.raw in frequency only mode)  Expand source

 

DEPRECATED since 1.6.8 version

Dialplan Example 6 (running avmd on given example answering_machine_sound_effect_8000.raw)  Expand source

Dynamic configuration of avmd session

When avmd session is started it's parameters are configured with values that are currently stored in FreeSWITCH's  RAM. These values are taken from avmd XML configuration file avmd_conf.xml and can be altered by configuration commands (set/load/reload). There is also a possibility to set each parameter of avmd session's individually, per session. It is done by passing comma separated list of parameters and their values with avmd start command from dialplan (as of 1.6.9 version - it is not available from APP interface, e.g. event socket).

Parameters explained

The meaning of parameters which can be set dynamically is explained in the table below. 

ParameterScopeDescriptionData typeDefault valueNotes
debugGLOBAL

enables/disables logging of avmd intermediate computations to log

uint8_t0 
report_statusGLOBAL

enables/disables verbose logging (and reporting to the console)

of detection status and other diagnostics like parameters avmd

session has been started with, change of configuration parameters,

beep detection status after session ended (stop event is fired

independently of this setting and beep status included there)

uint8_t1 
fast_mathGLOBAL

enables/disables faster computation of arcus cosine - table is created with

mapping of floats to integers and returning arc cos values given these

integer indices into table

uint8_t0

not available on Windows

(setting is ignored)

require_continuous_streakPER SESSION

used in frequency estimation:

enables/disables check of the required number of consecutive elements

(given in sample_n_continuous_streak) in the SMA buffer

without reset needed to classify beep detection as valid

uint16_t1 
sample_n_continuous_streakPER SESSION

used in frequency estimation:

specifies the required number of consecutive elements in the SMA buffer

without reset needed to classify beep detection as valid.

uint16_t3

This parameter helps to avoid false beeps, bigger this value is - smaller

the probability of getting false detection but also smaller the chance of getting

detections of true beeps.

require_continuous_streak_ampPER SESSION

used in amplitude estimation:

enables/disables check of the required number of consecutive elements

(given in sample_n_continuous_streak) in the SMA buffer

without reset needed to classify beep detection as valid

 uint16_t 1 
sample_n_continuous_streak_ampPER SESSION

used in amplitude estimation:

specifies the required number of consecutive elements in the SMA buffer

without reset needed to classify beep detection as valid.

 uint16_t 3This parameter helps to avoid false beeps, bigger this value is - smaller

the probability of getting false detection but also smaller the chance of getting

detections of true beeps.

sample_n_to_skipPER SESSION

specifies the number of samples to skip starting from the beginning

 of the frame and/or after reset has happened.

 

uint8_t0

This parameter serves the purpose of skipping first few estimations in each frame,

as these estimations may be inaccurate. Increasing value of this parameter

(up to certain limit of about 60) may help to give more robust detections in some cases.

simplified_estimationPER SESSION

enables/disables simplified estimation of frequency based on approximation

of sin(x) with (x) in the range x=[0,PI/2]

uint8_t1

Since 1.6.10 release.

In releases before this parameter is ignored.

This would be used only if DESA original algorithm is used

in which arcus cosine computation is needed

per each estimation, but as of 1.6.10 version we use tweaked DESA

algorithm to avoid computation of cosine at all.

 

inbound_channelPER SESSIONenables/disables avmd on internal channeluint8_t0 
outbound_channelPER SESSIONenables/disables avmd on external channeluint8_t1

 

detection_modePER SESSION

sets the characteristics of the sound AVMD will detect, this is the constant

amplitude sound or constant frequency sound or sound characterized

by both constant amplitude and frequency simultaneously

(0 = AVMD_DETECT_AMP, 1 = AVMD_DETECT_FREQ,

2 = AVMD_DETECT_BOTH)

uint8_t2default value is 2 which means AVMD_DETECT_BOTH
detectors_nPER SESSION

sets the number of detection threads running at different resolution and/or

offset within the avmd session

uint8_t36

increasing these numbers gives better detection quality

but can make CPU very busy, it may be needed to decrease

detectors_n value on slow machines. The default setting allows

to handle about 300 sessions peak on 16GB machine with 8 cores

Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz

detectors_lagged_nPER SESSION

sets the number of detection threads which process audio skipping few

frames at the beginning of voice stream

uint8_t1

 

Events

Event types

There exist following types of avmd events:

Event nameDescriptionCustom fieldsUsage (how to subscribe)Notes
AVMD_EVENT_BEEPconstant frequency signal has been detected
  • Unique-ID
  • Call-command
  • Beep-Status
  • Frequency
  • Frequency-variance
  • Amplitude
  • Amplitude-variance
  • Detection-time
  • Detector-resolution
  • Detector-offset
  • Detector-index
event plain CUSTOM avmd::beep

This event is fired only if beep has been detected

 

AVMD_EVENT_SESSION_STARTavmd session has been started
  • Unique-ID
  • Call-command
  • Start-time
event plain CUSTOM avmd::start

 

 

AVMD_EVENT_SESSION_STOPavmd session has been stopped
  • Unique-ID
  • Call-command
  • Beep status
  • Total-time
event plain CUSTOM avmd::stop

 

 

 

To get all avmd events you can subscribe to all FS events and filter them by custom avmd field, a calling command "Call-command":

 

Event examples

Example avmd session start event (from running above dialplan examples)

Example avmd session start event (from running above dialplan examples)  Expand source

Example beep event (from running above dialplan examples)

Example avmd beep event (from running above dialplan examples)  Expand source

Example avmd session stop (from running above dialplan examples)

Example avmd session stop (from running above dialplan examples)  Expand source

 

Channel variables

avmd_detect

When avmd session starts on a given channel, it's channel variable "avmd_detect" is set to FALSE. Avmd sets this variable to TRUE just before AVMD_EVENT_BEEP is fired, so detection result can be checked by inspecting the value of this variable on a given channel. This is not however the supported way of getting detection indications. You should instead subscribe to and process avmd::beep event or check value of Beep-status field in avmd::stop event. Please take a look at "Start AVMD on the call and check detection result" below for more details.

 

Scripts

Few useful scripts written in Perl can be found in AVMD source folder (src/mod/applications/mod_avmd/scripts).

Subscribe to AVMD events

mod_avmd Subscribe to AVMD events example

 

Start AVMD on the call

mod_avmd Start AVMD session on the originated call

Start AVMD on the call and check detection result

mod_avmd Start AVMD session on the originated call and check detection result

Configuration

Add avmd module to 'modules.conf':

applications/mod_avmd

Rebuild and install:

To have FreeSWITCH load it on startup simply add this to '$FS_ROOT/conf/autoload_configs/modules.conf.xml':

Feedback/questions

Any voicemail that avmd doesn't detect? Please do let us know by sending an email to FreeSWITCH™ users mailing list - and we will take care of it (please could you prefix the topic with 'mod_avmd' and address an email to freeswitch-users@lists.freeswitch.org). However, please do not report bugs to users list - for these please could you use jira.freeswitch.org (see Reporting Bugs to JIRA).

1 Comment

  1. Here is console output:

    ------------------------------
    freeswitch@ip-172-30-13-241> uuid_buglist 85f6990e-e3b3-463c-8765-ae645336259c 
    <media-bugs>
    </media-bugs>
     
    freeswitch@ip-172-30-13-241> avmd 85f6990e-e3b3-463c-8765-ae645336259c start
    +OK
     [85f6990e-e3b3-463c-8765-ae645336259c] [sofia/internal/1234@192.168.175.140] started!
     
     
    2017-07-12 01:17:02.428332 [INFO] mod_avmd.c:372 Avmd session initialized, [8000] samples/s
    2017-07-12 01:17:02.428332 [DEBUG] switch_core_media_bug.c:828 Attaching BUG to sofia/internal/1234@192.168.175.140
    2017-07-12 01:17:02.428332 [INFO] mod_avmd.c:1520 Avmd on channel [sofia/internal/1234@192.168.175.140] started!
    2017-07-12 01:17:02.468346 [DEBUG] switch_core_io.c:448 Setting BUG Codec PCMU:0
    freeswitch@ip-172-30-13-241> uuid_buglist 85f6990e-e3b3-463c-8765-ae645336259c
    <media-bugs>
     <media-bug>
      <function>avmd</function>
      <target>N/A</target>
      <thread-locked>0</thread-locked>
     </media-bug>
    </media-bugs>
     
    freeswitch@ip-172-30-13-241> avmd 85f6990e-e3b3-463c-8765-ae645336259c stop   
    +OK
     [85f6990e-e3b3-463c-8765-ae645336259c] [sofia/internal/1234@192.168.175.140] stopped
     
     
    2017-07-12 01:17:14.668350 [DEBUG] switch_core_media_bug.c:1198 Removing BUG from sofia/internal/1234@192.168.175.140
    2017-07-12 01:17:14.668350 [ERR] mod_avmd.c:495 Error, avmd session object not found in media bug!
    2017-07-12 01:17:14.668350 [INFO] mod_avmd.c:1419 Avmd on channel [sofia/internal/1234@192.168.175.140] stopped!
    freeswitch@ip-172-30-13-241> uuid_buglist 85f6990e-e3b3-463c-8765-ae645336259c
    <media-bugs>
    </media-bugs>
    ------------------------------
     

    The issue I want to point is that even when media bug is successfully removed, and everything seems to be working ok, we do see an error logged.