Solutions

Voiceware blocks all TCP/IP connections including any made from Psip-PMS.

This results in no PMS being able to use TCP/IP to connect to any Voiceware system.

You’ll want to first confirm who is the client and who is the server. There’s a known bug in Voiceware version (needToFIndVersion) where Voiceware can only operate as (client or server, need to confirm).

For Voiceware 1.X, 2.X, 3.X run these commands:

iptables -I INPUT -p tcp --dport -j ACCEPT
iptables -I trusted -s -j ACCEPT
service iptables save
or
/etc/init.d/iptables save

For Voiceware 4.X run these commands:

firewall-cmd --permanent --new-zone=pms
firewall-cmd --permanent --zone=pms --add-port /tcp
firewall-cmd --permanent --zone=pms --add-source=
firewall-cmd --reload

To test you can also run these commands:

firewall-cmd --get-active-zones
firewall-cmd --list-all
firewall-cmd --list-all-zones

To test the ports locally on Voiceware you can run these commands:

[root@vw-00301802af83 ~]# netstat -tuwln | grep -i
tcp 0 0 0.0.0.0:5555 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:5556 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:3551 0.0.0.0:* LISTEN
[root@vw-00301802af83 ~]#

[root@vw-00301802af83 ~]# telnet 127.0.0.1
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
[root@vw-00301802af83 ~]#

Troubleshooting with tcpdump and wireshark:

tcpdump -envi port

or

tshark -i -f 'tcp port '

(If you want to use tshark you will have to install tshark it does not come pre-installed like tcpdump does)

Here is an example of a capture from a site where the issue was the connection was closing and reopening again multiple times in the same second.

We can see the TCP/IP flow and where the PMS vendor was sending us a FIN to end communication.

SYN TO

09:36:57.406927 50:6b:8d:8c:35:65 > c4:24:56:c7:a3:10, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 22567, offset 0, flags [DF], proto TCP (6), length 60)10.0.90.60.35194 > 208.98.218.10.telelpathstart: Flags [S], cksum 0x0ed8 (incorrect -> 0xea4d), seq 129419826, win 29200, options [mss 1460,sackOK,TS val 43740008 ecr 0,nop,wscale 7], length 0

SYN-ACK FROM

09:36:57.408496 c4:24:56:c7:a3:10 > 50:6b:8d:8c:35:65, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 122, id 35166, offset 0, flags [DF], proto TCP (6), length 52)208.98.218.10.telelpathstart > 10.0.90.60.35194: Flags [S.], cksum 0xd474 (correct), seq 1528152534, ack 129419827, win 65535, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

ACK TO

09:36:57.408533 50:6b:8d:8c:35:65 > c4:24:56:c7:a3:10, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 22568, offset 0, flags [DF], proto TCP (6), length 40)10.0.90.60.35194 > 208.98.218.10.telelpathstart: Flags [.], cksum 0x0ec4 (incorrect -> 0x1463), ack 1, win 229, length 0

FIN-ACK FROM

09:36:57.410130 c4:24:56:c7:a3:10 > 50:6b:8d:8c:35:65, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 122, id 35167, offset 0, flags [DF], proto TCP (6), length 40)208.98.218.10.telelpathstart > 10.0.90.60.35194: Flags [F.], cksum 0x1145 (correct), seq 1, ack 1, win 1026, length 0

ACK TO

09:36:57.410595 50:6b:8d:8c:35:65 > c4:24:56:c7:a3:10, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 22569, offset 0, flags [DF], proto TCP (6), length 40)10.0.90.60.35194 > 208.98.218.10.telelpathstart: Flags [.], cksum 0x0ec4 (incorrect -> 0x1462), ack 2, win 229, length 0

FIN-ACK TO

09:36:57.428685 50:6b:8d:8c:35:65 > c4:24:56:c7:a3:10, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 22570, offset 0, flags [DF], proto TCP (6), length 40)10.0.90.60.35194 > 208.98.218.10.telelpathstart: Flags [F.], cksum 0x0ec4 (incorrect -> 0x1461), seq 1, ack 2, win 229, length 0

ACK FROM

09:36:57.430077 c4:24:56:c7:a3:10 > 50:6b:8d:8c:35:65, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 122, id 35168, offset 0, flags [DF], proto TCP (6), length 40)208.98.218.10.telelpathstart > 10.0.90.60.35194: Flags [.], cksum 0x1144 (correct), ack 2, win 1026, length 0

From the command line issue the command

psql -Upostgres asgi_cc -c"UPDATE devices SET nat=true, qualify='yes';"

If it works you should see a number of devices updated.

Open

This issue happens when a VM is licensed then cloned or coppied and the license file no longer matches the systems ID.

To correct login and issue the commands:

rm /data/ring/licenses/license.bin
y
voiceware restart all

Bandwidth is the amount of information that can be transmitted, measured in bits per second (bps), or some multiple thereof. When monitoring, you need to distinguish between the nominal data link/Ethernet bit rate, the throughput of a link at Layer 3, and the throughput available to an application.

Bandwidth for audio depends on the sampling frequency (Hertz) and bit depth of each sample. For example, telecommunications links are based on 64 Kbps channels. This was derived through the following calculation:

The voice frequency range is 4000 Hz. This must be sampled at twice the rate to ensure an accurate representation of the original analog waveform.
The sample size is 1 byte (or 8 bits). Therefore, 8 KHz x 8 bits = 64 Kbps.
For VoIP, bandwidth requirements for voice calling can vary, but allowing 100 Kbps per call upstream and downstream should be sufficient in most cases.

Bandwidth required for video is determined by image resolution (number of pixels), color depth, and the frame rate, measured in frames per second (fps).

Problems with the timing and sequence of packet delivery are defined as latency and jitter. Latency is the time it takes for a transmission to reach the recipient, measured in milliseconds (ms). Jitter is defined as being a variation in the delay. Jitter manifests itself as an inconsistent rate of packet delivery. Jitter is also measured in milliseconds, using an algorithm to calculate the value from a sample of transit times.

Latency: The time it takes for a signal to reach the recipient. A video application can support a latency of about 80 ms, while typical latency on the Internet can reach 1000 ms at peak times. Latency is a particular problem for 2-way applications, such as VoIP (telephone) and online conferencing.

Jitter: Variation in the time it takes for a signal to reach the recipient. Jitter manifests itself as an inconsistent rate of packet delivery. If packet loss or delay is excessive, then noticeable audio or video problems (artifacts) are experienced by users.

Latency and jitter are not significant problems when data transfer is burst, but real-time applications are much more sensitive to their effects because they manifest as echo, delay, and video slow down. If packets are delayed, arrive out of sequence, or are lost, then the receiving host must buffer received packets until the delayed packets are received. If packet loss or delay is so excessive that the buffer is exhausted, then noticeable audio or video problems (artifacts) are experienced by users.

You can test the latency of a link using tools such as ping, pathping, and mtr.

You can also use mtr to calculate jitter.

When assessing latency, you need to consider the Round Trip Time (RTT).

VoIP is generally expected to require an RTT of less than 300 ms. Jitter should be 30 ms or less.

The link should also not exhibit more than 1 percent packet loss.

Hunter Jung’s online resource dumping ground. Please don’t litter! (That’s my job)

Classic SysAdmin: How to Securely Transfer Files Between Servers with scp - Linux Foundation

Using SNGREP — FusionPBX Docs documentation

https://www.macmonster.co.uk/

Wireshark · Go Deep

tshark(1)

http://www.postfix.org/

How To Start / Stop / Restart Postfix Mail Server

https://www.arclab.com/en/kb/email/list-of-smtp-and-pop3-servers-mailserver-list.html

https://whatsmyip.com/

https://docs.fortinet.com/document/fortigate/6.0.0/cli-reference/84566/fortios-cli-reference

http://www.subnet-calculator.com/

MAC Address Lookup - MAC OUI IAB IEEE Vendor Search

How NAT traversal works

SQL List All tables

https://www.postgresqltutorial.com/postgresql-show-tables/

https://wiki.asterisk.org/wiki/display/AST/Simple+CLI+Tricks

http://the-asterisk-book.com/1.6/musiconhold.conf.html

The LemmonsLab RS232 Kit

If you hate RS232 you will love this!

Over the years of working with RS232 I have bundled together a very small yet extremely versatile kit that makes working with the mysterious (or some would say painful) serial connections. Below are various pieces and parts that make up that kit. Most of this is inexpensive and every phone tech would benefit from having one in their bag of tricks.

CONSOLE ADAPTER

Let's start simple! Many network devices use the ubiquitous “Cisco Console Cable” for maintenance access. You have surely seen their famous light blue cable with a RJ45 connector on one end and a serial DB9 female on the other. That is a lot of cable to carry around and I have a smaller version you can make for just $3 or so.

Purchase a DB9 female to RJ45 adapter from any of the myriad of suppliers such as https://www.cablewholesale.com/products/serial-modem-cable/modular-adaptors/product-31d1-1740bl.php . I use a blue one to remind me this is a Cisco style console adapter.

These adapters have a standardized color code for each wire and you simply insert wires into the connector body to link the connectors in whatever fashion you wish.

COMPUTER CONSOLE
DB9F RJ45 CAT CABLE
DTR –> 4 BROWN DSR 7 White Brown
CTS <– 8 BLUE RTS 1 White Orange
N/C 1 GREEN GND 5 White Blue
RXD 3 YELLOW RXD 6 Green White
GND — 5 RED GND 4 Blue White
RTS –> 7 WHITE CTS 8 Brown White
DSR 4 BLUE DSR 1 White Orange
CTS <– 8 ORANGE RTS 2 Orange White
N/C 1 BLACK GND 3 White Green
RXD 3 GREEN RXD 5 White Blue
GND — 5 YELLOW GND 6 Green White
RTS –> 7 BROWN CTS 7 White Brown
DSR 4 BLUE DSR 1 White Orange
CTS <– 8 ORANGE RTS 2 Orange White
N/C 1 BLACK GND 3 White Green
RXD 3 GREEN RXD 5 White Blue
GND — 5 YELLOW GND 6 Green White
RTS –> 7 BROWN CTS 7 White Brown
DSR

Components:

CentOS 7 Renamed Voiceware OS

Dispatcher

Docker daemon

Postgresql

Asterisk

Pai

Paiproxy

Psip

Omegadial

Sipnp

Trunktester

Webapps

Polyonymous

Jais

Scheduler

Tftpserver

Reporter

Installation Disk The installation disks are generated by a program called KIWI, currently maintained by the OpenSUSE Team. The installation is streamlined so that it ships with the most basic of components, but does include an onboard copy of voiceware-osconfig package with the repository information baked in. The system comes in ISO (hybrid) and TFTP media. The factory is currently installing via TFTP from a server located on the CMP/Phonesuite HQ network.

Operating System The base of the operating system is CentOS 7. CentOS is a rolling release within major versions, meaning an earlier CentOS releases is compatible with later releases, meaning that upgrades are safer to perform. CentOS 7 is currently supported until June 30, 2024. The voiceware-osconfig package relabels the system as Voiceware OS. While this is not a fully separate distribution of CentOS, the labeling allows us to denote that it is being configuration and maintained by Voiceware.

OS Configuration The main OS configuration is handled by a SaltStack recipe that is distributed by the voiceware-osconfig package stored on the Voiceware download mirror. The recipe is responsible for setting up:

System security (Users/Groups/sudo, firewall, keys and encryption)

Packages (including upgrades)

Various system configurations required for basic use

DAHDI installation

SSH Access The remote login differs slightly from the previous versions of Voiceware. Now instead of using a single user with a single password, we have moved to a more secure SSH Keys. The support user is now support and the phonesuite user is reserved for resellers. When in the field, once the reseller logs in the first time, a secure password is required to be set to continue. If using it in the PSD configuration, this user should be configured and locked before leaving the factory.

Daemon Control / Monit System daemons are no longer controlled by monit and monit is no longer used. SystemD has now taken over most of what Monit did, start/stop/ensure processes stay running. SystemD is also the main process (init, pid 1) for CentOS/Voiceware OS. The systemd utility accepts the following actions: start stop restart status.

System Journal / Logging The new system contains a unified logging system. The logs are initially written to an internal (in-memory) journal before being flushed to disk. The in-memory journal can be accessed with journalctl. This will display the ENTIRE journal that is in memory in less and you can use the familiar less keys to navigate. If the time/date you need is not available in the online journal, the journal is flushed to disk and stored as /var/log/messages, with the familiar rotation. To view only a specific process/services logs, you can use journalctl -u, or you can view a specific syslog tag by using journalctl -t.

Dispatcher An installable daemon (voiceware-dispatcher) that handles communication between unprivileged containerized software and the Operating System. This daemon reads a list of commands from the database and inserts the responses accordingly.

Docker - Docker is a set of products that uses OS-level virtualization to deliver software in packages called containers. Docker is used to manage the Voiceware Software itself. It controls the building of the software packages, deployment (via a person Docker Registry server), starting, stopping, and maintenance.

Docker Storage All user-accessible files are stored in the /data directory on the OS. These files may or may not be updated via automated processes within the containers.

Voiceware Containers - While it is possible to access the containers and modify files directly, this is highly discouraged as the changes may be lost on container restart and will be lost on upgrade. Instead any modifications needing to be made inside the container should be sent to engineering for review.

Containers:

Postgresql This container is the main database process. It stores all of it's data in /data. There are no user-serviceable components. The alias psql is configured to talk to this database for scripting needs.

This container is the main asterisk process. the alias asterisk is configured to talk to asterisk within this container. There is no mechanism to change any configuration files for asterisk.

PAI Pbx Abstraction Interface. The main messaging bus for all telephony components. This container has no serviceable parts

PAIProxy Websockets proxy for the PAI process. Container has no serviceable parts.

PSIP Phonesuite PMS Interface software, originally developed by Frank Melville. Data for this container is stored in /data/psip and should be configured via the web interface. This container has no serviceable parts.

Omegadial Blast dialer used to handle emergency notifications. All of the process data comes from the database and is configured via the web interface. No serviceable parts inside.

SIPNP (ships inside phpapps image) - PNP Responder for Vtech/Snom/Yealink SIP boot provisioning. No serviceable parts, but has extensive logging (journalctl sipnp)

TrunkTester Trunk testing daemon. Configuration comes from the web interface. No serviceable parts.

Webapps (ships inside phpapps image) - Main web interface. Contains both configuration and Browser console components. This image also feeds licenses to all subcomponents, so must be running to enable. Downloads necessary templates and help files on start. No serviceable parts.

Polyonymous (ships inside phpapps image) - Phone Discovery Daemon Listens to ARP requests on the network and attempts to derive make/model/software based on network events. No serviceable parts.

JAIS Main call routing software. Handles all the nitty-gritty call routing details. No serviceable parts.

Scheduler (ships inside phpapps container) - Main cron/scheduling daemon. Handles setting up the database and filesystem on initial start. There is also an unprivileged version of the dispatcher running here that enables cross-container communications.

TFTPServer Contains the TFTP server and (shipped) firmware files. Firmware is copied to /data/tftpboot on container start. No serviceable parts.

Reporter Contains the legacy Asteria queue reporting running on a very old version of ubuntu. Connections to this are proxied via the main Webapps container/web server. No serviceable parts.

Adtran hand-off to second NIC on external server

Example:

Trunk/Adtran provider should supply something like the following:

=======================================================================================================
We have installed our Adtran 908e out on-site and changed this from a PRI to a SIP.

I have the SIP information for you below along with the cut sheet with the phone numbers on the SIP.

SIP TRUNK SETUP
****************
User: 9305402059
Password: LtBha]g-Gf5yW
PBX IP: 10.57.5.2
Gateway: 10.57.5.1
Subnet 255.255.255.0
Registration IP/SIP Domain: 10.35.65.4

Serial: CFG1888946
CONNECTION: ETH0/1 - WAN
CONNECTION: ETH0/2 - PBX

Given the above information, you’ll want to go ahead and static the secondary eth port on the server for an IP of 10.57.5.2, a subnet of 255.255.255.0, and leave the gateway IP blank in most cases you do not want to add the default gateway as this can traffic to route incorrectly.

Once that is done, go into SSH and run “nmcli” to look at the default routes for each interface, you should see something like this:

[root@vw-00301809dc6b ~]# nmcli
enp1s0: connected to Wired connection 1
"Intel I211"
ethernet (igb), 00:30:18:09:DC:6B, hw, mtu 1500
ip4 default
inet4 172.16.0.120/24
inet4 10.11.12.1/24
route4 10.11.12.0/24
route4 172.16.0.0/24
route4 0.0.0.0/0
inet6 fe80::69e8:ebe2:d672:3f8e/64
route6 fe80::/64
route6 ff00::/8

ens1: connected to Wired connection 2
"Intel I211"
ethernet (igb), 00:30:18:09:DC:6C, hw, mtu 1500
inet4 10.57.5.2/24
inet4 10.11.12.1/24
route4 10.11.12.0/24
route4 10.57.5.0/24
inet6 fe80::dbea:5c1c:e9a7:d32/64
route6 fe80::/64
route6 ff00::/8

docker0: unmanaged
"docker0"
bridge, 02:42:4F:7E:7F:71, sw, mtu 1500

lo: unmanaged
"lo"
loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
servers: 8.8.8.8 8.8.4.4 4.2.2.2
interface: enp1s0

Run nmtui and edit the desired interface, in our example, that will be “Wired connection 2” please not the interface is subject to change so look carefully.

[root@vw-00301809dc6b ~]# nmtui

edit the “Wired connection 2” interface and under “Routing” add the following:

Destination/Prefix
10.35.65.4/32

Next Hop
10.57.5.1

Afterwards you’ll want to activate the connection by running nmtui again.

[root@vw-00301809dc6b ~]# nmcli
enp1s0: connected to Wired connection 1
“Intel I211”
ethernet (igb), 00:30:18:09:DC:6B, hw, mtu 1500
ip4 default
inet4 172.16.0.120/24
inet4 10.11.12.1/24
route4 10.11.12.0/24
route4 172.16.0.0/24
route4 0.0.0.0/0
inet6 fe80::69e8:ebe2:d672:3f8e/64
route6 fe80::/64
route6 ff00::/8

ens1: connected to Wired connection 2
“Intel I211”
ethernet (igb), 00:30:18:09:DC:6C, hw, mtu 1500
inet4 10.57.5.2/24
route4 10.57.5.0/24
route4 10.35.65.4/32 <——————- NEW ROUTE IS PRESENT HERE.
inet6 fe80::dbea:5c1c:e9a7:d32/64
route6 fe80::/64
route6 ff00::/8

docker0: unmanaged
“docker0”
bridge, 02:42:4F:7E:7F:71, sw, mtu 1500

lo: unmanaged
“lo”
loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
servers: 8.8.8.8 8.8.4.4 4.2.2.2
interface: enp1s0

you can also look at the routing by running the following command:

[root@vw-00301809dc6b ~]# ip r l
default via 172.16.0.1 dev enp1s0 proto static metric 100
10.11.12.0/24 dev enp1s0 proto kernel scope link src 10.11.12.1
10.35.65.4 via 10.57.5.1 dev ens1 proto static metric 101 <——————- NEW ROUTE IS PRESENT HERE
10.57.5.0/24 dev ens1 proto kernel scope link src 10.57.5.2 metric 101
172.16.0.0/24 dev enp1s0 proto kernel scope link src 172.16.0.120 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown

And if we do a tracepath to the SIP registration gateway we can see it’s routing correctly:

[root@vw-00301809dc6b ~]# tracepath 10.35.65.4
1?: [LOCALHOST] pmtu 1500
1: 10.57.5.1 1.670ms
1: 10.57.5.1 1.322ms
2: no reply

If we go into asterisk and run “sip show registry” we can also see the trunk is now registered:

[root@vw-00301809dc6b ~]# asterisk -r
siAsterisk 16.6.2, Copyright (C) 1999 - 2018, Digium, Inc. and others.
Created by Mark Spencer
Asterisk comes with ABSOLUTELY NO WARRANTY; type ’core show warranty’ for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type ’core show license’ for details.

Connected to Asterisk 16.6.2 currently running on vw-00301809dc6b (pid = 1)

vw-00301809dc6bCLI> sip show registry
Host dnsmgr Username Refresh State Reg.Time
10.35.65.4:5060 N 9305402059 24 Registered Tue, 18 Jan 2022 11:52:55
1 SIP registrations.
vw-00301809dc6bCLI>
Disconnected from Asterisk server
Asterisk cleanly ending (0).
Executing last minute cleanups

“What is that database doing?!”

[root@vw-1866da9fb658 ~]# psql -Upostgres asgi_cc
asgi_cc=# \x
Expanded display is on.
asgi_cc=# select * from pg_stat_activity where state= ’active’;

i like state ’idle’ better

cause you can see if something got hung

At times is might be confusing to a caller to be told to press 0 to reach the front desk operator (i.e. when the system is in use in a assisted living facility).

The two sound files used for guest voicemail can be changed using the process below.

File Explanation

vm-nobodyavail.wav

"Nobody is available to take your call at the moment."

vm-guestintro.wav

"To reach the hotel operator press 0 now, otherwise please leave your messages after the tone, when finished you may hang up or press 1 for more options."

Prep Work

You need to upload a sound file to the sounds page that you want to use for one or both of the above greetings. If you want it to say nothing upload a short file or silence.

Instructions

1. Edit the sound file in question in the sounds page, note the address bar will show a file ID, in the below screenshot its 13, make note of this.

Open

2. Log into the system via SSH, make yourself root.

sudo -i

3. Change directories to the system sound files directory.

cd /data/asterisk/sounds/en

4. Make a backup of one or both of the sound files you intend to replace.

cp vm-guestintro.wav vm-guestintro.wav.bak

cp vm-nobodyavail.wav vm-nobodyavail.wav.bak

5. Copy over the custom sound file using the file number found in step 1 above. Note that all custom sound files start with “s” and then the number.

cp /data/asterisk/sounds/ivr/s13.wav vm-guestintro.wav

cp /data/asterisk/sounds/ivr/s13.wav vm-nobodyavail.wav

6. Change the ownership of the file or files.

chown root:tape vm-guestintro.wav

chown root:tape vm-nobodyavail.wav

Symptoms:
Server is acting “sluggish” from CLI.

The following messages in dmesg are spammed present and spammed repeatedly:

[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!
[Tue Jan 11 19:31:44 2022] neighbour: arp_cache: neighbor table overflow!

Explanation and Solution:

The issue is that there are more clients on the network (1200+ voip endpoints) than we have space to store their MAC addresses in the ARP table.

To fix this, we need to adjust the gc (garbage collection) thresholds for the arp cache.
There are 3 levels:
gc_thresh1
The minimum number of entries to keep in the ARP cache. The garbage
collector will not run if there are fewer than this number of entries in
the cache. Defaults to 128.

gc_thresh2
The soft maximum number of entries to keep in the ARP cache. The
garbage collector will allow the number of entries to exceed this for 5
seconds before collection will be performed. Defaults to 512.

gc_thresh3
The hard maximum number of entries to keep in the ARP cache. The
garbage collector will always run if there are more than this number of
entries in the cache. Defaults to 1024.

There is a small trade-off between the size of the cache and the time-cost of doing ARP requests.
The difference in this network is not large enough to warrant a concern, but large flat- class B networks would need scrutiny.

In our case, we’re going to create a configuration file “/etc/sysctl.d/50-neighbors.conf” where we will increase those limits to /more than/ the number of voip clients, using an 8-bit aligned number.
1.2 * 1024 = 1228 entries
1.5 * 1024 = 1536 entries
2.0 * 1024 = 2048 entries

Therefore we can set our thresholds like so:
net.ipv4.neigh.default.gc_thresh1 = 1024
net.ipv4.neigh.default.gc_thresh2 = 1536
net.ipv4.neigh.default.gc_thresh3 = 2048

Once those are written into the file, we can load it with…
sysctl -p /etc/sysctl.d/50-neighbors.conf

And verify it with:
[root@vw-1866da9fb658 ~]# sysctl net.ipv4.neigh.default | grep thresh
net.ipv4.neigh.default.gc_thresh1 = 1228
net.ipv4.neigh.default.gc_thresh2 = 1536
net.ipv4.neigh.default.gc_thresh3 = 2048

Create swap space out of nothing:

(/dev/zero is a special file in Unix-like operating systems that provides as many null characters (ASCII NUL, 0x00) as are read from it.[1] One of the typical uses is to provide a character stream for initializing data storage.)

dd if=/dev/zero of=/swap bs=1024k count=2048
chmod 0600 /swap
mkswap /swap
swapon /swap

That creates a blank 2G file
Set’s it so no one can read/write it (except root)
Creates a swap signature in /swap
Mounts /swap as swap space

last thing is to add that to your fstab.

edit the /etc/fstab file and add a line:

/swap none swap sw 0 0

Example from a server:

root@voiceware-000c2964df59:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 987M 4.0K 987M 1% /dev
tmpfs 200M 524K 200M 1% /run
/dev/dm-0 63G 18G 45G 29% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1000M 28K 1000M 1% /run/shm
none 100M 0 100M 0% /run/user
/dev/xvda1 472M 100M 348M 23% /boot
root@voiceware-000c2964df59:~# free -h
total used free shared buffers cached
Mem: 2.0G 1.5G 442M 151M 1.5M 571M
-/+ buffers/cache: 983M 1.0G
Swap: 1.0G 0B 1.0G
root@voiceware-000c2964df59:~# swapon -s
Filename Type Size Used Priority
/dev/mapper/voiceware–vg-swap partition 1048572 0 -1
root@voiceware-000c2964df59:~# dd if=/dev/zero of=/swap bs=1024k count=3072
3072+0 records in
3072+0 records out
3221225472 bytes (3.2 GB) copied, 47.2775 s, 68.1 MB/s
root@voiceware-000c2964df59:~# chmod 0600 /swap
root@voiceware-000c2964df59:~# mkswap /swap
Setting up swapspace version 1, size = 3145724 KiB
no label, UUID=6ba96bec-6c26-4355-8896-511ca37ffb45
root@voiceware-000c2964df59:~# swapon /swap
root@voiceware-000c2964df59:~# vi /etc/fstab
root@voiceware-000c2964df59:~#

ortinet
Last edited: 10/11

When to use SIP ALG
Proxy, attached to VW Hosted -> NO

Proxy, direct to carrier, no 4G -> NO

Proxy, direct to carrier, 4G -> YES

Standalone System, no 4G failover -> NO

Standalone System, w/ 4G failover -> YES

Old solution, use info found here: http://174.129.208.130/mediawiki/index.php/SIP_ALG_on_Fortinet_Routers_Info

How to see if SIP ALG is on
Enter “diagnose sys sip status” in the command line of the router
dialogs: max=131072, used=0 [the 0 shows that its turned off]

(Added by TJB 10/25/2019)

Email from Branden on how to actually disable ALG?

[Fortinet] Disable SIP ALG

config system session-helper
delete 13
end

config system settings
set default-voip-alg-mode kernel-helper-based
end

config voip profile
edit default
config sip
set status disable
end
end

This will remove ALL e911 callback entries and metadata.

That should release your DIDs.

[root@vw-00224dd46264 ~]# psql -h localhost -Upostgres asgi_cc -c 'truncate e911_callback;'

response:
TRUNCATE TABLE

Example/Symptom:

A user/reseller calls in and states that Asterisk keeps stopping or is unable to start using asterisk -r.

Or maybe many processes are stuck trying to “initialize” if you run a “sudo monit status” command.

This could be an indication that the disk usage on a particular server has been exceeded and is now full.

Solution:

To start addressing this issue you will want to begin uncovering what partition of the disk has the highest amount of usage. From there you can start the clean up process and allow the server to begin operating normally again.

Steps:

To begin you will want to run the following command to find the partition you need to look at. In the below example you can clearly see that /dev/dm-0 has run out of available space and is at 100% disk usage.
The directory you will want to focus on is the / (root) directory.

1. phonesuite@voiceware-00224dd555ae:~$ df -h
   Filesystem Size Used Avail Use% Mounted on
   udev 979M 4.0K 979M 1% /dev
   tmpfs 199M 728K 198M 1% /run
   /dev/dm-0 58G 58G 0M 100% /
   none 4.0K 0 4.0K 0% /sys/fs/cgroup
   none 5.0M 0 5.0M 0% /run/lock
   none 992M 28K 992M 1% /run/shm
   none 100M 0 100M 0% /run/user
   /dev/sda1 472M 42M 406M 10% /boot
   phonesuite@voiceware-00224dd555ae:~$

2. Use the cd command to change directories: cd /

3. You can then use: sudo du -shc /* to discover what sub directory(s) are consuming all the space.

4. In most cases this is due to the /var/log directory where all of the system log files exist.

5. To check /var/log run...sudo du -shc /var/log/*

6. You can then remove bloat log files as need be but please be careful, you may want to consider just finding the largest log file, removing it, then taking a back up of the directory and cleaning up the rest of the log files.

7. To remove a log file see the commands below.

8. cd /var/log

9. sudo rm fileNameHere (use extreme caution when using the rm command, it can be a very dangerous command if you do not know how to use it properly and I do not recommend using it with wild cards (*./$ etc..) unless you are experienced).

10. Assuming the issue was within the /var/log directory, you may want to check /etc/logrotate.d/ and verify that log files are being rotated properly. To do so, see the commands below.

11. ll /etc/logrotate.d/
    11a. Expected result:

phonesuite@voiceware-00224dd555ae:~$ ll /etc/logrotate.d/
total 88
drwxr-xr-x 2 1005 1005 4096 Jan 23 2018 ./
drwxr-xr-x 128 1828600001 1828600000 8192 Jun 14 15:21 ../
-rw-r–r– 1 root root 434 Apr 18 2018 apache2
-rw-r–r– 1 root root 173 Apr 10 2014 apt
-rw-r–r– 1 root root 79 Feb 17 2014 aptitude
-rw-r–r– 1 1005 1005 170 Jan 23 2018 asteria
-rw-r–r– 1 1005 1005 1455 Jan 23 2018 asterisk
-rw-r–r– 1 1005 1005 84 Jan 23 2018 audit
-rw-r–r– 1 root root 135 Mar 4 2013 consolekit
-rw-r–r– 1 root root 232 Mar 7 2014 dpkg
-rw-r–r– 1 root root 338 Nov 17 2013 fail2ban
-rw-r–r– 1 root root 268 Nov 16 2013 monit
-rw-r–r– 1 root root 173 May 10 2018 postgresql-common
-rw-r–r– 1 root root 94 Apr 21 2015 ppp
-rw-r–r– 1 1005 1005 195 Jan 23 2018 psip-pms
-rw-r–r– 1 root root 515 Dec 4 2013 rsyslog.disabled
-rw-r–r– 1 root root 298 Nov 14 2013 stunnel4
-rw-r–r– 1 root root 519 Dec 25 2013 syslog-ng
-rw-r–r– 1 root root 178 Feb 28 2014 ufw
-rw-r–r– 1 root root 122 Apr 11 2014 upstart
phonesuite@voiceware-00224dd555ae:~$

1. In the above example, you can see that asteria, asterisk, audit, and psip-pms log files have incorrect ownership (1005) whereas all other files have the correct ownership (root)

2. To fix the above issue you will want to run: chown root:root /etc/logrotate.d/*
   13a. Expected result once you run the above command and ll the directory again:

phonesuite@voiceware-00224dd555ae:~$ ll /etc/logrotate.d/
total 88
drwxr-xr-x 2 1005 1005 4096 Jan 23 2018 ./
drwxr-xr-x 128 1828600001 1828600000 8192 Jun 14 15:21 ../
-rw-r–r– 1 root root 434 Apr 18 2018 apache2
-rw-r–r– 1 root root 173 Apr 10 2014 apt
-rw-r–r– 1 root root 79 Feb 17 2014 aptitude
-rw-r–r– 1 root root 170 Jan 23 2018 asteria
-rw-r–r– 1 root root 1455 Jan 23 2018 asterisk
-rw-r–r– 1 root root 84 Jan 23 2018 audit
-rw-r–r– 1 root root 135 Mar 4 2013 consolekit
-rw-r–r– 1 root root 232 Mar 7 2014 dpkg
-rw-r–r– 1 root root 338 Nov 17 2013 fail2ban
-rw-r–r– 1 root root 268 Nov 16 2013 monit
-rw-r–r– 1 root root 173 May 10 2018 postgresql-common
-rw-r–r– 1 root root 94 Apr 21 2015 ppp
-rw-r–r– 1 root root 195 Jan 23 2018 psip-pms
-rw-r–r– 1 root root 515 Dec 4 2013 rsyslog.disabled
-rw-r–r– 1 root root 298 Nov 14 2013 stunnel4
-rw-r–r– 1 root root 519 Dec 25 2013 syslog-ng
-rw-r–r– 1 root root 178 Feb 28 2014 ufw
-rw-r–r– 1 root root 122 Apr 11 2014 upstart
phonesuite@voiceware-00224dd555ae:~$

Detailed Explanation:
The log rotation file may not be setup or have proper permissions, see above for what should be listed in the logrotate.d directory had incorrect ownership.

Symtpom(s):
It has been noted that when using a TA2400 GW for room phones there is sometimes a lack of ring-back present on outbound calls that are traveling over T1. This is because the carrier /should/ be providing the ring back but since they are not we have to let Voiceware know that we should be providing the ring-back.

Version(s):
4.X

Summary:
[root@vw-00032d3ca462 ~]# cd /tmp/

[root@vw-00032d3ca462 tmp]# docker cp asterisk:/etc/asterisk/asg_callrouter.conf .

Then edit the file, line 255.

root@vw-00032d3ca462 tmp]# vi asg_callrouter.conf +255

exten => s,n(dodial),Dial(${TRUNK}/${FILTER(+0123456789*#w,${ARG1})},90,${ARG2})
Add the option to the ARG2 section.

exten => s,n(dodial),Dial(${TRUNK}/${FILTER(+0123456789*#w,${ARG1})},90,R${ARG2})
Please note this goes outside of the ${...} , but after the last comma.

After you've finished with the file, copy it back into place.

[root@vw-00032d3ca462 tmp]# docker cp asg_callrouter.conf asterisk:/etc/asterisk/

[root@vw-00032d3ca462 tmp]# asterisk -rx 'dialplan reload'

Dialplan reloaded.

FORTIGATE | ISP Cutover Quick Script

====================================

SCRIPT | CUTOVER IP CHANGES

====================================

config system interface

edit “wan1”

set vdom “root”

set ip [NEW IP] [NEW SN]

end

end

config system virtual-wan-link

config members

edit 1

set interface “wan1”

set gateway [NEW GW]

next

end

end

config router static

edit 1

set gateway [NEW GW]

next

end

end

====================================

AWS IPS TO WHITELIST/CHANGE

====================================

Custom UDP 123

CUSTOM UDP 5060

HTTP TCP 80

HTTPS 443

Custom TCP 8080

To setup the SMTP with Voiceware 4.X using a Gmail account, you will need to do the following:

Create a gmail account for SMTP relay setup

Example account and credentials that are already created:

UN: pss.voiceware@gmail.com
PW: ukK]ntH%

1. Sign into gmail.

2. At the top right hand corner click on your Gmail icon and click on “Manage your Google Account”.

3. Left hand side click on “Security”.

4. Scroll down to and enable “2-step verification” (will need this enabled for first time setup, if already enabled skip this step).

5. Once setup is done, go back under security and now a new field called “App passwords” is available.

6. CLick on “App passwords”

7. Login again with gmail password

8. Select app

9. Select device

10. Name it: Voiceware SMTP (or other sensical name)

11. Save password from pop window “Your app password for your device”

Example:

vbpylyoepriiafcl

Use new credentials under voiceware settings:

*Destination
*Gateway Host:
smtp.gmail.com
Port:
Username
Password

1. SSH into Voiceare.

2. Become root:

sudo -i

1. Call postgres:

psql asgi_cc postgres

1. Select all from the E911_callback table:

select * from e911_callback;

1. Update the stop time for all records where “stop” is NULL:

update e911_callback set stop = ’2022-08-13 15:46:00.00000’ where stop is NULL;

-bash-4.2$ sudo -i
[root@vw-00301809dcc3 ~]# psql asgi_cc postgres
psql (9.2.24, server 10.18 (Debian 10.18-1.pgdg90+1))
WARNING: psql version 9.2, server version 10.0.
Some psql features might not work.
Type “help” for help.

asgi_cc=# select * from e911_callback;
id | etn | start | stop | channel | call_id
—-+—–+—————————-+——+———+——————————
1 | 208 | 2022-08-21 12:36:42.385952 | | SIP/320 | PhoneSuite-1661103401.232408
(1 row)

asgi_cc=# update e911_callback set stop = ’2022-08-13 15:46:00.00000’ where stop is NULL;
UPDATE 1
asgi_cc=# \q
[root@vw-00301809dcc3 ~]#

Add label

Example/Symptom:

A user/reseller calls in stating that the FD cannot call rooms or vice versa.

Solution:

To start addressing this issue you will want to monitor asterisk while test calls are placed and look for any further clues. From there you may also want to check and verify the status of the span as it could be down i.e. the SIP FD phones are unable to call the room phones which are on virtual ports and vice versa.

Steps:

1. To check the status of the span go into: Devices/Hardware Spans
   1a. Expected result:
   1 (DYN:eth1/003018a4c908) PHY:OK / LOG: Up, Active Yes Connected to PhoneSuite Cabinet 23 / 23
   1b. NOTE: You may see more than 1 span built if this is an external server with multiple Series 2 cabinets “blue boxes” if it’s a large property with many rooms.

2. Try restarting the span by clicking on the “RESTART NOW” button.

3. Try rebuilding the span.
   3a. Click: Edit Span
   3b. Take a screenshot of the span settings.
   3c. Delete the span.
   3d. Click: Add Span
   3e. Recreate the span with the same settings referencing your screenshot if needed.

Some Useful Commands:

ip a l
tcpdump -ei eth1

span 1 should ALWAYS be the hardware card when one exists

root@voiceware-00224dd55752:~# dahdi_alarms
1: OK Wildcard TE132/TE134 Card 0

You can fix it by either finding the magic setting that tells the router not to close the ports so quickly.

Or generating a ton of traffic.

docker cp asterisk:/etc/asterisk/sip.conf /tmp

edit /tmp/sip.conf

edit keepalive=29 > keepalive=2

docker cp /tmp/sip.conf asterisk:/etc/asterisk/sip.conf

asterisk -rx ’sip reload’

Case number #: 26065

Serial Loop back test (short out pins 2 & 3 on DB9 or DB25 )

This can help determine if the transmit & receive pins are working on the serial port and the cable being used.

\1 2 3 4 5/ Looking at pins \1 2 3 4 5 6 7 8 9 10 11 12 13/
\6 7 8 9/ on male connector \14 15 16 17 18 19 20 21 22 23 24 25/
—— ———————————–

9-pin 25-pin
3 2 TxD Transmit Data –> Transmits bytes out of PC
2 3 RxD Receive Data RTS/CTS flow control
8 5 CTS Clear To Send <– RTS/CTS flow control
6 6 DSR Data Set Ready I’m ready to communicate
1 8 DCD Data Carrier Detect<– Modem connected to another
9 22 RI Ring Indicator

the purpose of this solution is to document any sort of PMS related configuration changes that might come up and that others might run into.

Symptom:
Last Name First Name Affiliation VIP? Current Room DID
( ) 3947P6-IMBRIANI/JOSE No Room 934
None
( ) 3929X4-BOUGHTON/NAT/ No Room 925
None
( ) 3933OFLAHERTY/FIACHR No Room 614
None
( ) 3934OFLAHERTY/OISIN/ No Room 633

Solution:
There is a box called Additional Parameters in the PMS setup screen, add the below three lines in that box and restart the interface, that should help. That said we still might run into issues. The Psip PMS code was designed to take a guest's first and last name only. When a Mr or Ms is added to the name as a separate field there might be some issues with the data parsing. Let's try the below fix first and then see where that leaves us. After that there are some other adjustments we can look at doing.

[pbx-masks]
chkdelim=11 20 MASK_LITERAL /
namdelim=6 20 MASK_LITERAL /

Case#:
00026463

Symptom:

Inbound/outbound/device to device calling has a 10, 20, 30 second or longer delay.

You can see this in asterisk by becoming root and entering the asterisk CLI:

login as: phonesuite
phonesuite@10.110.240.28’s password:

UA Infrastructure Extended Security Maintenance (ESM) is not enabled.

1 update can be installed immediately.
0 of these updates are security updates.
To see these additional updates run: apt list –upgradable

Enable UA Infrastructure ESM to receive 225 additional security updates.
See https://ubuntu.com/advantage or run: sudo ua status

Your Hardware Enablement Stack (HWE) is supported until April 2019.
Last login: Fri Jun 10 14:33:48 2022 from 10.253.255.2

** Welcome to Voiceware **

phonesuite@voiceware-00301802af45:~$ sudo -i
[sudo] password for phonesuite:
root@voiceware-00301802af45:~#

root@voiceware-00301802af45:~# asterisk -r
[Jul 7 14:35:12] Asterisk 11.25.1-1675634.STABLE, Copyright (C) 1999 - 2013 Digium, Inc. and others.
[Jul 7 14:35:12] Created by Mark Spencer
[Jul 7 14:35:12] Asterisk comes with ABSOLUTELY NO WARRANTY; type ’core show warranty’ for details.
[Jul 7 14:35:12] This is free software, with components licensed under the GNU General Public
[Jul 7 14:35:12] License version 2 and other licenses; you are welcome to redistribute it under
[Jul 7 14:35:12] certain conditions. Type ’core show license’ for details.
[Jul 7 14:35:12] =========================================================================
[Jul 7 14:35:12] Running as user ’asterisk’
[Jul 7 14:35:12] Running under group ’audio’
[Jul 7 14:35:12] Connected to Asterisk 11.25.1-1675634.STABLE currently running on voiceware-00301802af45 (pid = 2614)
[Jul 7 14:35:13] PRI Span: 1 TEI=0 Sending SABME
voiceware-00301802af45*CLI>

While monitoring the CLI you will have to either have the customer or reseller help you place test calls if you are unable to register to a SIP device remotely for your own testing.

Test inbound, outbound, and device to device to verify if the delay is on the carrier/trunk, TDM, or system wide.

Some notable causes for the delayed dialing have been:

1. Database.

2. Broken DNS.

3. Many DIDs assigned to the same extension number.

4. STUN server ports and protocols being blocked.

5. Vacuum solution for database related delayed calling:
   root@voiceware-00301802af45:~# psql -Upostgres asgi_cc -c ’vacuum full analyze’
   VACUUM
   root@voiceware-00301802af45:~#

6. Test for DNS:
   Note that the firewall might be blocking ICMP packets but if ping is still able to resolve http://goog.com (as you can see below it resolves to 142.251.32.46) then the DNS should be working.
   root@voiceware-00301802af45:~# ping http://google.com
   PING http://google.com (142.251.32.46) 56(84) bytes of data.
   64 bytes from sfo03s26-in-f14.1e100.net (142.251.32.46): icmp_seq=1 ttl=48 time=33.3 ms
   64 bytes from sfo03s26-in-f14.1e100.net (142.251.32.46): icmp_seq=2 ttl=48 time=36.4 ms
   64 bytes from sfo03s26-in-f14.1e100.net (142.251.32.46): icmp_seq=3 ttl=48 time=33.7 ms
   ^C
   — http://google.com ping statistics —
   3 packets transmitted, 3 received, 0% packet loss, time 2001ms
   rtt min/avg/max/mdev = 33.302/34.477/36.413/1.395 ms
   root@voiceware-00301802af45:~#

7. Look at the dial plan and verify if there are a large number of DIDs pointing to the same location.

8. Test connection to STUN server:
   bash-4.2$ ./stun sats.voiceware.com
   STUN client version 0.97
   Primary: Blocked or could not reach STUN server
   Return value is 0x00001c
   -bash-4.2$

Need to allow outbound connections from the PBX/Voiceware to port 3478 (tcp/udp) and 5349 (tcp) on sats.voiceware.com and *.sats.voiceware.com

If you do not have access to the Voiceware GUI and need to get the Zen ID off a server, follow the steps below.

1. SSH/PuTTY into the Voiceware server

2. Switch to root user: sudo su

3. type: /usr/local/Zend/bin/zendid

Example:

phonesuite@voiceware-000c2964df59:/$ sudo su
[sudo] password for phonesuite:
root@voiceware-000c2964df59:/# /usr/local/Zend/bin/zendid
M:SMW2X-AQ4N4-SZYGX-X8D8W

4.X:

[root@ip-172-31-4-183 logs]# ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 02:86:39:45:14:e3 brd ff:ff:ff:ff:ff:ff
inet 172.31.4.183/20 brd 172.31.15.255 scope global noprefixroute dynamic eth0
valid_lft 2864sec preferred_lft 2864sec
inet6 fe80::86:39ff:fe45:14e3/64 scope link
valid_lft forever preferred_lft forever

3: docker0: mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:83:d4:38:f7 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever

Verify if turned off:
[root@ip-172-31-17-136 ~]# asterisk -rx ’sip show settings’ | grep T.38
T.38 support: No
T.38 EC mode: Unknown
T.38 MaxDtgrm: 4294967295
[root@ip-172-31-17-136 ~]#

Edit asterisk container file:
[root@ip-172-31-17-136 ~]# docker exec -it asterisk bash
[root@ip-172-31-17-136 /]# vi +754 /etc/asterisk/sip.conf
[root@ip-172-31-17-136 /]#

remove the “;”

Before:
;t38pt_udptl = yes ; Enables T.38 with FEC error correction.

After:
t38pt_udptl = yes ; Enables T.38 with FEC error correction.

esc:wq

[root@ip-172-31-17-136 /]# exit
[root@ip-172-31-17-136 ~]# asterisk -rx 'sip reload'
[root@ip-172-31-17-136 ~]#

Verify change:
[root@ip-172-31-17-136 ~]# asterisk -rx 'sip show settings' | grep T.38
T.38 support: Yes
T.38 EC mode: FEC
T.38 MaxDtgrm: 4294967295
[root@ip-172-31-17-136 ~]#

Access logs:
grep -i login /var/log/apache2/access.log <— 3.X
grep -i login /var/log/messages

Example/Symptom:

A user is forwarding calls out of the server to their cell phone. When someone calls a user DID that is currently set to forward calls to their cell phone, the call builds and connects both parties but there is no audio.

Solution:

Add a 1-second delay or 1-second of silence before the call out-routes from the server to the carrier.

Steps:

1. SSH into VW

2. Run: sudo vi +215 /etc/asterisk/asg_callrouter.conf

3. Add: exten => s, n, Playback(silence/1)

4. type: :wq to save and exit vi editor

Prior to editing the asg_callrouter.conf file:

[macro-cr-outbound-call]
;number, options
exten => s,1,NoOp(Outbound Call)
exten => s,n,Set(OUTBOUND=TRUE)
exten => s,n,Set(CDR(outbound)=true)
exten => s,n,ExecIf($["${call_log_id}" != ""]?ForkCDR())
exten => s,n,Set(CALLERID(num)=${CALLERID(num)}) ; Previously CALLERID(num):-10
exten => s,n,Set(REDIRECTING(from-number-valid)=0)
exten => s,n,Dial(${TRUNK}/${ARG1},90,${ARG2})
exten => h,1,Macro(handlehangup)

After editing the asg_callrouter.conf file:

[macro-cr-outbound-call]
;number, options
exten => s,1,NoOp(Outbound Call)
exten => s,n,Set(OUTBOUND=TRUE)
exten => s,n,Set(CDR(outbound)=true)
exten => s,n,ExecIf($["${call_log_id}" != ""]?ForkCDR())
exten => s,n,Set(CALLERID(num)=${CALLERID(num)}) ; Previously CALLERID(num):-10
exten => s,n,Set(REDIRECTING(from-number-valid)=0)
exten => s,n,Playback(silence/1)
exten => s,n,Dial(${TRUNK}/${ARG1},90,${ARG2})
exten => h,1,Macro(handlehangup)

After making changes enter asterisk (asterisk -r) and run: dial plan reload

Detailed Explanation:

When the SIP protocol is doing a call setup (and answer) it requires that the servers send their IP address for audio in the session description. Sometimes, phones, misconfigured asterisk servers, and other dumb devices don’t realize they need to send an external IP address and blindly send whatever address they have on hand.

This means that you will send the audio to nowhere. 
 
When you check the NAT box, you tell Asterisk to ignore (most of) what it’s told, at least for signaling. With audio, asterisk will still gladly send audio to nowhere. This is because the audio server is not always the signaling server. It’s still possible to offload audio.

What causes asterisk to “lock on” is the far end device when it sends audio to asterisk, with the correct ID, to the correct port. After a probation period, asterisk will then send audio to the same address it received from.

Further Distillation:
Asterisk sometimes requires inbound audio to come in first before it will even attempt to send audio out. Since all of the AWS instances are behind NAT, you get weird hairpin audio issues. As the call comes in from the carrier, the ports for audio are closed; sending audio to the carrier (or audio relay) will open them.

If asterisk does not want to send audio first, those ports never open, so asterisk refuses to send audio. Asterisk then sets up an external call but refuses to send audio because there is nothing to say yet. The outbound call has the same problem. The ports are closed because asterisk hasn’t said anything yet.
 
When you send the silence/1 you force asterisk to say something first, which starts the flow of traffic, that opens the ports on the outbound call, now audio from the inbound call flows. Now asterisk has something to say on the outbound call, which opens ports on the outbound call and things begin to flow.

This is all true, but we open the AWS firewall (10k-20k/UDP), but “why still the problem” you may ask? Because asterisk doesn’t send “blank” packets; unless it has something to say, it won’t. Sending the silence/1 prompts the carrier to send something, then asterisk relays to the other side and now you have two-way audio again.

[root@vw-00301802ae39 ~]# nmcli c
NAME UUID TYPE DEVICE
Wired connection 1 a4a4a1d1-f5da-3f1e-b6b5-e7760cc48f1e ethernet enp1s0
Wired connection 2 710fe24e-cd72-3337-8d7c-cc7fe18d2283 ethernet ens1
[root@vw-00301802ae39 ~]#

[root@vw-00301802ae39 ~]# nmcli c modify 710fe24e-cd72-3337-8d7c-cc7fe18d2283 ipv4.method “disabled” ipv6.method “ignore”

run: nmcli c

[root@vw-003018087347 ~]# nmcli c
NAME UUID TYPE DEVICE
Wired connection 1 813d9fc5-af0a-3aac-af5b-6d2848048a81 ethernet enp1s0
Wired connection 2 7ac48477-b27a-3b1a-8c5e-d9c4621ead2f ethernet –
[root@vw-003018087347 ~]#

verify which wired connection needs to be blown out and run the following commands:

nmcli c modify ’Wired connection 2’ ipv4.method auto

nmcli c modify ’Wired connection 2’ ipv4.addresses ""

[root@vw-003018087347 ~]# nmcli c modify ’Wired connection 2’ ipv4.method auto
[root@vw-003018087347 ~]# nmcli c modify ’Wired connection 2’ ipv4.addresses ""
[root@vw-003018087347 ~]#

Some time a system will act “sluggish” or just generally slow or odd. In this case the below command will “Vacuum” (re-order / clean) the database.
NOTE: while this process runs (might take up to 30 minutes) the system is effectively locked and no new calls can start (existing calls will not end).

psql -Upostgres asgi_cc -c 'VACUUM FULL;'

This solution will work with 4.X, no changes.

Sometimes psip-pms can get stuck in sync mode.  During this time, you will not be able to modify the system and there appears to be no way to release it from this mode.

You can override it and return the system to normal operation by issuing the following command:

echo "test xml synch 0" > /var/lib/docker/volumes/voiceware_sockets/_data/psip-pms-cli-pipe

There should be no output and should return immediately. After issuing this command, the web interface should start working again.

At times the Psip process pmsping.php can become stuck. When this happens Psip stops processing check in/out messages however they continue to display in the Psip debug window. If this happens do the following.

as root, run:
ps auwwx|grep pmsping.php
and look for output like this:
 

 
Notice how it's been running since 2022?  That means it's stuck.  Another way to tell is to take the timestamp on the command (the very last parameter -- 1663136941 in this case) and convert that to a timestamp using https://www.epochconverter.com/

Open

In any event, if you see that process still running on the host it's probably safe to kill it.  Take the PID of that stuck process and issue a kill, as such:
sudo kill -9 10993
Replacing 10993 with the actual PID from the output above.  Within a few minutes the host should clear up and things should get back to normal in situations such as this.

If your psip-pms interface is not working correctly and you are seeing a similar message in your logs

11:16:02 1 RPRSP 308 XMLRPC client threw ERROR: Unable to transport XML to server and get XML response back. HTTP response code is 403, not 200

This means that your XML RPC url is set incorrectly (or to 'localhost').  On newer dual-stack system, 'localhost' resolves to '::1'.  Unfortunately, the ACLs system in Voiceware only compares IPv4 addresses (127.0.0.1).  That means you need to change the URL from 'localhost' to '127.0.0.1' to force IPv4 connection.

You can edit it in the file:

/data/psip/configs/stub/psip-pms.stub

Log files and the log rotate process was changed in Voiceware version 4. As such the below documentation will be split between versions 1-3 and 4.

Run away log files, common causes

1. Psip-pms log file being filled with garbage data when a Serial interface is denoted but does not exist.

2. Hacking bot scanners. If a system is left open on port 80 a hacking bot trying to download Polycom configuration files can find and start scanning a system. For every file scanned for a log entry will be created. Depending on the bot this can cause large log files.

Voiceware version 1-3

Log files are located in /var/log or /var/log/voiceware

Log files are rotated with the based on a varying schedule based on process. The log rotation is handled by logrotate. You can find the definitions in /etc/logrotate.d.

The most common reason for the logs to not properly rotate is due to the permissions of the logrotate definitions. To fix:

chown -R root:root /etc/logrotate.d

To diagnose a possible problem with logrotation, you can issue:

logrotate -d /etc/logrotate.conf

This will show what would be done, but does not execute any commands. Removing the -d would cause it to run, but I do not recommend doing this by hand as it changes the timestamps on the files managed by logrotate and can alter how rotations are handled.

Voiceware version 4

Almost all logging is handled via journald. This is the logging process that is part of the new SystemD project. All of the major Linux distributions use this new system. The binary log files are stored persistently in /var/log/ journal but those files are not text files you can read with an editor. They can be accessed using the journald command. These files are rotated as needed, but are limited by the amount of disk space available. The smaller or less free disk space, the shorter a period is covered by the running journal.

Persistent text logs written by rsyslogd are located in the /var/log/. These is rotated weekly and compressed to save space.

To follow the running journal:

journalctl -f

To limit it by Unit or Process (Note this starts from the beginning of the journal):

unit: journalctl -u <unit>

tag: journalctl -t <tag>

You can further limit this by time.

journalctl [-t <tag>] [-u <unit>] --since=<YYYY-MM-DD|'today'|'yesterday'>

If a systems CDR retention settings are changed from say 200 days to 100 days older logs might not be removed. To purge the older CDR records run the below command.

sudo docker exec -it scheduler /opt/asgi_scripts/recorded_calls.php

There will not be much output but it will remove all older CDR records and reduce the size of backups. (It would be advisable not to run this during the day on a busy system)

At times an ISP will block port 25. This port is the default port that Voiceware will use to send email messages. If the port is blocked the below commands can be ran to place a temporary fix into IPTables.

iptables -t nat -A OUTPUT -d 3.225.168.171 -p tcp --dport 25 -j DNAT --to-destination 3.225.168.171:587
iptables -t nat -A OUTPUT -d 107.21.150.142 -p tcp --dport 25 -j DNAT --to-destination 107.21.150.142:587
iptables -t nat -A OUTPUT -d 35.169.76.128 -p tcp --dport 25 -j DNAT --to-destination 35.169.76.128:587
iptables -t nat -A OUTPUT -d 34.204.28.179 -p tcp --dport 25 -j DNAT --to-destination 34.204.28.179:587
iptables -t nat -A OUTPUT -d 52.0.106.229 -p tcp --dport 25 -j DNAT --to-destination 52.0.106.229:587

Sometimes a system will fail to renew the certificate and will have the error “PR_END_OF_FILE_ERROR” when you try and browse to the site. The fix is to remove the webapps container and rebuild it then do the SSL cert installation process again from scratch.

docker rm -f webapps
voiceware start

voiceware ssl

If a DID is set as a user extension in the Dial Plan page when you load the Settings page you might receive the below error.

Fatal error: Uncaught PDOException: SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value "13039009102" is out of range for type integer in /usr/local/lib/php/PDOWrapper.class.php:355 Stack trace: #0 /usr/local/lib/php/PDOWrapper.class.php(355): PDOStatement->execute(NULL) #1 /var/www/html/phonesuite/settings.php(365): PDOWrapper->getAll('SELECT extensio...') #2 {main} thrown in /usr/local/lib/php/PDOWrapper.class.php on line 355

To fix this issue simply set the dial plan type to DID (from User) and the issue will be resolved. The issue will also be fixed in an upcoming version of Voiceware.

A bug in Voiceware that would cause Voiceware to fail to start processes (psip in observed cases) and would not allow you to restart Voiceware. This bug has been fixed. This bug only effected version 4.0.3.5.

Details

When attempting to run the Voiceware status command (or other Voiceware commands) the below would display.

A lock file already exists.
This means a process is already running
Check with other technicians and/or support personnel

Fix

Run the command rm /tmp/.voiceware* to remove the lock file and proceed. To permanently fix run the command yum install voiceware-osconfig and answer yes when asked if you want to download the files. No restart or other action is required.

Overview

Some times a SIP trunk provider will require a different registration timer period than the default in Voiceware. Below is how to change it.

1. Log into the system via SSH

2. Make yourself root sudo -i

3. Find the correct sip.conf file find / -name 'sip.conf'

4. The find command will return several lines, find the one with “merged” in the line, in the below screenshot its the middle line.

Open

5. Edit the sip.conf file using a text editor, in this example I use vi. vi /var/lib/docker/overlay2/429970c6d893c5b73d21fa80a8b6159b9b30d344ee3fdb6c0a3d46e1c8cabf20/merged/etc/asterisk/sip.conf

6. Go down the file to the about 6 pages to the “maxexpiry” line. Change both the maxexpiry and defaultexpiry to the registration time the SIP trunk provider requires (i.e. 3600). Once done save the file.

Before:

Open

After:

Open

7. Enter into the asterisk command line using the command asterisk -r

8. Issue the command sip reload

9. Exit asterisk using exit and test the SIP trunk.