Sunday, April 29, 2007

Accessing Exchange 2007 Unified Messaging: Troubleshooting

The most important thing to remember about troubleshooting this setup is to make sure you are only testing one part at a time. I learnt this the hard way! Don't bother trying to find out why you can't call the Exchange Server from Asterisk, if you aren't 100% sure that calls directly from the sipX server to the Exchange Server are working.

Packet Captures - Wireshark

The life saving tool for you is going to be Wireshark (available as a free download from http://wireshark.org/) which is a packet analyzer. Run it on your network to look at the SIP traffic going to and from each endpoint through both of the gateways. You can filter packets in Wireshark using the filter string sip.

When running wireshark, keep in mind that if you are on a switched network, you might only see packets going to and from the computer you are running it on. Using a hub instead is preferable, otherwise, you will need to run wireshark simultaneously on multiple PCs. If you are using VMWare, you can run wireshark in a Virtual Machine, and it will see the traffic going to all VMs on that are running on the host system (provided they are all operating in bridged mode).

Make sure you read my post on understanding the relationship between SIP and RTP. This will make troubleshooting packet captures a lot easier.

SipX Troubleshooting

Read my post on Understanding sipX dial plan configuration files. Make sure that you have confirmed that your dial plan has been merged successfully and ACTIVATED. The most common problems that people contact me about end up being caused by typos and other mistakes in the XML files. Check them all, then double check them, then active the dial plans again just to be sure.

SipX log files can be found in the folder /var/log/sipxpbx/ and the most helpful file here will be sipproxy.log. To get detailed information, you will need to change the logging level to debug from the sipX web interface.

To read the last 40 lines of these files from the command prompt, type

tail -40 /var/log/folder/filename

or to read the entire file, type

less /var/log/folder/filename

If sipX gives you some grief, you can always wipe the database. I had to do this a few times, when I made configuration changes that broke the system, but couldn't actually remember what I changed.

dropdb -U postgres SIPXCONFIG

If you need to restart the sipX services, you can type the following command

service sipxpbx restart

If you forget the superadmin password, you can reset it with the following command line

/usr/bin/sipxconfig.sh --database reset-superadmin

The sipX wiki is a good source of troubleshooting information. Specifically, the pages on configuring dial plans, and troubleshooting ConfigServer.

Asterisk Troubleshooting

Asterisk troubleshoot starts with the Asterisk log files located in /var/log/Asterisk/ with the most useful being the file called full. Use tail and less to view this log file as shown above.

Additionally, you can log onto the Asterisk console by typing

asterisk -vvvr

at the command prompt. Add more 'v's for more debugging information (verbosity).

The Asterisk support pages are a great source of information that I often use when troubleshooting or researching how to do new things with my set up. Voip-info.org has an Asterisk wiki that is also very helpful.

Troubleshooting the Microsoft Exchange Server

As always, the Exchange Server's event logs will contain valuable information in diagnosing problems. You can increase the level of logging by the Exchange UM services by running the following commands in the Exchange Management Console

set-eventloglevel -id "MSExchange Unified Messaging\UMClientAccess" -level expert

set-eventloglevel -id "MSExchange Unified Messaging\UMCore " -level expert

Check out the Technet library for more information on configuring and troubleshooting Exchange UM.

References

The following web sites were used as references for the information presented in this guide.


Feedback

If you have any feedback or comments about these instructions, please feel free to email me, ryan@lithiumblue.com, or leave a comment on this blog. If you have problems with your setup, and have gone through the troubleshooting steps above, email me with a description of your problem, and any relevant packet captures, etc.


39 comments:

Henrik Walther said...

Hi Ryan,

First of all I must they this is a great article series on how you get PBX/UM up and running in your home or lab environment - very good work :)

I've finally got my system up and running properly, but when I'm greeted by the UM Auto Attendant, the voice is very very sloppy. I can barely here what she's saying.

The Exchange 2007 server runs on pretty slick hardware, so I was wondering if you know if there's a setting on either the sipx or UM server than can be tweaked to eliminate this issue?

Ryan said...

Hi Henry,

If you have the UM role installed on the same server as the Client Access role, you can run into resource contention issues.
Basically, store.exe agressively caches data into RAM, and the seldom-used UM processes get paged out. When they need to be used, store.exe has to be paged out to disk, and the UM processes paged back in. This process takes some time, and results in timeout delays, and the broken voice you describe. I had the same problem in my lab setup.

Best practice is to seperate the roles onto different servers. If you can't do that, try reading this support article at http://support.microsoft.com/kb/815372, which talks about modifing the ESE buffer size.

Make sure you have plenty of RAM in the system, check the guidelines at http://www.microsoft.com/technet/prodtechnol/exchange/2007/plan/hardware.mspx

See how you go with that, let me know how you go ryan at lithiumblue dot com. It is unlikely to be anything to do with sipX, as the voice traffic is not sent through the sipX server at all. SIP (and sipX in this case) is only used to set up the communication session between two endpoints, in this case, your phone and the exchange server.

Leigh Honeywell said...

Hey Ryan,

Ethereal is deprecated due to some trademark crap. Have a look at http://wireshark.org .

Thanks for the great info on Exchange / Asterisk integration!

-Leigh

Ryan said...

Hi Leigh,

Thanks for that info, I hadn't realised it had been renamed. I have updated the posting with the new details.

Regards

Ryan

Sander de Rijk said...

Hi Ryan,

Thanks for your great article. I wouldn’t have managed to set it up without this great website. I would like to suggest 2 additions to your great guide. At the moment your guide does not allow the Exchange server to dial out while dialing out is indeed configured.

There are two parts missing to make Exchange a fully functional UM server with dialing out possibilities:
- Add dialing rule groups which add the leading extention for the outside line and add those groups to the UM policy created. That way exchange allows you to dial to the outside.
- Modify the sipX to forward calls with the leading 9 (in my case 0) to the asterix server with any number of digits behind it. I think I am doing something wrong here because the Asterix server is nog dialing out.

Unfortunatly I cannot dial an outside number from Exchange or even when I am connected to the sipX server using x-lite. I can however dial an outside number when I am connected to the Asterix server using x-lite. If you have any idea’s on how to fix this I would love to hear them.

Sander de Rijk said...

Hi Ryan,

Here is a readup how I enabled the Exchange server to dial out (thanks to your help through email)

To enable exchange dial-out I did a few things.
On the exchange server:
- Open the Exchange management console
- Go to Organization -> Unified Messaging
- Open the 3DigitDialPlan
- On the tab Dialing Rule Groups add a in-Country rule group
o Name: All National Calls
o Number mask 0xxxxxxxxx
o Dialed Number: 90xxxxxxxxx
o Comment: All national Calls
- On the tab Dial Codes fill in the codes that are specific for your country:
o Outgoing line access code: 9
o International Access Code: 00
o National number prefix: 0
o Country code: 31
- Click OK and go to the UM Mailbox Policies tab and open the 3DigitDialPlan Default Policy
- Go to tab Dialing restrictions, check both Allows and add the freshly created in-country rule group
- Click OK
On the sipx server:
- Go to system -> dial plans
- Open AsterixDialRule
- Add a prefix with the numbers 90 with any number of digits
- Apply the new dial plan (important)
On the asterix server
- Go to setup -> trunks
- Open the existing SIP/sipXOut trunk
- Scroll down to “Incoming Settings”
- User Context: sipXIn
- User details:
context=from-internal
host=(sipx server)
insecure=very
type=peer
- Submit changes and apply the changes
Testing and debugging:
To test the outgoing dial rules open OWA or Outlook and select “Play on phone” on a voicemail message. A dialogbox will popup showing your extention (400 probably). Just remove that number and fill in your mobile phone without the leading 9. It should convert the 0xxxxxxxxx to 90xxxxxxxxx by itself. And if all goes correctly your phone should be ringing.
If your phone does not ring there are several things that could be wrong. Exchange will not allow you dial out. In that case you will get a message in Outlook or OWA stating that Exchange does not allow you to dial out according to the policies. Please check your Exchange dial plans again and see if they are correct for your situation. In my situation this was created for the dutch number plan.
Also check the Exchange eventlog. If it says that the sipx server gave a timeout (error 408) then there is probably something wrong between your sipx server and asterix server. Then it is time you start wireshark to see which server gives what error codes

Ryan said...

Hi Sander,

Thanks very much for providing us with the details of your configuration. It will be appreciated by many people. I'll integrate it into the guide shortly.

Regards,

Ryan

Ulrik said...

Hi Ryan.
I have followed your great guide now, and doubled checked everything, and had almost given up, when suddenly I got connection to Exchange UM!! Everytime I tried I got time out, but after I tried at least hundred times (I think), I got through....
I can see in your guide that you are aware of the time out issue, but I didn't read it like the problem was that a big issue.
Have you found out anything more about this time out issue?

Ryan said...

Hi Ulrik,

There are a couple of reasons you can get a timeout error. The 'once in a blue moon' problem generally means that sipX is trying SIP/UDP before SIP/TCP. This results in call timeouts 99% of the time. Verify this behaviour using a wireshark capture. If you see your sipX server sending SIP/UDP invites to the Exchange server, then your mappingrules files are not configured properly.

Check your external_mappingrules.xml file, and ensure it has merged with mappingrules.xml. Validate your XML using the sipx-validate-xml command. See my post on understanding how these files work for a better understanding of the process here.

Pleae remember that you need to activate your dial plans in order for changes to dial rules to take effect. Its a simple thing, but you would not believe how many people email me and we later find out this is the problem.

Let me know how you go.

Ryan

Ulrik said...

Hi again Ryan.
Thanks for your quick answer :-)
but unfortunately it didn't help me much. I checked again my external files, everything is OK. I re-activated the dialplan - View XML shows "transport=tcp" so that should be OK. BUT validation of external_mappingrules.xml comes out with this error:

[root@sipx sipxpbx]# sipx-validate-xml /etc/sipxpbx/external_mappingrules.xml
==== /etc/sipxpbx/external_mappingrules.xml:

Error at file /etc/sipxpbx/external_mappingrules.xml, line 4, char 53
Message: Datatype error: Type:InvalidDatatypeValueException, Message:Value '${SIPXCHANGE_DOMAIN_NAME}' does not match regular expression facet '[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*(\.[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*)*(:[0-9]+)?'.

Error at file /etc/sipxpbx/external_mappingrules.xml, line 5, char 47
Message: Datatype error: Type:InvalidDatatypeValueException, Message:Value '${MY_FULL_HOSTNAME}' does not match regular expression facet '[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*(\.[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*)*(:[0-9]+)?'.

Error at file /etc/sipxpbx/external_mappingrules.xml, line 6, char 42
Message: Datatype error: Type:InvalidDatatypeValueException, Message:Value '${MY_HOSTNAME}' does not match regular expression facet '[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*(\.[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*)*(:[0-9]+)?'.

Error at file /etc/sipxpbx/external_mappingrules.xml, line 7, char 41
Message: Datatype error: Type:InvalidDatatypeValueException, Message:Value '${MY_IP_ADDR}' does not match regular expression facet '[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*(\.[a-zA-Z0-9]+(-[a-zA-Z0-9]+)*)*(:[0-9]+)?'.

Ryan said...

Hi Ulrik,

Does the transport=tcp and associated information appear in the mappingrules.xml file in /etc/sipxpbx/?

What does sipx-validate-xml /etc/sipxpbx/mappingrules.xml produce?

ryan

Ulrik said...

Hi Ryan.
Thanks for your persistency :-).

Yes, transport=tcp and associated information appear in the mappingrules.xml file in /etc/sipxpbx.

sipx-validate-xml /etc/sipxpbx/mappingrules.xml produces nothing (no output).

Ryan said...

Hi Ulrik,

Might be best to email me. Can you send me a packet capture of a failed call attempt using Wireshark?

Regards

Ryan

Anonymous said...

Thanks for the great tips. Everything works great except when trying to play on phone, which always fail. Asterisk reponds to call from Exchange through sipX with a "407 proxy authentication required" message which cause sipx to send a 408 timeout back to Exchange.
We've enabled Trixbox to allow anonymous calls and made the changes to our sip.conf file, but nada, still getting 407s.

Has anyone else experienced this and anyone know of a solution?

TYVM

Ryan said...

Hi,

Yeah, a 'known issue'. Theres a bug in asterisk somewhere thats causing it to ask for authentication even though the trunk is configured to accept unauthenticated calls. I am working on it at the moment, but its a tough nut to crack. If anyone has figured this out, let me know.

Ryan

Anonymous said...

Thanks Ryan, I'm glad to hear it is bug and not something we've done wrong.

Have you ever gotten this to work with Asterisk, or have you also always come face to face with the 407 error?

I can also confirm that if the phone is on SipX, Exchange is able to call it without issues.

There is one case when I can get Exchange to successfully ring an Asterisk extenstion, but this involves calling into Exchange from an Asterisk-connected phone and requesting to call a colleague at the office. I guess this scenario bypasses the auth check since the original source is Asterisk itself?

Cheers.

Ryan said...

Hi,

Yes phone calls outbound from OVA work fine. The structure of the call is slightly different to that of a "play on phone" call.

I have been experimenting the last few days, and notice that if you change the line "type=friend" under each extension definition in sip_additional.conf, to "type=peer", play on phone works. well for me at least. Try that and see if there are any side effects.

Digium state the following;

As of Asterisk 1.2, there is no reason to actually use 'user' entries
any more at all; you can use 'type=peer' for everything and the behavior
will be much more consistent.

All configuration options supported under 'type=user' are also
supported under 'type=peer'.

The difference between friend and peer is the same as defining _both_ a
user and peer, since that is what 'type=friend' does internally.

The only benefit of type=user is when you _want_ to match on username
regardless of IP the calls originate from. If the peer is registering to
you, you don't need it. If they are on a fixed IP, you don't need it.
'type=peer' is _never_ matched on username for incoming calls, only
matched on IP address/port number (unless you use insecure=port or higher).


So it shouldn't break anything. but i'm keen to hear on anyones experiences with this.

I do notice that once you do this, if you use FreePBX to modify the extension config, they get reset back to type=friend. I don't recomment combining the use of manual editing of these files with the use of FreePBX for configuration.

Ryan

Anonymous said...

Yep that did it, great find. I'll let it run like that for a while and post shoudl I encounter any problems.

Ryan said...

great stuff. let me know how it goes

ryan

Anonymous said...

If you are haiving an error where you are able to call 300, but not 222 or 299 and the X-Lite program is displaying the error
'Call failed: forbidden' when you call thoes extensions
then check the 'Application' event log on your Exchange server.
It should be logging a WARNING log with source 'MSExchange Unified messageing' everytime a call is rejected.

TO RESOLVE
- Open the log
- Find the line 'Cannon find a valid UM IPGateway for...'
- After for it may be displaying an IP address or FQDN. Write it down!
- Open Exchange Management
- Expand 'Org Config'
- Expand 'Unified Messageing'
- Click the 'UM IP Gateways' TAB
- Open the sipXServer Gateway
- Input the info you wrote down before.
- Press OK
- Dial 222 on the X-Lite

GOOD LUCK!

Jay
jay@blpromotions.com.au

JayS said...

Hopefully someone is still watching this thread!

Great article.

I'm having a problem with the Exchange UM Play on Phone option. For some reason if I specify '400' as the play on phone number or record greeting number the call doesn't come through to the SIP phone when logged in to Asterisk.

However, if I log into the SIPX server with '300' and use that for play on phone everything is fine.

The odd thing is that the Exchange server can send calls to the operator which is extension 400 on Asterisk. So there doesn't seem to be a general problem with it communicating that way.

Here is the Exchange event log entry:


Event Type: Warning
Event Source: MSExchange Unified Messaging
Event Category: UMCore
Event ID: 1087
Date: 11/16/2007
Time: 10:17:02 AM
User: N/A
Computer: E2K7
Description:
An outbound call to 400 could not be established. The selected outbound gateway "sipX.area51.com" returned the error: "408 Request timeout". The Caller ID used for this call was '400'. Contact the vendors who support your VoIP gateway and IP/PBX hardware for help troubleshooting the SIP response error code specified in the event description. You can also run diagnostic tests on your VoIP Gateway or IP/PBX hardware to make sure that the devices are operational. Additional information follows: A Sip request has failed. The current operation is 'Opening'. The session state is 'Connecting'. The remote participant is 'sip:400@sipX.area51.com;user=phone'. The response code was '408'. The response text was 'Request timeout'.

JayS said...

Just a note. Just noticed the posting regarding the issue I described.

Your solution of changing "type=friend" to "type=peer" in the sip_additional.conf didn't work as it was already set to peer by default.


[400]
type=peer
secret=11223344
record_out=Adhoc
record_in=Adhoc
qualify=yes
port=5060
nat=yes
mailbox=400@device
host=dynamic
dtmfmode=rfc2833
dial=SIP/400
context=from-internal
canreinvite=no
callerid=device <400>

[sipXtrunk]
type=peer
insecure=invite
host=sipx.area51.com
context=from-internal
canreinvite=no

Ryan Newington said...

Hi Jay,

Do you want to send me a wireshark trace of a failed call?

Ryan

Drago said...

First of all - your guide is excellent! Saves a lot of time and gives us great insides for how the integration actually works…

Will you please elaborate why we should use Asterix ->sipX –Exchange and not sipX ->Exchange directly? I kind of struggle to convince myself (although I know you can do it in just a few lines).

Thanks again for your hard work!

Drago (Georgia, US)

Ryan Newington said...

Hi Drago,

You can use sipx on its own, however, not all features may be available. Asterisk is the more widely used PBX, therefore the guide is really about getting Asterisk and Exchange to talk to each other. I have found that most people are looking to integrate Exchange with their existing Asterisk environment. Asterisk also allows easy integration with the PSTN, its a bit trickier in sipx.

So in summary, if you dont need or already have asterisk, you can use sipx!

Hope that helps

Ryan

Rolfkunst said...

Hi Ryan,

I installed Trixbox just as shown in the guide, every step worked fine, until I should go to the webbrowser and continue from there, the only thing I'm getting is the welcome page from Apache. Am I doing anything wrong?

Greetings Rolf (from Holland)

Anonymous said...

Awesome guide. Thank you!

Just wanted to ask you to comment on the fact that SIPX seems to be able to direct the RTP from one end straight to the other (peer to peer mode) while Asterisk seems to be designed to pass all traffic through the PBX (as old TDM PBXes were passing everything through their backplane).

The Asterisk approach seems to have the advantage of being able to convert codecs on the fly while the SIPX approach seems to have the advantage of reducing in half the bandwidth usage and increasing voice quality for cases where the PBX is in one place while both phones are at a different location.

For me this bandwidth issue was an important argument in choosing the solution and I suspect this was the main reason behind the questions you received about the SIPX OCS integration without Asterisk.

Hope to see you converted to Linux soon. Best regards!

Ryan Newington said...

Hi,

The behaviour you are describing is correct in that sipX is a SIP Proxy, while Asterisk is a Back to back user agent (b2bua). Asterisk will always function as a SIP Proxy provided that:

- No NAT is in use (both endpoints are on network segment that have a direct route between them)
- Both endpoints are capable of using the same codec
- CANREINVITE is not set to NO

Unfortunately, in order for communication with exchange 2007 UM to work correctly, CANREINVITE must be set to NO, as UM does not support this properly. Therefore, all the traffic is routed via the Asterisk box. I'm not sure of the situation with this and OCS, I suspect it may be more tolerent, but I haven't personally try it.

People seem to be having a bit more luck with OpenSER, rather than sipX.

Ryan

Harold Crane said...

Ryan,

This is a fantastic article. Thank you! Now here are two problems I hope you can help me with...

If I dial an extention from an internal SIP phone on Asterisk, everything works great. However I have two different SIP trunk providers and am having a problem with both of them.

The procedure is...
I call into the Trixbox PBX from the outside. (On one or the other provider.) I get the auto attendant and dial extension 111. User 111 does not answer and the call goes to voice mail.

On the first SIP trunk provider, the caller can hear the greating and all audio coming from Exchange, but Exchange gets no audio, either voice or DTMF.

When a call is placed through the other SIP trunk provider, the call goes to the same auto attendant and extension 111 is dialed but not answered. When the call goes to voice mail, the asterisk box receives back from sipX a 415 message...415 Unsupported media type.

Any idea on how to fix either of these issues?

Ryan Newington said...

Hi Harold,

Can you send me a wireshark capture of the failed calls.

ryan

CunningPike said...

We're trying this with SER and are seeing the Exchange UM server respond with "302 Moved Temporarily" no matter what we try - has anyone found a solution to that?

Ryan Newington said...

302 Moved Temporarily is normal behaviour!

It does to redirect the caller to a free UMWorkerProcess port.

Are the calls failing? What is the behaviour you are seeing?

ryan

Kumar P said...

I too am getting 302 errors. Asterisk goes into a loop after that. I mean it does local@default not found and gives up claiming that extension is not there. I believe 302 is not fully supported in asterisk. RFC suggests the the client connect back providing the new address present in the header but asterisk does nto follow. This is problem.

BTW, we are not using mediation server in between. We are directly trying to connect to speech server as thats the only thing we need.

Anyone in same shoes. Worst case, I will try specifying the ports manyally for each application which should do the trick for now but its pretty rogue approach.

Ryan Newington said...

Hi Kumar,

302 is not an error. It is a redirection. It is fully supported in Asterisk. There must be a problem elsewhere in your configuration. Check the steps in the guide once again and ensure you have followed the correctly.

You only need a mediation server when using OCS to connect to an endpoint that doesnt not support Microsoft's Real Time Audio protocol.

Anonymous said...

An outbound call to "905xxxxxxx" could not be established. The selected outbound IP gateway "192.168.1.152" returned this error: "404 Not Found". The caller ID for this call was "300". For help troubleshooting the SIP response error code that was specified in the event description, contact the vendors who support your IP gateway or IP PBX hardware. You can also run diagnostic tests on your IP gateway or IP PBX hardware to make sure that the devices are operating correctly. Additional information: "A SIP request has failed. The current operation is 'Opening'. The session state is 'Connecting'. The remote participant is 'sip:9905xxxxxxx@192.168.1.152;user=phone'. The response code was '404'. The response text was 'Not Found'."

Hi Ryan,

I'm struggling to make an outbound call from UM or Sipx, no luck so far.

I've also tried from Sipx using xlite, it's same 404 error.

But from Asterisk(trixbox) using X-lite, everything is OK, 299, 222 and even outbound calls and from PSTN, I can access exchange UM.

I think something wrong with SIPX probably dialing plan, but...

My issue is pretty similar to Sander's and tried his instruction, same.

Can someone help me on this please?

Charlie From Canada,

Ryan Newington said...

Hi Charlie,

I think your correct in assuming its a problem with your sipx dial plan. Do you want to send me your /etc/sipxpbx/external_mappingrules.xml file and I will take a look.

Ryan

Anonymous said...

Hi Ryan,

Thanks for your quick reply,
I'm just back from vacation, Looks like my coworker wiped all out.

Next time, i will try pbxnsip with Voip unless Asterisk with SIP/TCP support comes first.

Great website, great people.
Thanks again and I will be back.

Charlie

Anonymous said...

This is regarding the 415 Unsupported media type.

I recently had this problem:
Connect from X-lite to Asterisk, go to the auto attendant and call a number for which the user is not connected. The voicemail goes to Exchange nicely. I see sipX proposing G.711 PCMA and G.711 PCMU to Exchange.

To fix this, I changed the line
exten => s,n,Dial(SIP/222@sipx.lithnet.local|30)

show in this tutorial,l to

exten => s,n,Dial(SIP/222@sipXTrunk|30)

where sipXTrunk is the name of your sipX trunk. I am not sure exactly what happens but if you call with @FQDN and not @trunkName Asterisk does not seem to negotiate the codec properly.

Hope this helps,
Daniel

Oliver Green said...

Hi Ryan,

Superb comprehensive article set, with some tweaks I managed to get exactally what I wanted out of my setup.

I'm going to try and eliminate / consolidate the 'Exchange Proxy (sipX)' box and the trixbox boxes. I'll post how I get on.

Thanks for a brilliant post!