My first Wireless Distribution System setup

The company I work for was asked to provide a Wifi solution for a little known hideaway place just under an hours drive away. The place in question is out near Sherwood Forest and has a reception which is basically a converted wood lodge and then a few wood lodges. That's really all there is to it!!

Anyway we were asked if it was possible to install an Access point on the reception and then install client devices inside the lodges themselves and housed away out of view. We were sceptical it would work and our scepticism proved to be correct! As I mentioned earlier it's at the edge of Sherwood forest which meant there was no shortage of trees around! At this time of year the trees were also in full bloom (which is good from a surveying point of view) and being Britain the trees were wet! Now Wifi uses RF and RF does not get on well with water!! So our initial survey doing it this way was a failure. We did hold the devices outside the cabin after confirming no signal inside the cabin and some that were close enough to the reception with a reasonable line of sight did manage to get one bar of signal strength but it was never going to work like this!

We suggested that as the park was going to expand that an ideal solution would be to build a lattice tower and mount an access point with a 360 degree high gain sector aerial that could be viewed from all the lodges (present and future). However customers being customers wanted it installing sooner which meant there was not enough time to build up the lattice tower. So it was decided that as a lot of the lodges did have line of sight to each other (including one near the reception that houses the Internet router) that we would mount Ubiquiti Nano locos on each lodge and set them up in Access Point WDS mode. Then internally we would have indoor TPlink access points that could all be mounted behind the TVs out of sight.

Before going to site I had a plan of the park and had to plan which Access Points would talk to which Access Point. I had 15 Ubiquiti Nano Locos to program so I drew on the map which loco would point where and how many MAC addresses would be needed for each Loco. For Access Point WDS mode on these devices you can list up to 6 MAC addresses of other devices that it will talk to so I had to plan this carefully.

The above diagram roughly shows how they're setup. As you can see this isn't ideal as some Locos are responsible for a lot of connections even if they aren't on their MAC list so a failure of one Loco could take down half the parks Internet access. But despite this it does work and works well.

Cabling worked very well as we used one power supply with a splitter that fed a PoE injector for the Nano Loco and another PoE injector for the internal access point. A small 1/2 meter Ethernet cable then joined each PoE injector via their respective LAN ports. All this is mounted inside the lodges behind the TVs so are out of sight and don't affect the internal aesthetics of the lodges.

Draytek Woes

For years I've used Draytek Vigor routers on and off and have loved them. User friendly and pretty much do as they're told! So when I started migrating customers over from private 4-wire circuits to VPNs, Draytek seemed the natural choice! The routers I chose had to support multiple VPN tunnels (well 2!) and be able to have 1 tunnel connected permanently with the second tunnel able to connect should the first tunnel fail! After looking at the models the Vigor 2930 and Vigor 3300 seemed ideal as they supported a feature called VPN trunking which did exactly what I wanted!

However initially setting this up was hard as the only documentation I could get hold of to set up these features were available on the US website! I only found that out by contacting UK support! NOT a good start! As we were bringing a few customers on board with this the idea was to use the more expensive (and much less user friendly) 3300 at our radio site which has a fibre-to-cab connection and the customers premises would use the 2930.

As we brought our second customer online (although first with these units), we setup them up with a VPN trunk which for a week or so seemed fine. But then they were reporting random loss of connectivity! The whole point of this setup was so they didn't have this problem! I checked the VPN uptimes and sure enough they were a lot lower than the WAN1 (ADSL) uptimes! Without actually needing to the VPN tunnel was failing over to the backup VPN tunnel and then going back to the primary VPN tunnel! This particular customer is literally down the road from the local BT exchange and had one of the best ADSL circuits I've seen! What's worse is that they had drop outs when the failover happened!

After talking with Draytek support and effectively getting nowhere I had to go back to conventional VPNs. I found that if I put the 'backup VPN' in VPN index 0 and the 'always on' primary VPN in index 1 then it seemed to work the way I wanted. However after a couple of months a new problem came to light! Each customer using this setup was using Voice Over IP units as a pair either side of the VPN to stream voice traffic. These units send a UDP stream to the other unit the other side of the VPN and vice versa. Simple and effective setup. However we were getting reports that the customer was getting one way audio traffic!! Further investigation showed that this only happened when the primary 'ADSL' VPN had dropped, failed over to the backup 'wireless' VPN and then returned to the primary VPN when the ADSL connection had been restored and stabilized! Rebooting the VOIP units didn't cure the problem but rebooting the routers did! Not good! Especially when the customer rings out of hours with this fault!!!

We didn't automatically blame the routers and set out to prove it. We replicated the customers setup in our workshop and replicated the failure straight away! Using Wireshark to monitor the ethernet packets going to and from the VOIP units we could see that they were sending their UDP streams to the router. But only one stream came out the other side! The manufacturer of the VOIP units also has software that replicates these units that can be ran on a PC (or anything else that runs Java). Again with a laptop each side of the VPN running this software and Wireshark we were able to prove that the stream was reaching one router but not coming out the other side until that router was rebooted! So there we had it, another Draytek issue!!

Around this time we had another customer come online and had no choice but to install this setup as at the time we had no alternative! We did get a firmware update (that Draytek said they had no intention of making it a readily available firmware upgrade!!!!) and it cured the fault! Just in time for our next customer to come on board!! Things seemed to be running smoothly except that our very first customer who was using a pair of Vigor 2820 routers also had this issue!! That's three different Vigor models all with the same fault!!! Draytek really dragged their heals in providing a firmware update for this model (in fact they didn't give us a firmware update).

We also noticed an issue with the 'fixed firmware' they sent us. The VPN tunnel wouldn't stay up for more than 7 hours without failing over for no reason at all (yup back to square 1 AGAIN!!). I logged on and actually watched the primary and secondary VPN tunnels 'fight' for control for a good 5 minutes before the primary tunnel won! This happened every 7 hours!!

All this happened in just under a year and to cut an already long waffle of a story short Draytek admitted they didn't have the resources to dedicate to fixing the issues and offered us a refund for all the equipment! My MD is also pushing for compensation for the shear man hours lost to this whole episode!!

We have now sucessfully removed all the Draytek routers and replaced them with Mikrotik routers. We can monitor these better and program them to do EXACTLY what we want. Plus since installing them we haven't as of yet had a single call about a fault which they'd caused! I'll write more about these another time :-)

Migrating a private 4 wire circuit to a VPN over ADSL

The company I work for is traditionally a radio communications company. We do so much more nowadays but radio communications is core to the business. As such we look after a lot of taxi companies. Most of the taxi companies we look after had several private 4 wire circuits between their premises and our aerial mast site. A private 4 wire circuit is similar to a telephone line except that all 4 wires are used and they are a dedicated line between two sites. As such they're maintained by either BT or Virgin Media and are expensive.

Each taxi company has a minimum of 2 radio channels (1 has 4). One channel is dedicated for voice and the other(s) for data. Diagram below shows the basic setup for one channel whether it be voice or data:

As already mentioned a dedicated 4 wire private circuit is expensive and most of our customers had at least 2! We came up with a more cost effective and modern solution that also included a backup! So how would we achieve this? ADSL and Wifi! The 4 wire circuit was replaced with a dual WAN router which would have a VPN tunnel configured per WAN port to another router at the aerial mast site. A single VPN connection could of course service more than one radio channel therefore saving the customer £1000s! To get the voice channel across the VPN we used identical VOIP units at either end. They convert line level audio to IP (the main advantage of this is there's no loss and you can literally stream anywhere in the world!) and stream to an identical unit which converts it back to line level audio.

As for the data channels, the operators enter jobs into a PC which then transmits it to an IP-to-serial unit. This then converts the serial data to audio using a modem and then connects to the radio base station. All this equipment is at the aerial mast site so the dispatcher PCs connect to the IP-to-serial unit over the VPN.

As specified earlier the primary VPN connection routes via ADSL. As ADSL fails now and again we had to come up with a backup plan. So we utilised another companies wireless network (whom we have an excellent working relationship with) and put up a wifi connection either by direct line of site to the aerial mast or via other nearby wifi devices which will then route to the aerial mast!

Below is a diagram of the improved setup.

Initially we used Draytek Vigor modems and routers which worked well to begin with but then we had no end of problems which were all VPN related! I'll go through all that in another post. We now use Mikrotik routers which are simply excellent and do exactly what they're programmed to do! The customer has saved £1000s a we have can monitor all IP equipment effectively and know of a fault before the customer does which is how it should be!

A Volume Shadow Copy Service operation failed.Unknown error (0x800423f0)

Having backup issues with Small Business Server 2011? I was and was getting the exact error message in the title! After some digging around it appears the error was down to Sharepoint 2010 SP1!! If left not only do you not have an up-to-date backup but you run the risk of filling up your hard drive with Exchange transaction logs!! Rather than go through the steps needed to cure this individually; here is a link to the official SBS blog that details the fix. It's actually not that long winded!!

Anyway, good luck!!

http://blogs.technet.com/b/sbs/archive/2011/07/06/potential-issues-after-installing-sharepoint-foundation-2010-sp1.aspx

My first 802.1Q VLAN

The other day I had to create my first 802.1Q VLAN. Why? To stop 2 customers potentially seeing each other's networks!! At our radio mast site we have a Draytek Vigor 3300 router which our customers connect to via IPSEC VPNs. We have a VPN tunnel that we use ourselves to link our Nottingham radio trunk with our Mansfield radio triunk and we share the local IP subnet (192.168.1.x) with one customer which is fine.

However we don't want our next customer to also be on that subnet. This is where the VLAN comes into play! On the Draytek 3300, go to Advanced and then LAN VLAN setting. Select 802.1Q VLAN as in the picture below:

Make sure that all ports are ticked as active (you could lock yourself out otherwise!!!) and make sure all frames are 'untagged'.  Equally important is to make sure 'enable management port for P4' is still ticked our again you could lock yourself out (not good if you already have customer's VPNs setup on there!!!).

You will then have to reboot the router as you've re-configured the LAN subnet for the router.
When the router has rebooted, click on Network and then LAN and you will now be presented with the following window:

You can now assign each of the 4 LAN ports their own unique subnet; therefore keeping them seperate from the other ports!

You may need to adjust any existing VPN settings to include the new subnet created on the other LAN ports.





I've whipped up a couple of Visio diagrams as a before and after which I hope better illustrates what I wanted to and have achieved.

Before:

After:

If at all possible it's worth forward planning with the subnets of the other LAN ports. As I said earlier, making a change to the local LAN address requires a reboot of the router (the more customers you have on the less desirable this is!!!).

Remote Work Webplace times out for SBS2011 with a Draytek Vigor2930

Recently I migrated our companies old & tired SBS2003 server over to a nice new SBS2011 server. All went well inlcuding migrating the old sharepoint database (had to use an interim Win 2003 server running Sharepoint 3 in a VM as a staging server) as well as Exchange and all the files. One big change we had to make was how we connect to the internet. The old server had two network cards in (1 for the LAN and the other for the WAN with a public IP that connected directly to a cable modem). We also used a POP connector to collect emails.

As SBS2011 only supports one network card I decided to install a Draytek Vigor 2930 router that connects to the cable modem for internet access. Whislt I was at it I changed our emails over from using a POP connector to using direct SMTP. All very straight forward stuff (or so I thought!!). On the router I forwarded the following ports to our new server:
25 - for emails
80 - for remote web access
443 - for remote web access
987 - for remote web access
3389 - for remote web access and access to the server

I then proceeded to test Remote Web Access from a computer outside our network and it timed out! Internally it worked fine but externally it didn't. My first thought was that it was the router so I double checked all the port settings and ran a GRC test as well as telnetting to each port from an external PC but it seemed fine. I also enabled UPNP on the router and ran the SBS connect wizard which detected UPNP was on and was happy with it but still nothing. To cut an already long story short I logged the case with Microsoft and gave their technician remote access to the server. He couldn't see that I did anything wrong so asked me to download a packet tracing program and to install it on both the server and the external PC. I did this and sent the traces to him. It looked as though the HTTPS requests weren't reaching the server.

This confirmed what I initially suspected about the router being at fault. To further prove this I gave the server the router's public IP address and plugged it straight into the cable modem. I ran my test from the external PC and Remote Web Workplace all worked fine!!

So back to the router. After digging around I found that someone else had had similar issues. Now at this point I hadn't enabled remote access to the router itself (this defaults to 443 for HTTPS but can be changed) but even though it wasn't configured it was interfering with HTTPS traffic and not forwarding the requests as it should do.

So to get around this I changed the HTTPS port number anyway to 4443:

I also read somewhere that SSL VPN defaults to 443. Again I wasn't using this but changed this anyway. On the router this can be found under: SSL VPN - General Setup & I changed the port to 8443:



I then rebooted the router and it works fine!! As a courtesy I sent my findings and solution to the Microsoft technician that was very helpful for which he was very grateful. So for what turned out to be a few hours of getting to the bottom of this problem; it could have been solved within 5 minutes by changing 2 simple settings!! Isn't that always the case though?

How to check that PTZ is working on a DVR/Codec that uses the Pelco D protocol

Sometimes if a PTZ CCTV camera doesn't PTZ (Pan, Tilt & Zoom) then it may not always be the camera that's at fault! It could be the device that controls it whether it be a DVR or a codec. To test this isn't as hard as it seems. You will need the following:

• Laptop with either a serial port or a USB-serial converter
• An RS-232 to RS-485 converter (preferably 4 wire for full duplex communication) I use a device that uses the USB port so it is bus powered: http://www.easysync-ltd.com/product/542/es-u-3001-m.html
• Realterm installed on the laptop (http://realterm.sourceforge.net/index.html#downloads_Download)
• You'll need to know the baud rate that the controller works at and set it in real term

Run Realterm on your laptop and setup the following:

1. On the Display tab (under Display as) select Hex (space)
2. Next to this on Data Frames change the Bytes to 7 and click single
3. Make sure to click Change which will save these settings
4. Now click on the Port tab and select the Com port you have your 232-485 converter on and select the correct Baud Rate (the cameras we install are usually on 2400 baud).

That's Realterm setup ready. Now connect the RS485 input of your converter to the output of your DVR/Codec (making sure that + goes to + and - to -). When finished if you logon to your DVR/Codec and select the camera who's PTZ output your testing and tell it to move the camera you should see some hexadecimal bytes appearing in Realterm.

If you don't see any data then check your Realterm settings (mainly com port and baud rate) and check your connection. Another even simpler test to see if the RS485 output of your DVR/Codec is working is to put an LED on the output (making sure + goes to the anode (longer leg) of the LED and - goes to the cathode). Make sure the LED is a simple red type and not blue! Blue LEDs require a larger forward voltage to turn them on which RS485 won't provide!!

When you do have Hex on your screen in Realterm you can now see if it's outputting what it should.

The first thing to look at is that each command starts with FF. This is fixed and cannot be changed so if this is wrong you've found your fault. The next byte is the camera number; again if this number is different (i.e. says 2 when you know the camera ID is 1) then again you've found a fault. They are the two main items I check when troubleshooting.

Pelco-D consists of 7 hexadecimal bytes (all byte data used in this page are in Hexadecimal format unless otherwise noted)
	Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7 Sync Camera Address Command 1 Command 2 Data 1 Data 2 Checksum
	Byte 1 (Sync) - the synchronization byte, fixed to FF Byte 2 (Camera Address) - logical address of the camera being controlled (Address 1 is 01) Byte 3 & 4 (Command 1 and 2) are shown below Byte 5 (Data 1) - pan speed, range from 00 (stop) to 3F (high speed) and FF for "turbo" speed (the maximum pan speed that the device can go) Byte 6 (Data 2) - tilt speed, range from 00 (stop) to 3F (maximum speed) Byte 7 (Checksum) - sum of bytes (excluding the synchronization byte), then modulo 100 (Decimal code: 256)
	Command 1 and 2 details
	Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1 Bit 0 Command 1 Sense Reserved Reserved Auto / Manual Scan Camera On/Off Iris Close Iris Open Focus Near Command 2 Focus Far Zoom Wide Zoom Tele Tilt Down Tilt Up Pan Left Pan Right Fixed to 0
	Example (Command 2):
	Pan Left - 0 0 0 0 0 1 0 0, which equals to 04 (both hexadecimal and decimal)
	Some other commands
	Command Byte 3 Byte 4 Byte 5 Byte 6 Go to Preset 00 07 00 01 to FF Set Zoom Speed 00 25 00 00 to 33 Set Focus Speed 00 27 00 00 to 33 Alarm Ack. 00 19 00 Alarm no.